# NULLSIGHT — DGA triage arena

*king of the hill · machine learning · 500 pts*

---

A mid-size enterprise resolver logged everything it was asked to look up over
24 hours. Somewhere in that noise is a botnet rotating through algorithmically
generated domains. Everything else is ordinary corporate traffic: CDN cache
keys, mail relays, Kubernetes service names, a lot of marketing microsites.

Build the classifier. Beat the reference detector.

## The arena

Register a team, start a **compute session**, upload `solution.py`. Your code
runs in a sandbox against the round's corpus and gets a public macro-F1 on the
leaderboard within a minute or so.

The corpus **rotates**. Every round is a fresh draw: different DGA family mix,
different base rate, shifted telemetry distributions, and — importantly — two
DGA families that appear in **no training split**, ever. Whatever you upload
has to train itself from scratch on data it has not seen and generalise to
families nobody showed it.

Two scores come out of every run:

| split | share | what it does |
|---|---|---|
| public | 30% | shown on the leaderboard, ranks the hill |
| hidden | 70% | never shown, decides the flag |

A reference detector is run against each round the moment it opens, and its
hidden-split score is published as **par**. Land within **0.030** of par on the
hidden split and you get the flag.

## The contract

Your file runs as `python solution.py` with:

- `/data/train.csv` — labelled, column `label` (1 = malicious)
- `/data/test.csv` — same columns, no `label`
- write `/out/predictions.csv` with exactly `id,label`

All three paths are also in `NULLSIGHT_TRAIN` / `NULLSIGHT_TEST` /
`NULLSIGHT_OUT`; read those and the same file runs locally and in the arena.

- numpy, pandas, scipy, scikit-learn, joblib. Nothing else.
- **No network.** The sandbox has no route out. Don't plan around downloading
  anything.
- 300 s wall clock, 2 CPU, 4 GB. The reference detector fits in about 20 s.
- 512 KiB of source, 40 runs per round, 60 s between them.

## Columns

| column | meaning |
|---|---|
| `id` | row key; your predictions join on this |
| `domain` | fully-qualified hostname as queried |
| `ttl` | TTL on the returned record, seconds |
| `n_resolved_ips` | distinct A records observed |
| `n_distinct_asns` | distinct ASNs behind those records |
| `nxdomain_ratio` | fraction of lookups that returned NXDOMAIN |
| `query_count_24h` | lookups in the window |
| `first_seen_days_ago` | age of the first observation |
| `n_unique_clients` | distinct internal clients that asked |

The telemetry has missing values and a slice of corrupted readings. Some
labels are simply wrong — the upstream sandbox that produced them is not an
oracle. Macro-F1 of 1.0 is not on the table for anyone, including the
reference.

## What's in the box

```
sample/train.csv        30k labelled rows from a sample round
sample/test.csv         20k unlabelled rows from the same round
sample/test_labels.csv  the answers, with the public/hidden split marked
solution_template.py    skeleton with the I/O contract wired up
baseline_solution.py    entropy threshold, scores ~0.40, included as a floor
score_local.py          the arena's scorer, so you can iterate offline
```

The sample round is generated from a published seed. It is a faithful sample
of the task and tells you nothing about the live rounds.

```bash
cd nullsight
NULLSIGHT_TRAIN=sample/train.csv NULLSIGHT_TEST=sample/test.csv \
NULLSIGHT_OUT=predictions.csv python baseline_solution.py
python score_local.py predictions.csv sample/test_labels.csv
```

## Rough guide to the terrain

- The entropy baseline in the box gets ~0.40. It is not a starting point so
  much as a demonstration of a specific mistake.
- Character n-grams over the raw hostname get you to roughly 0.89 and then
  flatten out, for two separate reasons. Both are visible in the data if you
  look at what it gets wrong.
- The gate sits above that. Getting there is not about a bigger model.

Scoring the arena itself is out of scope: the labels and the flag never enter
the sandbox your code runs in, and there is nothing there to find.
