what's Distil (SN97)?

Distil (SN97) is a Bittensor subnet designed for competitive knowledge distillation of large language models. Miners develop smaller student models, limited to 5.3B parameters, that aim to replicate the output probability distributions of a fixed teacher model, Qwen/Qwen3.5-35B-A3B (35B total parameters). Evaluations occur on 60 prompts from the ClimbMix-400B dataset, seeded by on-chain block hashes to ensure fairness and prevent overfitting. Scoring uses full-distribution KL divergence across the teacher's 248,044-token vocabulary, computed on 512-token continuations generated via vLLM-accelerated inference. The subnet employs a winner-take-all emission model where only the miner with the lowest KL divergence receives rewards. A king-of-the-hill dynamic requires new challengers to outperform the reigning top model by more than 1% relative improvement. Models are permanently bound to miner hotkeys, publicly hosted on HuggingFace with safetensors weights, and undergo pre-checks for architecture, tokenizer compatibility, quantization, and uniqueness via SHA256 hashes. This incentivizes genuine compression techniques like distillation, pruning, or architecture search while maintaining transparency and verifiability.

Distil (SN97) receives approximately 3.5% of daily Bittensor network emissions.