# Kimi K3 Resources

## Knowledge

### Primary — K3 itself

- [Kimi K3: Open Frontier Intelligence — Kimi Team (arXiv:2607.24653)](https://arxiv.org/abs/2607.24653)
  The technical report, 27 Jul 2026. The source of record for every number in this course. Extracted to `sources/k3-report.txt` for offline grep. Use for: architecture equations (§2), scaling law and config table (§3.2, Table 1), post-training (§4), infrastructure (§5), benchmarks and cost (§6).
- [Kimi K3 launch post — Moonshot AI](https://www.kimi.com/blog/kimi-k3)
  Announcement with pricing, availability, case studies, and a candid limitations section. Use for: API pricing, deployment guidance, the three stated failure modes, what K3 is like to actually use.
- [Kimi K3 architecture notes — Sebastian Raschka](https://sebastianraschka.com/blog/2026/kimi-k3-architecture-notes.html)
  Independent read that places K3 against DeepSeek V4 and Nemotron 3. Use for: outside-view framing, the "all-NoPE is unusual" observation, cost figures for AttnRes.
- [Kimi-K3 weights — HuggingFace](https://huggingface.co/moonshotai/Kimi-K3)
  Released weights. Use for: config JSON to check any number in Table 1 against the shipped artefact.

### Ancestors — read these to understand K3's inheritance

- [Kimi Linear (arXiv, ref [63] in the report)](https://arxiv.org/abs/2510.26692)
  The 48B model K3 scales up. KDA and NoPE originate here; AttnRes has its own preprint (report ref [57]) though Raschka notes it also featured in Kimi Linear. Use for: the UT transform, the full chunkwise KDA derivation, ablations K3 doesn't repeat.
- [Kimi K2 (arXiv, ref [58])](https://arxiv.org/abs/2507.20534)
  The baseline every "2.5× better" claim is measured against. Use for: MLA-only architecture, MuonClip, the rephrasing data recipe.
- [DeepSeek-V2 (arXiv, ref [28])](https://arxiv.org/abs/2405.04434)
  Where Multi-head Latent Attention was introduced. Use for: understanding MLA before meeting Gated MLA.
- [DeepSeekMoE (arXiv, ref [23])](https://arxiv.org/abs/2401.06066)
  Shared + routed expert organisation, which K3's LatentMoE follows. Use for: MoE fundamentals.
- [Auxiliary-loss-free load balancing (arXiv, ref [30])](https://arxiv.org/abs/2408.15664)
  The bias-update method Quantile Balancing replaces. Use for: seeing exactly what problem QB fixes.

### Background — build these concepts as needed

- [Gated DeltaNet (arXiv, ref [138])](https://arxiv.org/abs/2412.06464)
  The delta-rule + gating recurrence KDA extends. Use for: linear attention as fast weight memory.
- [Muon optimizer — Keller Jordan (ref [53])](https://kellerjordan.github.io/posts/muon/)
  Newton–Schulz orthogonalisation of momentum. Use for: understanding what Per-Head Muon makes per-head.
- [Artificial Analysis](https://artificialanalysis.ai/)
  Third-party leaderboard the report cites for GDPval-AA and the Intelligence Index. Use for: checking whether K3's standings have drifted since July 2026.

## Wisdom (Communities)

- [r/LocalLLaMA](https://reddit.com/r/LocalLLaMA)
  Where open-weight releases get stress-tested within hours. Use for: real deployment reports on K3 — quantization quality, supernode requirements, whether the thinking-history sensitivity bites in practice.
- [EleutherAI Discord](https://discord.gg/eleutherai)
  High-signal research discussion, strong on architecture and scaling. Use for: asking whether an architectural claim actually holds, checking your reading of an equation.
- [Moonshot AI on HuggingFace — community tab](https://huggingface.co/moonshotai/Kimi-K3/discussions)
  Direct line to people running the weights and sometimes the authors. Use for: config-level questions and reproduction issues.

## Gaps

- No independent replication of the 2.5× scaling-efficiency claim exists yet. Treat it as the authors' measurement on their own held-out validation set, not a settled result.
- No third-party ablation isolating AttnRes from KDA at K3 scale. The 4% training / 2% inference cost figure is the report's own, as relayed by Raschka.
- The report defers the UT transform and full chunkwise derivation to Kimi Linear. If the math is ever needed, that's the only place it lives.