Lesson 12 · Post-training and limitations

Last piece of new material. The architecture explains what K3 can compute; this explains what it actually does, and the three caveats that matter if you deploy it.

Three stages

    ① SFT                      cold start
      synthesised agent trajectories from earlier Kimi models,
      multi-stage verified, human-in-the-loop annotated,
      serialised in an XTML-based chat template

                    ▼

    ② RL                       specialise
      3 domains  ×  3 effort levels  =  9 expert models

      general tasks      ┐
      general agents     ├──  each trained at low / high / max
      coding agents      ┘

                    ▼

    ③ MOPD                     consolidate
      Multi-Teacher On-Policy Distillation:
      all nine experts distilled into ONE model that keeps
      each specialisation at each effort level

The post-training pipeline (§4.1). The report's phrasing for why RL is scoped this way: rather than training specialized RL models for individual tasks, it covers three broad domains each spanning a wide spectrum of sub-tasks.¹

SFT (supervised fine-tuning): Training on curated examples of desired behaviour. Establishes a starting policy good enough for RL to improve on.
Reinforcement learning, here: Letting the model attempt tasks in real environments and reinforcing what verifiably worked. This is where long-horizon agentic behaviour comes from.
Reasoning effort: An inference-time setting controlling how much computation K3 spends thinking. Trained explicitly at low, high, and max rather than emerging by accident.
Distillation: Training one model to reproduce another's outputs. Here it merges nine specialists into a single deployable model.

The choice worth noticing is that effort level is a trained axis, not a knob bolted on afterwards. Nine experts means each domain got its own model at each of low, high, and max. When you set effort on the API, you are selecting behaviour that was optimised at that budget.

How nine models become one

MOPD is conceptually simple once you see the reward. For each token the student generates, compare the probability the appropriate teacher would have assigned it against the student's own. The log-ratio is the reward, clipped to keep extremes from destabilising training.¹

    MULTI-TEACHER ON-POLICY DISTILLATION

    sample a domain d and an effort level e
              │
              ▼
    pick the matching teacher from the nine
              │
              ▼
    student generates a token  yₜ
              │
              ▼
    reward = clip( log [ π_teacher(yₜ) / π_student(yₜ) ] , ±R_max )
                        └── dense: one signal per token, not per task ──┘

    ON-POLICY means the student learns from its OWN outputs,
    so it improves where it actually goes wrong

§4.1.3, eq. 15. A dense per-token reward integrates into the existing RL framework, which the report notes naturally enabl[es] infrastructure-level optimizations such as partial rollout training for long-horizon tasks.

One negative result reported in passing, and negative results are worth collecting: they also tried more fine-grained top-k distillation objectives and observed no clear advantage in either convergence speed or final performance. The simple log-ratio was enough.

What the RL environments teach

The environments are where K3's agentic behaviour is actually built. The report describes training a general loop of reasoning, acting, observing, verifying, and adapting, often over hundreds or thousands of tool calls and millions of accumulated context tokens.¹

Two infrastructure details make that loop trainable, and both are worth knowing because they explain what K3 is good at.

Resumable microVM sandboxes. A trajectory spanning thousands of tool calls cannot restart from scratch when something interrupts it. The RL system keeps persistent rollout and environment state, with partial rollouts and external KV-cache retention.
Verifiable environments. Tasks are constructed so success can be checked automatically, including knowledge-graph-guided task synthesis and a unified white-box RL environment. Without automatic verification, RL at this volume is not possible.

The quantization detail that matters

QAT runs through the entire post-training stage, SFT and RL both. During RL, rollout and training share the same quantization scheme — eliminating the train–inference mismatch.¹ The model does its reinforcement learning at the precision it will be served at, so it never has to adapt to a format change at deployment. MoE expert weights go to MXFP4; attention projections, latent MoE projections, shared experts, and routers stay higher precision.

Three published limitations

The launch post includes a limitations section written against the team's own model. This is the most useful part of the release for anyone deciding whether to deploy, and the first one is a genuine operational trap.

    ① SENSITIVITY TO THINKING HISTORY                    ← the trap

      K3 was trained in preserved-thinking-history mode.
      If you do NOT pass its reasoning back on the next turn,
      or switch sessions mid-stream, quality
      "may become highly unstable."

      ▸ Practical: persist and replay the reasoning blocks.
        Dropping them to save tokens is the expensive kind
        of saving.


    ② EXCESSIVE PROACTIVENESS

      May "make unexpected decisions on the user's behalf."

      ▸ Practical: constrain scope explicitly in the system
        prompt or AGENTS.md. Assume it will act unless told
        where the boundaries are.


    ③ USER-EXPERIENCE GAP

      A "noticeable gap in user experience" versus Claude
      Fable 5 and GPT-5.6 Sol, beyond what benchmarks capture.

      ▸ Practical: benchmark scores will overstate how it
        feels to use. Trial it on your own work.

From the launch post.² Publishing these costs the team something and is a mark of a credible release. Take them at face value.

Limitation ① deserves emphasis

Most API integrations strip reasoning blocks between turns, because they are large and look like scratch work. For K3 that is a correctness issue, not an optimisation. If you evaluate K3 and find it erratic on multi-turn tasks, check whether your harness is preserving thinking history before concluding anything about the model.

Judging K3

Your mission included being able to make a grounded call on where K3 fits. Here is the summary the evidence supports.

Drawn from §6 and §6.4, with the report's own caveats attached.¹
Question	What the evidence says
Is it the best model?	No. It trails Claude Fable 5 and GPT-5.6 Sol overall, and the report says so in its own abstract.
Where does it lead?	BrowseComp (91.2), ProgramBench (77.8), SWE-Marathon (42.0), WebDev Arena (1,678 Elo, first of 99).
Is it the best open model?	Yes, on this suite, and by a clear margin over GLM-5.2.
Is it good value?	Consistently. 38% of Fable 5's cost on KCB 2.0; best BrowseComp score at half GPT-5.6 Sol's price.
What is it for?	Long-horizon agentic work, web development, deep research at long context, and anything where open weights matter.
What would rule it out?	Under 64 accelerators for self-hosting; a harness that discards reasoning between turns; tasks needing the last few points of raw capability.

Check yourself

Why did K3's post-training produce nine models before producing one?

Three domains crossed with three effort levels, then merged
Nine independent runs, averaged together to cut variance
Nine specialists are served as an ensemble at inference
Nine successive curriculum stages, each building on the last

General tasks, general agents, and coding agents, each trained at low, high, and max effort. Multi-Teacher On-Policy Distillation then merges all nine into one model that retains each specialisation at each effort level. Effort is a trained axis, not a knob added afterwards.

Your K3 integration gives erratic results on multi-turn agentic tasks. What do you check first?

Whether the harness passes K3's reasoning blocks back on each turn
Whether reasoning effort is set too low for the task's difficulty
Whether prefix-cache misses are corrupting the conversation state
Whether MXFP4 quantization is degrading quality on longer inputs

The first published limitation: K3 was trained in preserved-thinking-history mode, and quality "may become highly unstable" if reasoning is not passed back or sessions switch mid-stream. Most harnesses strip reasoning blocks by default, which makes this a correctness issue rather than an optimisation.

What is notable about how QAT was applied during K3's reinforcement learning?

Rollout and training shared a single quantization scheme
Quantization was deferred until after RL had finished
Every component of the model was quantized uniformly to MXFP4
Each effort level received its own quantization scheme

QAT covers the entire post-training stage, SFT and RL both, with rollout and training using the same scheme. The model learns at the precision it will be served at, so nothing shifts at deployment. This is also why SiTU-GLU's activation bound exists.

The course, complete

    ✅ L1   the shape of K3            three axes, config table
    ✅ L2   KDA part 1                 fade / erase / write
    ✅ L3   KDA part 2                 chunkwise, g_min, MLA, NoPE
    ✅ L4   Attention Residuals        depth as retrieval
    ✅ L5   Review A                   interleaved recall
    ✅ L6   LatentMoE                  sparsity 56, deriving 2.78T
    ✅ L7   SiTU-GLU                  bounded activations
    ✅ L7b  Quantile Balancing         even expert load
    ✅ L8   native vision              from-scratch MoonViT-V2
    ✅ L9   Per-Head Muon              optimiser + scaling law
    ✅ L10  Review B                   interleaved recall
    ✅ L11  serving                    cost, cache, quantization
    ✅ L12  post-training              RL, MOPD, limitations

    ▢  L13  Review C                  capstone

The whole course, one screen. Every lesson and the one thing it established. This is the checklist to test yourself against before the capstone.

One lesson left, and it is all retrieval. Review C asks you to reconstruct the whole architecture from a blank page, explain K3 to three different audiences, and read an unseen passage from the report cold. That last exercise is the real test of the mission. Remembering K3 was never the goal; reading the next frontier model report without help is.