Milestone 7 of 7

Write the project report

Connect configuration, checks, training, generation, intervention, conclusions, limitations, artifacts, and rerun commands.

Turn the seven milestones into one reproducible argument. The report should let another reader recreate the run, find every raw artifact, and distinguish calculation, observation, intervention, interpretation, and limitation.

Milestone goal

Write report.md, run every command from a fresh process, and complete an artifact checklist. Do not paste only the best loss, sample, or intervention.

Use a fixed report structure

Question and scope — state the toy question and what is not tested.
Data — record vocabulary, corpus, sampling, split, and realized counts.
Model — include equations, dimensions, tied readout, and 368-parameter ledger.
Training contract — record seeds, initialization, optimizer, clipping, dtype, device, runtime versions, and commands.
Correctness checks — report shapes, target alignment, causal intervention, and parameter assertions.
Training evidence — include one-sequence diagnosis, complete train and validation traces, entropy references, and fixed-context probabilities.
Reproducibility — report checkpoint fields, inference reload, and exact next-step resume separately.
Generation — include full-versus-cached errors, cache shapes, position trace, policy settings, token probabilities, and stopping reasons.
Interpretation — state the question, trace fidelity, observations, intervention, controls, alternatives, and next test.
Conclusion and limits — answer the frozen question without expanding it into a claim about general LLMs.
Artifact index — link every source file, configuration, checkpoint, raw output, figure, and exact rerun command.

Preserve evidence at the right level

Use sentences such as:

Under a zero-write replacement, ablating layer 2 head 1 reduced the clean X-minus-Y margin from 6.4618 to -2.4767 for the prompt <bos> X Y.

This names the component, replacement, metric, input, and effect. It does not say the head “understands alternation,” is sufficient, or implements the only possible mechanism.

Similarly, write:

Validation loss approached the known conditional-entropy floor on the frozen toy distribution.

Do not replace it with “the model learned language.” The project did not test natural-language data or broad capabilities.

Include failures and decisions

Preserve failed checks and the repair that followed. A concise table is enough:

Milestone	First result	Decision	Change	Recheck

This makes the reasoning reproducible. Deleting failed runs can make a project look cleaner while making its conclusions less trustworthy.

Audit every number

For each table or figure, identify:

script and configuration that produced it;
whether values are exact, rounded, sampled, or fitted;
axes, units, reduction, and tolerance;
population, training-sample, validation-sample, or single-prompt scope;
excluded memory, compute, or system costs.

The report should never make a rounded displayed subtraction disagree with its reported result. Keep additional precision in raw output, then recompute tables from the values shown or say explicitly that unrounded values were used.

Question. Repair the final claim

A report says, “The model learned language because its training loss fell.” Which rewrite matches the toy evidence?

Choose one

The model learned predictive structure in the toy corpus; broader language ability was not tested.

The lower training loss proves that the model understands language.

The model will generalize to any language task because the optimizer converged.

Select one choice, then check.

Hint

The corpus contains symbolic toy sequences, not general language.

Solution

The model learned predictive structure in the toy corpus. The experiment did not test broader language ability.

Final reproducibility test

On a fresh checkout or clean directory:

create the documented environment;
run model checks;
run one-sequence and corpus training;
reload and resume the checkpoint;
run cached generation and deliberate failures;
run the interpretation trace and intervention;
regenerate every reported table from raw output.

If full retraining is intentionally omitted for time, say so and identify which stored checkpoint and raw output were verified instead. Do not imply a rerun that did not occur.

Completion checklist

Deliverable: the final report, artifact index, completed checklist, regenerated tables, and a short statement of remaining limitations.