Milestone 6 of 8

Create fair numerical and visual summaries

Use the same label order and count scale across confusion plots, and keep paired and slice plots tied to saved numerical data.

Plots should make comparison easier without giving one system a different scale, order, or visual advantage.

Goal

Create shared-scale confusion heatmaps, a paired-outcome count plot, and slice plots with visible denominators. Check the underlying plot data before interpreting it.

Inputs

Use the confusion tables, per_label_results.csv, slice_results.csv, and the paired category counts. Keep the configured label order and use identical axis order and count scales for baseline and candidate heatmaps.

The paired plot shows the four categories in a stable order. Slice plots show group item counts or otherwise keep the denominator visible. Do not use a leaderboard, decorative gauge, or score card.

Deliverables

Implement plotting and plot-data preparation in src/report.py. Save figures under output/, including:

  • baseline and candidate confusion heatmaps with shared label order and count scale;
  • a paired-outcome count plot; and
  • slice plots that show group denominators.

Record the arrays, labels, counts, and groups used for each figure so an audit can reconstruct the visual data. Add two evidence-backed observations to the report without turning them into causal or significance claims.

Checks

Check that heatmap axes use expected labels as rows and predicted labels as columns, in the same order and scale for both systems. Check that every plotted count comes from its saved table, paired bars sum to item count, and slice plots retain denominators.

Read figure data back and use a deliberate label-order or count mutation to confirm that the audit catches an unfair or stale plot. A file-exists check is not enough to prove that a plot represents the saved evaluation.

Workspace

Keep plot-data and figure code in src/report.py. Read saved artifacts from output/ and leave the tables unchanged. Use the same output directory for all figures and do not create a dashboard-style summary.

Hints

HintShared scale means shared questions
If one heatmap uses a different color range, a visual difference may come from the scale rather than the counts.
HintPut denominators beside fractions
A slice bar without its item count can make a one-item group look as important as a larger group.

Review

Read the tables before the figures. Identify one visible count and its source cell, then state what the plot shows without claiming why it happened.

How to check your work

Checks compare axes, scales, category order, denominator treatment, and plot-data identity with the supplied fixture. The reference avoids decorative comparison treatment.

LLM PrimerCreate fair numerical and visual summarieshttps://llmprimer.com/python/projects/build-an-evaluation-report/create-fair-numerical-and-visual-summaries© 2026 LLM Primer