Milestone 2 of 8

Calculate overall and confusion counts

Build each expected-row, predicted-column count table and prove its total and marginal-count invariants.

Counts are the foundation for every later measure. Build a small table by hand before asking a program to build the complete one.

Goal

Calculate each system's overall summary and square expected-row/predicted-column confusion table in the configured label order.

Inputs

Use output/aligned_items.csv and the ordered labels from the validated configuration. For each item, increment the cell whose row is its expected label and whose column is its system's predicted label.

For each system calculate correct_count, total_count, and match_fraction = correct_count / total_count. The project calls the result a match fraction for these records; keep that name in saved evidence.

Deliverables

Implement the count construction in src/counts.py and save:

  • overall summaries for baseline and candidate;
  • output/baseline_confusion.csv; and
  • output/candidate_confusion.csv.

The CSV tables must show the configured label order on both axes. Record one small hand-built confusion table and its sums in the report or a stable evidence section.

Checks

For each system, check that the table total equals item count, every row sum equals the expected count for that label, and every column sum equals the predicted count for that label. Check that the diagonal total equals correct_count and that the match fraction uses the full item denominator.

Use a small public fixture whose row and column sums can be calculated by hand. Check that changing array orientation or label order is detected rather than silently accepted. Do not infer axis meaning from an implementation's matrix layout alone.

Workspace

Keep table and overall-count logic in src/counts.py. Read aligned records from src/data.py and preserve the configured label order in both output tables. Do not calculate per-label ratios until the table invariants pass.

Hints

HintWrite the axes down
Expected labels are rows. Predicted labels are columns. Write that sentence beside the table before filling a cell.
HintThe diagonal has one job
A diagonal cell counts an item whose expected and predicted labels agree. Use the diagonal to check correct_count, not to replace the table.

Review

Read one cell and trace the item IDs that contributed to it. Explain how the total, row sums, column sums, and diagonal provide different checks on the same aligned records.

How to check your work

Checks compare the tables, axis order, and invariants with the supplied fixture. The supplied fixture keeps table meaning explicit.