Milestone 3 of 8

Calculate per-label measures

Derive expected, predicted, and correct counts from each table and preserve every zero-denominator ratio as undefined.

Per-label measures are derived from the same count table. Keep the counts and the denominator beside every ratio.

Goal

Calculate expected, predicted, and correct counts for every configured label, then apply the supplied recall and precision formulas while preserving undefined ratios.

Inputs

Use the validated label order and the baseline and candidate confusion tables. For each label, read:

  • expected_count from its row total;
  • predicted_count from its column total; and
  • correct_count from its diagonal cell.

Calculate recall = correct_count / expected_count when the expected denominator is non-zero and precision = correct_count / predicted_count when the predicted denominator is non-zero.

Deliverables

Implement per-label calculation in src/counts.py and save output/per_label_results.csv. Include system name, label, all three counts, both ratios, and a clear representation of whether a ratio is defined.

When serializing JSON elsewhere, use null for an undefined ratio. In the reader-facing report or table, display it as not defined; never replace it with zero.

Checks

Check every label in configured order for both systems, including a label with no expected items and a label with no predicted items. Verify that counts match the confusion tables, defined ratios use the correct denominator, and undefined ratios are exactly null in JSON.

Use a hand-calculated label with a non-zero denominator and one with a zero denominator. Check that the function does not mutate the tables or remove a label merely because its ratio is undefined.

Workspace

Keep per-label calculation in src/counts.py and derive it only from the validated confusion tables. Write per_label_results.csv under output/ with stable columns and configured label order.

Hints

HintUndefined is not zero
No expected items means recall has no supplied denominator. No predicted items means precision has no supplied denominator.
HintKeep the source count
A ratio without its expected and predicted counts is difficult to audit. Save all three counts even when a ratio is null.

Review

Read one defined and one undefined result against the corresponding table row and column. Explain which denominator each ratio uses without making a claim about what the measure means in a later subject.

How to check your work

Checks compare the per-label schema and null handling with the supplied fixture. The supplied fixture preserves undefined values as evidence rather than hiding them.