Skip to content

Inspect the saved experiment

Kind: tutorial.

This exercise recalculates the receipt pilot's scores without a GPU. Install Git and uv before you start. The initial clone and dependency installation need internet access. uv can install Python 3.12 if it is missing. After setup, the scoring command uses only the saved responses. It needs no model server, GPU or dataset image download.

  1. Clone the public repository.
git clone https://github.com/Alberto-Codes/inference-lab.git
cd inference-lab
  1. Install the locked client environment.
uv sync --locked
  1. Recalculate scores into a new file.
uv run --offline python scripts/score_receipts.py \
  --input evidence/cord-test-24 --output runs/inspection/scores.json

The command prints three summaries: warmup, measured concurrency one and measured concurrency four. The concurrency-one summary reports 212 numeric matches out of 241 annotated numeric strings. Its diagnostic ROUGE-L F1 is approximately 0.801.

  1. Open runs/inspection/scores.json.
  2. Find the measured concurrency-one record with ID 20.
  3. Compare its reference with plain_prediction and missing_numbers.

The reference contains 79,500. The prediction contains 79,300. The request completed, but it changed an amount.

The scorer refuses to replace an existing output file. Choose a different output filename when you repeat this exercise. The original evidence stays in evidence/cord-test-24/.

Read receipt results for limits and image-based observations. Continue with Evaluate receipts to submit new parser requests.