RVL-CDIP-Classifier: appendix
Slice table · prompt registry · cost
Supporting tables for the RVL-CDIP case study. All accuracy figures use prompt v11.8 on qwen/qwen3.7-flash unless stated.
Accuracy by slice
| Slice size | Tuning set? | Exact-match accuracy |
|---|---|---|
| 160 | Yes, tuned against | 99.4% (157/158) |
| 320 | No; informed later rules | 87.2% (279/320) |
| 480 | No; superset of the 160 | 89.1% (424/476) |
| 800 | Not the tuning slice | 83.1% (665/800) |
| 1,120 balanced | Not the tuning slice; shares at least 320 images with the 800 | 82.6% (925/1,120) |
The tuning number is an upper bound, not a generalisation result. The 1,120 slice overlaps the 800 slice, so those two results are not independent.
Prompt history
| Version | Change | Result (tuning slice unless stated) |
|---|---|---|
| v0 | Short baseline | 66.1% on the 800-image control slice (529 of 800), not the tuning slice |
| v10 | Rules written for every v9 miss | +6.4 points over v9 |
| v11.8 | Form-versus-budget and memo-versus-letter fixes | 99.4% (157/158) |
| v16 | Two worked examples | 96.2% |
| v17.2 | Current code default | — |
| v19 | Vote preamble | 96.9% (155/160) |
The registry holds 29 versions including subversions such as v11.8 and v17.2; “v0 to v19” names the endpoints.
Cost
| Measure | Value |
|---|---|
| 1,120-image run, billed | $0.4937 |
| List-price projection | $0.6815 |
| Per image | About $0.00044 |
| v19 run, per image | $0.000377 |
About 61–63% of prompt tokens were cache hits, which explains the gap between billed and projected cost.
What the simulations did not reproduce
Voting, stronger-model routing and worked examples looked promising in simulation. On measured slices, routing did not improve accuracy and cost 7.5–7.8 times more; exemplars moved accuracy down. Treat the simulation table in the report as predictions with a separate measured column.