Store the printed page on ingest, not derive it from an offset

See overview and citation provenance.

Context

The original schema (migration 0002) stored one page_offset per book and derived the printed page at render. The reasoning was that only one number then had to be right.

Decision

Reversed 2026-09-07 by issue #44, inserted ahead of the library ingest and blocking it: store the printed page, and where it was read from, on the chunk itself. The offset is kept only as a fallback for the two books that print no folio.

Alternatives rejected

Keeping the single-offset design and relying on a spot-check to catch drift was rejected once the ingest walked every page-cited PDF and found the offset was constant in only two of six page-cited books.

Why

Three separate mechanisms broke the constant-offset assumption, none catchable by a spot-check small enough to actually run: BD Chaurasia Vol 1 and Vol 2 each step by two at a section divider, Vol 3 and 4 is two books in one file with two independent page-1s, and Guyton loses one to three pages at roughly thirty chapter breaks and drifts 47 pages end to end.

Consequences

Chunks whose printed page was read directly off the page are correct by construction, so the post-ingest spot-check narrows to chunks whose page was derived: five per book, weighted toward Guyton (38 offset runs, 552 derived chunks) and toward Rafi, which is 100 percent derived on a single hand-verified offset and so is either entirely right or entirely wrong. The ingest also surfaced that BD Chaurasia Vol 3 & 4 needed a PDF page range per book rather than one offset, and that Netter needed two plate-label corrections after its matcher read plate numbers off the table of contents and collapsed the Electronic Bonus Plates onto the main series.

A folio is only trusted after it continues an already-accepted one for a minimum run of pages; a first pass trusted a run of two and produced false chains on the two books with no real folio, Rafi and diFiore, moving them off their hand-verified constant offsets (Rafi's entire 820 pages disagreed with its verified offset of 16). Raising the minimum run to five pages removed every false chain on both books while leaving the genuine section-divider and restart discontinuities in BD Chaurasia intact, verified against the real PDFs rather than synthetic fixtures.

A manifest row can now carry a PDF page range (pdfPageStart/pdfPageEnd), so a book that ships as one file with two independent page-1s, like BD Chaurasia Vol 3 & 4, resolves as two rows instead of one folio chain running across the boundary: bdc-vol3 and bdc-vol4, each numbering on its own terms. Front matter is dropped by resolved page, not by offset, reading the resolved number rather than recomputing it: a chunk whose printed page is below 1 is not written. After two plate-label corrections to a matcher that was reading plate numbers off the table of contents and collapsing the Bonus Plates onto the main series, Netter ingested clean over its plate range (pdf 42-825): 435 chunks, 222,604 tokens, every one carrying a label.

Status

Shipped as part of the M2 corpus ingest, completed 2026-09-07.