projects / easemysemester / decisions
One Postgres store instead of Neon plus a SQLite VM
v2 design drops v1's split data layer, questions in Neon and the textbook corpus in a SQLite file on a VM, for one Neon Postgres database with pgvector.
decision · current · updated 2026-09-15 · EaseMySemester case study
See overview and v2.
Context
v1 split its data across two stores: the question bank in Neon Postgres, the entire textbook
corpus in a SQLite file on a VM. The split produced two backup stories, two consistency
models, and a book_id that exists in one store and not the other, which is why v1's
production answers cannot tell its two Lippincott ingests apart today. That book was
deliberately ingested twice under two different embedding models, a legitimate experiment, but
each model got its own vector table while the keyword index stayed a single global index over
every chunk, so the keyword arm double-counts it and sources.book_id is null on every
production answer row, making the two ingests unrecoverable from the app database alone. Even
the app-level question bank had already moved off SQLite by this point, an earlier port to
Neon Postgres deployed on Vercel; the corpus VM was the one store left outside it.
Decision
v2 design consolidates onto one Neon Postgres database, using pgvector with an HNSW cosine
index for chunk embeddings. The VM is dropped entirely. This was already specified in v1's own
spec/AlphaPatNeonGenesis.md and never built; v2 design builds the thing v1 had already
concluded was right.
Alternatives rejected
Keeping the SQLite/VM split and only fixing the missing book_id link was not pursued: it
would have kept two backup stories and two consistency models for no benefit, and the VM is
also the single point of failure that made the corpus unreachable during a prior outage.
Why
One store, one backup story, one consistency model. The schema is small enough to hold in
your head, and raw pg queries over ranking and retrieval logic are queries an ORM would
obstruct.
Consequences
Every chunk gets a stable book_id that travels with its citation, closing the gap that
made v1's two Lippincott ingests indistinguishable. Re-embedding the whole corpus if the
embedding provider changes costs about $0.09, a reversible decision recorded as one. Removing
the VM removes the outage class it caused.
Status
Shipped. The corpus ingest into this schema completed 2026-09-07: nine book rows, 7,731 chunks, 3,955,261 tokens, for about $0.079 (numbers).