Skip to content
How much to read

Knowledge base

projects / easemysemester / decisions

One Postgres store instead of Neon plus a SQLite VM

v2 design drops v1's split data layer, questions in Neon and the textbook corpus in a SQLite file on a VM, for one Neon Postgres database with pgvector.

decision · current · updated 2026-09-15 · EaseMySemester case study

See overview and v2.

Context

v1 split its data across two stores: the question bank in Neon Postgres, the entire textbook corpus in a SQLite file on a VM. The split produced two backup stories, two consistency models, and a book_id that exists in one store and not the other, which is why v1's production answers cannot tell its two Lippincott ingests apart today. That book was deliberately ingested twice under two different embedding models, a legitimate experiment, but each model got its own vector table while the keyword index stayed a single global index over every chunk, so the keyword arm double-counts it and sources.book_id is null on every production answer row, making the two ingests unrecoverable from the app database alone. Even the app-level question bank had already moved off SQLite by this point, an earlier port to Neon Postgres deployed on Vercel; the corpus VM was the one store left outside it.

Decision

v2 design consolidates onto one Neon Postgres database, using pgvector with an HNSW cosine index for chunk embeddings. The VM is dropped entirely. This was already specified in v1's own spec/AlphaPatNeonGenesis.md and never built; v2 design builds the thing v1 had already concluded was right.

Alternatives rejected

Keeping the SQLite/VM split and only fixing the missing book_id link was not pursued: it would have kept two backup stories and two consistency models for no benefit, and the VM is also the single point of failure that made the corpus unreachable during a prior outage.

Why

One store, one backup story, one consistency model. The schema is small enough to hold in your head, and raw pg queries over ranking and retrieval logic are queries an ORM would obstruct.

Consequences

Every chunk gets a stable book_id that travels with its citation, closing the gap that made v1's two Lippincott ingests indistinguishable. Re-embedding the whole corpus if the embedding provider changes costs about $0.09, a reversible decision recorded as one. Removing the VM removes the outage class it caused.

Status

Shipped. The corpus ingest into this schema completed 2026-09-07: nine book rows, 7,731 chunks, 3,955,261 tokens, for about $0.079 (numbers).

This page was written by a model from my own repositories and checked by a second one against the same sources. I did not write it, and no version of it was approved by me before it went up. How that works, and where it can still be wrong.