Clarity

Where this is: v1 is what the clinic runs on. The correctness gate and the eval suite are shipped and running every night, and the narration itself is still being tuned, because I'm not happy with the writing yet. The rebuild described near the end of this page has its scaffold and its guard rails and nothing else, on purpose.

What it is

A dentist I know runs a small clinic. He ran it out of a spreadsheet for sixteen months. This replaced the spreadsheet.

897 transaction records came across in the migration, and it has grown since. I do not have an exact live count and will not quote one: the number I can stand behind is the one I measured on the day I moved the data. Entered between patients, on a phone about as often as a laptop. One clinic, built for one clinic, though I'm talking to a few other dentists.

There's an LLM feature: it explains why a month moved. That feature sits behind a plain function that checks it, and 100 test cases that run every night and fail the build if it lies.

The clinic's records live in their own tenant, and the model never sees them. It is handed the aggregated monthly figures, the same ones on the insights screen, and nothing with a patient's name on it. That is a privacy decision and a correctness one at the same time: it is also why every number in a generated sentence can be traced back to a row.

A wrong number is worse than no number

This is a ledger. The dentist makes decisions off it. If the summary says margin fell 12% and it actually fell 2%, that's not a bad user experience, that's a bad business decision with my name on it.

So the check isn't a score I keep an eye on. It's a gate. A plain function walks every number in the generated sentence, traces it back to the row it should have come from, and if any figure doesn't reconcile it throws the whole thing away and shows nothing. No model, however confident, gets a number past it.

The other half of that decision matters as much: I separated what blocks a release from what doesn't. The number check blocks. There's also a model scoring the tone of the writing, and that one never blocks. It just tells me whether the thing is getting better. A subjective score is the wrong instrument to hold a release hostage to.

Pass rate, p95 latency and cost per call all get tracked the same way, as things that can fail a build rather than charts I remember to look at.

The suite that caught me out

100 test cases across four kinds of failure, generated from a fixed seed so the suite is the same every run. It runs nightly and fails the build on any miss.

The first full run scored 86 out of 100, and the failures were real. The model was working out percentages itself instead of quoting the numbers I'd handed it. Prompt constraint, fixed, written up.

A suite that passes on day one has usually been written to pass. This one found something, which is the only reason I trust it.

I organised a whole phase around a number that was wrong

For a while, one figure ran this project: entry took about 32 seconds, against about 7 in Excel. It was at the top of my notes in bold. I wrote a design review called Thirty-Two Seconds.

Then I asked the dentist about it.

He said he likes the entry screen. He specifically likes the autofill. And his spreadsheet was faster because it never asked who the patient was. It was dropdowns and an amount box, doing strictly less work. I'd been comparing two different jobs and calling it a gap. Three measurements, on someone still learning the tool.

I dropped the metric and rebuilt the roadmap around what he actually said: too many tabs, "History" sounds like patient history and this isn't a patient records app, and he wants to keep notes between visits.

Not all of that phase was wasted. The search ranking fix was a genuine bug and the contrast problems were real and measured. But it was aimed by me reasoning about him rather than asking him, and one question settled it.

The habit I took from it: when a number seems to be deciding the project, check whether the person using it agrees before building anything on top of it.

The writing was good and it said nothing

The first version of this narrated the whole month in a paragraph. The nightly eval scores one axis for tone and another for usefulness, and the two disagreed: the tone scores were fine, the usefulness scores were not. Which is a polite way of saying it produced well-phrased filler. Fluent, accurate, and no reason to read it.

So the narration stopped being a summary and became a diagnosis. A typed engine now splits a move in revenue or margin into volume, price and mix effects that add up to exactly the change, names the treatments driving it, and decides the severity itself. The model's only job is to say that out loud in his language. It never picks the cause, the number, the severity, or where the link goes.

The version I like best is the one where it doesn't run at all. When the case is clear, the sentence comes from a template and no model is called. Only the genuinely nuanced months are worth paying for.

Making 200 model calls a day cost nothing

Nightly evals plus real usage is roughly 200 calls a day, and this has no budget.

Chasing quota errors turned up something I hadn't assumed: the limits were per model, not per account. 20 a day on one model where the account-level number suggested 500. That turns a hard ceiling into a routing problem. So there's a chain of eight models it degrades through, and the nightly run finishes on its own without me touching it and without paying for a tier.

One bug worth keeping: a retired model name returns a 404, and my retry logic didn't recognise 404 as something to retry, so the chain gave up on the first model instead of falling through. Errors now carry which model failed and whether the failure was a quota problem or something else. A fallback chain is only as good as its ability to tell why a call died.

Three things he asked for that I did not build

The tone scorer never gates a release. The PhonePe QR automation he mentioned in passing stays unbuilt, because he was clear it wasn't now. And the Ctrl-key shortcuts I shipped are coming back out, because he told me he doesn't use them and expects Tab and Enter. Though not by turning the amount buttons into plain tab stops, because reaching the fifth one would then cost five presses, which is worse than what it replaced.

Serving the model myself

Two questions are still open on the narration: whether I can run the model on my own server instead of calling a cloud API, and whether it needs to run at all when the month has barely moved. The first is exactly the question Semantic Search was built to answer, so I'm using it as the bench.

The second is a sensitivity threshold. A clinic's monthly numbers don't move much. If nothing has meaningfully changed since the last summary, there's no reason to generate a new one. The cheapest call is the one you don't make.

The rebuild, and the user who must never see money

v1 has one user. The rebuild has two, and the second one is the whole design problem.

The dentist has an attendant who books the phone schedule. She needs the diary and enough context to tell a patient what they are coming in for. She must never see a rupee figure. That is not a preference, it is the condition the practice works under, and a product that gets it slightly wrong is worse than no product.

So it is enforced twice: once in the database, where the rows simply are not readable from her session, and once in a single authorisation module with a test named after the rule. Two enforcement points and one place to read the rule. I would rather have the duplication in the mechanism than have the rule live in four handlers where a fifth one will eventually forget it.

Her access model has its own shape, because it runs on her personal phone. Revoking her access has to work on the very next request, not whenever a token happens to expire. Sessions stay long, because a scheduler being logged out mid call is its own failure. No patient data is cached on the device. And there is no tap to call on her screen by default, because a phone number rendered as a button is a phone number that gets pressed by accident.

Consent starts at zero and fills one visit at a time

The clinic wants to message patients about recalls. The obvious build is to take the existing list and message it.

The list is sixteen months of people who never agreed to be messaged. So consent is captured at the patient's next visit, in the room, as part of the normal entry flow. The messageable population starts at zero and fills one appointment at a time, which means the feature is close to useless in month one and correct forever afterwards.

That is the same call as the narration gate. The version of a feature that is allowed to be wrong is not a faster version of the feature. It is a different feature.

Moving the guarantees into the database

The other change is where the rules live. In v1 they were in application code, which is to say they held for as long as every code path remembered them.

In the rebuild, double booking is impossible at the schema level rather than prevented by a check before the insert. Migrations are applied by CI with a drift check against the live schema, so what I think is deployed and what is deployed cannot quietly diverge. Scheduled jobs run inside the database and record every run, because a job that fails silently is indistinguishable from a job that had nothing to do.

Foundation ships before anything the clinic can see: sign in, the role rules, migrations, and a restore actually performed from a backup rather than a backup that exists. No clinic functionality gets built until that ground is under it. The cutover has to pass a rupee-level monthly comparison against v1's own figures before it is allowed to be the real system.

Right now that is all it is. A scaffold, guard rails that refuse a push to the main branch, CI running lint, test and build on every branch, and an app that renders a placeholder proving it can call its own typed API. No database yet, no sign in, no clinic. v1 is still what the practice runs on, and it will stay that way until the comparison passes.

The mistake I would not repeat

I'd have asked the question that killed the 32-second metric on day one. It cost a phase of work, and the conversation that resolved it took about a sentence. Everything I built in that phase was competent and pointed at the wrong thing, which is more expensive than building nothing.

I'd also make the demo and production tenants impossible to confuse. I've already closed a month against demo data once. That's not a discipline problem, it's a design problem, and I should fix it in the interface rather than by being careful.