Skip to content
How much to read
Adwitiya Rahman
EmailResume

The lab

Two products, two tools, and a model doing most of the typing. Each rule here cost me a week, or a wrong number in front of someone who trusted it, and the projects are where you can check whether I held to them.

The instruments

One user each, and that user is me. The products are measured with them.

  • Wiki Pipeline

    Writes and checks the knowledge base behind this site every hour. One model drafts a page, a second with no tools grades it against the sources, and what passes is readable here.

    Read how it was built10 min
    Decision
    Let it publish without me. A process that needs me every day is one I abandon, and an abandoned process still looks like it is working.
    Refused
    No remote on the repository and no hosting for the vault, and no attempt to have it write anything a reader sees. I write those.
    The caveat
    One user, running hourly since 14 September 2026. Separately, 0 of 204 harvested items got past the redaction scan.
    Still open
    Whether a checker that only ever sees a page and its sources can catch the failure I actually fear, which is a fact that was true the day it was written and quietly stopped being true. Matching a page to its source proves it was copied faithfully, not that the source is still the world.
    What it proves
    • An agent pipeline built as two roles that do not report to each other: a writer that drafts, and a checker with no tools that is given the sources inline so it cannot go looking for confirmation.
    • Hidden thinking treated as a product constraint with a measured cost, after one uncapped checker spent about 21,000 tokens to make 3 edits and a zero-capped one approved a page in 4 seconds.
    • A desktop app that is read only except for one cell, because applying and being rejected are things a person does, not things an agent can observe.

    200 pagesrewritten from my repos nightly, checked by a second model

  • Semantic Search

    The bench the other projects get measured on. Hybrid retrieval over my own textbooks, every run benchmarked and comparable, so a ranking change can be checked instead of argued about.

    No demo for this one. It runs on my own hardware with my own books on it, and it is the bench I test the other two projects against rather than something built to be handed out.

    Read how it was built6 min
    Decision
    Measure first. The benchmark runner existed before a single retrieval setting had been tuned.
    Refused
    No sweeps across model combinations, no answer-quality scoring yet, and a locking issue filed rather than rushed into a change that had no room for it.
    The caveat
    The headline throughput fix went last on purpose, so the measurement could not flatter the fixes underneath it.
    Still open
    What good actually looks like. I can measure speed to the millisecond and answer quality not at all, which means every ranking decision I make here is currently unfalsifiable.
    What it proves
    • Building the measurement loop before tuning a single retrieval setting, so that every decision afterwards could be compared against something rather than argued about.
    • Root-causing a failure that presents as a bad model and is not: a 4,096-token context window quietly evicting the retrieved passages mid-generation.
    • Sequencing five concurrency defects by blast radius, deadlock first and the headline throughput fix last, so the measurement could not flatter the fixes underneath it.

    5 defectsfixed in blast-radius order

What the pipeline wrotewiki

I know when not to use a model.

Six rules, each with the project that shows it.

  1. Don't reach for a model where plain code is better.

    Plain counting beat my own tuned weights on a year it had never seen, so counting shipped. In Clarity, typed code computes every figure and the model never does arithmetic. EaseMySemester v1

  2. Work out how I'd know it worked, before building it.

    The benchmark runner existed before a single retrieval setting had been tuned. Semantic Search

  3. Never let the thing that built it be the thing that grades it.

    A second model, with no tools and the sources given inline, checks every page the first one writes. Wiki Pipeline

  4. Believe the person using it over my own reasoning.

    I retired Clarity's own headline metric after one conversation showed it was comparing two different jobs. Clarity

  5. Fix what breaks worst first, not what I found first.

    Five concurrency defects, fixed deadlock first, with the headline speed fix held to last so the measurement could not flatter it. Semantic Search

  6. Write down what I chose not to build.

    Every case study carries the list. EaseMySemester's: no public signup, no ingestion pipeline, no paid tier. EaseMySemester

How I actually build

The rule I would bring to a team before any of the rest of it: the thing that built it cannot be the thing that grades it.

I learned that three times on Clarity, each time expensively. A commit that made an action atomic was correctly tested for exactly that, while the real flaw was an authorisation gap nobody had thought to test for. A commit that preserved an ownership rule passed a single-threaded test while the actual bug was a race between two callers, which that kind of test cannot see. A planning document said the metrics dashboard was collecting real usage, and querying the database showed every event came from a demo account. Tests written by whoever wrote the code confirm their mental model. They cannot falsify it. More evidence does not fix that. Independence does.

So everything now runs as two roles that do not report to each other. I write the spec, long ones, the kind that say what happens when the file is malformed and what the thing must never do. A model builds against it. A second model, which never sees the first one's reasoning and has no tools to go looking for agreement, checks the result against acceptance criteria that were written before the build started. The knowledge base behind this site runs on exactly that shape, and so does the Clarity rebuild.

I stopped writing code full-time in 2022 and I am not going to pretend otherwise. The stack has moved a long way and I would be slow. It turns out the specifying and the reviewing were the parts I was actually good at. The importer that rejects a bad file with line numbers, the gate that will not let an unverifiable number reach the dentist using it: that is not clever code. Those are decisions someone had to make before any code existed, and hold on to while it got written.

The other half of the same rule is knowing when to stop. I once built a four session, four role agent workflow for this work, measured it, and found that in its last fifteen merges it had produced zero lines of source, and that five of seventeen pull requests contained no work at all, only messages between sessions. There was one developer on the project. I took it apart the same day and deleted 2,833 lines of planning documents down to one file of 215 lines. Process cost is a product cost.

What a purchase order teaches you

I sell an AI-led delivery portfolio to enterprises that have not bought from us before. It is the best available read on which AI capabilities someone will sign for.

Discovery before proposing
"I'm one hour from the exam" became the sentence EaseMySemester's whole navigation was built around, and the scope questions came before Semantic Search's benchmark runner existed.
An objection is a specification
"It shows the word null" and "I got an error screen" went in as specs, not complaints. A vague complaint about a jarring colour scheme only became fixable once I asked what he actually meant by it.
Numbers instead of adjectives
A feature ships with a measured rate attached. 20 questions recommended, 8 of them turned up in the paper.
A deadline you do not control
Exam season is the launch date. Everything gets prioritised by what helps the most people before the next one.
Closing
Every EaseMySemester signup came from asking a specific person, which is slow and is also the only reason I know why each of them stayed.

The clearest instance is the one that cost me an argument. While building EaseMySemester the plan was to paywall the question bank. I argued the paywall belonged in front of the predictions instead. The objection was that we were already showing the numbers behind those predictions, so nobody would pay for them. That had it backwards. The bank is raw material. The prediction is the ranking already done for you, and people pay for the work already done, not for the insight.

I did not win that argument on taste. I won it because I spend my week watching which capabilities enterprises put a signature under, and that is the kind of thing worth knowing before asking anyone to build.

What I'm building right now

  • EaseMySemester

    Closing the last of the declined answers. 82 of 809 questions still decline rather than answer, and nearly all of the remainder are multi-part questions where one sub-part has no source behind it. Retrieving per sub-part is the obvious fix and I have deliberately not done it yet.

  • Clarity

    A rebuild with a second user in it: the dentist's attendant, who books the schedule and must never see money. Her access has to die on the next request when it is revoked, and consent to message a patient is captured at that patient's next visit, so the list starts empty and fills one visit at a time.

  • Wiki Pipeline

    Teaching it to write this site's own numbers, so the page and the knowledge base cannot drift apart again. Every Sunday it also rereads one project in full, because the thing I cannot yet catch is a fact that was true when it was written and quietly stopped being true.

  • Semantic Search

    Scoring answer quality, not just speed. That is the thing I left out of v1, because I had no ground truth for it yet.

The working out

Usually the week a rule cost me something.

Everything I have written

Set the dial to 3 min for how the work gets built and what the sales seat changes, or Deep for what is unfinished.