projects / semantic-search / decisions
Sequencing the concurrency fixes by blast radius, not discovery order
Five concurrency defects found in a deep review were fixed in an order chosen by dependency and failure severity, not by the order they were found.
decision · current · updated 2026-09-16 · Semantic Search case study
Context
A deep review of Semantic Search found five concurrency defects: a self-deadlock in the benchmark start path, settings type coercion with a torn write, an embedder lock that guarded the wrong call path, an ingest worker double-start with a lost wakeup, and blocking work running directly on the event loop.
Decision
Fix them in blast radius and dependency order: the self-deadlock first, because it was the only permanent, unrecoverable failure and it got strictly worse after a later fix; settings correctness and the embedder lock next, before letting requests run concurrently for real; the ingest worker fix independently, wherever it fit; and the event-loop fix last.
Alternatives rejected
Fixing in discovery order, or landing the event-loop fix (the one anyone would actually notice) earlier. Landed early, alone, it would have shown no measurable speed gain and could have read as a failed optimization, because the embedder lock still serialized work at that point.
Why
The deadlock was the only failure with no recovery path, so it had to go first. The event-loop fix's benefit was contingent on the earlier fixes; sequencing it last avoided a wrong reading of the result. Correctness (settings, embedder lock ownership) was hardened before concurrency was increased, so the increase wouldn't just surface more of the same bugs faster.
The event-loop fix itself is notable: the correct change is to remove an async keyword, not
to wrap the calls in a threadpool helper, because SQLite connections are bound to their
creating thread and separate threadpool calls can land on different workers.
Consequences
The project shipped its first test suite alongside the fix, and the author documented that one fix's throughput benefit is largely invisible in the default configuration, because reranking is on by default and the reranker lock still serializes. That honesty is recorded rather than claiming a win not yet earned; see known gaps.
Status
Done. Reranker lock narrowing (to unlock the real throughput benefit) is filed as a follow-up, not yet built.