A count of passing checks is not evidence that software works. It records that one particular set of questions was asked and answered, and says nothing at all about the questions nobody asked. The second set is the one that decides whether you can depend on something, and only the project itself is in a position to describe it. So this page is written as a boundary rather than a claim: what is checked, what is deliberately not checked, and what can honestly be guaranteed as a result. No total number of checks appears below, for the same reason: a total is a fact about scale, and scale is not an argument.

What we test.

Six areas. Every row describes checks that exist and run today; nothing here is planned or aspirational.

AreaWhat is checked
Correctness
search results match expected values
Return values are pinned exactly rather than summarised: the order of documents, the score attached to each, the chunk ids of the passages that matched, and the seed ids handed to the second stage. A change that leaves the document set and its scores identical but alters which passages come back with them still fails. One check runs the server as a real process and speaks the protocol over standard input and output, because every other check calls the tools in-process and would never touch the transport. Analyzer properties are asserted from outside the implementation — upper case, full-width characters and extra whitespace must not change a result, and the index side and the query side must agree. Ranking is held to scoring axioms that need no labelled data: more occurrences rank higher with diminishing returns, rarer terms weigh more, unrelated text added to a document lowers its score, and a document that repeats the same passage over and over is not over-penalised for its length.
Storage integrity
writes, restarts, deletions
After every create, replace and delete, seven invariants are checked over the whole file at once — no chunk without its document, no reverse-index entry pointing at a missing record, no document listing a chunk that is gone, live index count equal to live chunks, no edge without its document, no search result naming a chunk that does not exist, and nothing derived outliving the document it came from. Separately: records written in bulk, closed, reopened and compared value by value; freed blocks reused instead of extending the file; an empty value, a very large value and an unusually long key round-tripped; commit, rollback, rollback on exception, refusal of a nested transaction, and rollback of an abandoned transaction at the next open. File size is checked as a growth curve rather than a threshold: multiplying the number of documents must multiply the file by roughly the same factor and nowhere near its square.
Failure handling
abnormal exits, damaged and hostile input
A child process is terminated without cleanup in the middle of a transaction. On reopen the committed document must be present, the half-written one absent, the unclean shutdown reported rather than swallowed, search must work again, and recovery must not discard the relation graph it was never asked to rebuild — while a clean shutdown must not trigger a rebuild at all. Deliberately damaged index postings, an older file-format version, a file left marked dirty, and an oversized length field must each fail loudly instead of returning a plausible wrong answer. Query-language metacharacters must not raise anything outside the defined error codes, % and _ must stay literal characters, and a malformed identifier must not be reported as a missing one.
Compatibility
what has actually been run, and where
Windows x64 only, and on one Python version: every check described on this page has been run on Python 3.14 (Windows 11 x64, MSVC) and on nothing else. The package declares a version floor and the suite asserts it, but asserting a floor is not the same as having run against it, and no older interpreter has been exercised. Database paths outside ASCII are exercised in Japanese, in Latin-1 and in mixed directory names, and the check is not that the file opened but that it exists under the name that was asked for, that search returns the document, and that the undo journal was created beside it. The built package is installed into a clean environment with the development tree hidden from the child process, and the server is started from the installed copy; a package missing its compiled library fails the build instead of shipping. The Windows and POSIX build scripts are read and compared source file by source file — a script nobody runs drifts from the one everybody runs.
Regression
conditions that once held keep holding
Each defect that has been found is converted into a check that fails on the old behaviour, and that check stays after the fix. Dependence on the embedded SQL engine that the storage layer replaced is held under a ratchet: per-module counts are frozen, the total is capped, a module not already on the list may not start using it, and the frozen table is itself compared against reality so it cannot quietly loosen. Because a source scan cannot see an indirect import, the shipped server is additionally imported in a fresh process and must not pull that engine in at all. The machine-readable capability list published on this site is rebuilt from the running tool definitions and the recorded implementation status, then compared with the copy that is live, so a stale claim fails instead of sitting there; a name unreachable from the tool interface may not be labelled as shipped.
Security boundary
stored data stays data
No network surface: server-side HTTP and event-stream entry points must be absent from the source, and socket binding is intercepted at runtime and must never be called while a server is constructed. No tool parameter may be named like a path, a file, a directory or a URL; the database location arrives through an environment variable and nowhere else. The dependency allowlist is enforced across all product source by syntax tree, so document and image parsers — the traditional home of file-format exploits — cannot enter through a transitive dependency. An injection payload stored, searched for and read back must come out byte-identical with the document count unchanged. Tool descriptions must contain no imperative phrasing, search results must be labelled as data, and the database file and every sidecar — including the undo journal, which holds the plaintext of whatever is being overwritten — have restricted permissions applied when they are created.

Checked against a second implementation

A test that only asks the code whether it agrees with itself will pass while both the writer and the reader of a format share the same misunderstanding. Four places therefore carry an independent reference, and the check is a comparison rather than an assertion.

The order of the steps is part of the check

Every check that measures a property of the file follows the same four steps, in this order.

Checks that are meant to fail

When a check fails because of a known defect that has not been fixed yet, it is left failing. It is not weakened, marked as expected, or deleted, because it is the only machine-readable record that the defect exists. That means a red check in this suite should be read as an open problem with a name, not as a broken test.

What we do not test.

The items below are gaps, not omissions from this page. Each one is something a reader might reasonably assume has been covered, and has not been.

Not coveredWhat that means here
Other operating systems and architectures Every check described here has only ever run on Windows x64. The storage engine has never been compiled or run on macOS, Linux or ARM. The POSIX build script is compared against the Windows one by reading both, which catches a missing source file but nothing that only appears when a compiler runs. Nothing is claimed for those platforms, and the package for them does not exist yet.
Other Python versions Everything on this page was run on Python 3.14. The declared version floor is asserted by the suite but has never been exercised on an older interpreter, so a claim of support for anything below the version actually run would be a claim about untested ground.
Long-running operation No database in any check lives longer than the check itself. Audit-log rotation and expiry are exercised by supplying dates, not by waiting for them, so nothing is known about a file that has been written to continuously for months, about fragmentation accumulated over a year, or about a process that stays open for weeks.
Concurrency at scale The file takes an exclusive lock, and what is checked is that a second process opening it is refused. Inside one process, a handful of simultaneous tool calls and simultaneous log-maintenance passes are covered. Beyond that there is nothing: no many-client load, no sustained contention, no measurement of what happens at the lock boundary under pressure. Concurrent writers sharing one audit file can lose lines.
Production-scale data The largest corpus ever indexed for a published measurement is 5,183 documents (BEIR SciFact test split, Windows x64, Python 3.14, measured 2026-08-27), and the storage growth curve was measured up to 2,000 documents (single bulk load, Windows x64, Python 3.14, measured 2026-08-27). Nothing is known about a hundred thousand documents, a million, or a file of tens of gigabytes, and a growth curve measured at small sizes is not evidence about them.
Hardware and filesystem faults An abnormal exit is simulated by killing the process, which exercises the undo journal. A disk that loses a write it has already acknowledged, a full volume, a failing device and a filesystem that reorders writes are not simulated. Randomised testing exists at the interface level with a fixed seed; there is no fuzzer that mutates the file format at random, and the corruption cases are specific, hand-constructed mutations.
Independent audit There has been no third-party security audit, no external certification, and no published penetration test. Everything described on this page was written by the same people who wrote the code. That is the ordinary situation for a project at this stage; it is stated here so that it is not mistaken for something else.
Paths that are not connected Vector search is not reachable from the tools in this build, so it is not tested end to end; the checks that exercised it against the previous storage layer were removed rather than left skipped, because a skipped check reports nothing while looking like coverage. Backup and dump are in the same position while they are rebuilt for the current file format.

What the current tests establish.

Everything below holds on Windows x64 with Python 3.14, which is the only configuration that has been run. Each one corresponds to checks described above.

Not guaranteed

How results are recorded.

A number without its conditions is not a result, so numbers are published only alongside what produced them: the dataset and its size, the parameters, the platform and the date. Benchmark output carries the environment — language version, engine version, index format, embeddings mode, concurrency mode, platform — in the result file itself, so a figure can always be traced back to the implementation that produced it, and checks enforce the discipline that would otherwise decay: a metric that was not measured is recorded as null with a note rather than as zero, a measured metric must be a positive number, a tail percentile may not fall below the median, and the counts reported for a dataset must be the counts actually loaded. Figures measured on the storage layer that has since been replaced are not reprinted as current, a measurement still running appears as in progress rather than as a provisional number, and a result that rests on one model or one synthetic corpus is published with that limit attached instead of generalised.

Published numberRecorded with it
nDCG@10 0.5800
BEIR SciFact
5,183 documents, 300 queries, test split, top_k=50, lexical search only with embeddings off. 0 of 300 queries returned nothing. Windows x64, Python 3.14. Measured 2026-08-27.
Search latency
current storage layer
Not published, and no figure is quoted here in its place. The figures that existed were measured on the storage layer that has since been replaced. Repeated runs of the identical build on a machine carrying unrelated load differed from each other by more than the effect anyone would want to read out of them, so a single median cannot carry a latency claim: a timing number needs a distribution over several runs on a quiet machine, not just a stated condition. Until that exists, an old number under a new name would be worse than none.