A count of passing checks is not evidence that software works. It records that one particular set of questions was asked and answered, and says nothing at all about the questions nobody asked. The second set is the one that decides whether you can depend on something, and only the project itself is in a position to describe it. So this page is written as a boundary rather than a claim: what is checked, what is deliberately not checked, and what can honestly be guaranteed as a result. No total number of checks appears below, for the same reason: a total is a fact about scale, and scale is not an argument.
What we test.
Six areas. Every row describes checks that exist and run today; nothing here is planned or aspirational.
| Area | What is checked |
|---|---|
| Correctness search results match expected values |
Return values are pinned exactly rather than summarised: the order of documents, the score attached to each, the chunk ids of the passages that matched, and the seed ids handed to the second stage. A change that leaves the document set and its scores identical but alters which passages come back with them still fails. One check runs the server as a real process and speaks the protocol over standard input and output, because every other check calls the tools in-process and would never touch the transport. Analyzer properties are asserted from outside the implementation — upper case, full-width characters and extra whitespace must not change a result, and the index side and the query side must agree. Ranking is held to scoring axioms that need no labelled data: more occurrences rank higher with diminishing returns, rarer terms weigh more, unrelated text added to a document lowers its score, and a document that repeats the same passage over and over is not over-penalised for its length. |
| Storage integrity writes, restarts, deletions |
After every create, replace and delete, seven invariants are checked over the whole file at once — no chunk without its document, no reverse-index entry pointing at a missing record, no document listing a chunk that is gone, live index count equal to live chunks, no edge without its document, no search result naming a chunk that does not exist, and nothing derived outliving the document it came from. Separately: records written in bulk, closed, reopened and compared value by value; freed blocks reused instead of extending the file; an empty value, a very large value and an unusually long key round-tripped; commit, rollback, rollback on exception, refusal of a nested transaction, and rollback of an abandoned transaction at the next open. File size is checked as a growth curve rather than a threshold: multiplying the number of documents must multiply the file by roughly the same factor and nowhere near its square. |
| Failure handling abnormal exits, damaged and hostile input |
A child process is terminated without cleanup in the middle of a transaction. On
reopen the committed document must be present, the half-written one absent, the
unclean shutdown reported rather than swallowed, search must work again, and recovery
must not discard the relation graph it was never asked to rebuild — while a
clean shutdown must not trigger a rebuild at all. Deliberately damaged index
postings, an older file-format version, a file left marked dirty, and an oversized
length field must each fail loudly instead of returning a plausible wrong answer.
Query-language metacharacters must not raise anything outside the defined error
codes, % and _ must stay literal characters, and a
malformed identifier must not be reported as a missing one. |
| Compatibility what has actually been run, and where |
Windows x64 only, and on one Python version: every check described on this page has been run on Python 3.14 (Windows 11 x64, MSVC) and on nothing else. The package declares a version floor and the suite asserts it, but asserting a floor is not the same as having run against it, and no older interpreter has been exercised. Database paths outside ASCII are exercised in Japanese, in Latin-1 and in mixed directory names, and the check is not that the file opened but that it exists under the name that was asked for, that search returns the document, and that the undo journal was created beside it. The built package is installed into a clean environment with the development tree hidden from the child process, and the server is started from the installed copy; a package missing its compiled library fails the build instead of shipping. The Windows and POSIX build scripts are read and compared source file by source file — a script nobody runs drifts from the one everybody runs. |
| Regression conditions that once held keep holding |
Each defect that has been found is converted into a check that fails on the old behaviour, and that check stays after the fix. Dependence on the embedded SQL engine that the storage layer replaced is held under a ratchet: per-module counts are frozen, the total is capped, a module not already on the list may not start using it, and the frozen table is itself compared against reality so it cannot quietly loosen. Because a source scan cannot see an indirect import, the shipped server is additionally imported in a fresh process and must not pull that engine in at all. The machine-readable capability list published on this site is rebuilt from the running tool definitions and the recorded implementation status, then compared with the copy that is live, so a stale claim fails instead of sitting there; a name unreachable from the tool interface may not be labelled as shipped. |
| Security boundary stored data stays data |
No network surface: server-side HTTP and event-stream entry points must be absent from the source, and socket binding is intercepted at runtime and must never be called while a server is constructed. No tool parameter may be named like a path, a file, a directory or a URL; the database location arrives through an environment variable and nowhere else. The dependency allowlist is enforced across all product source by syntax tree, so document and image parsers — the traditional home of file-format exploits — cannot enter through a transitive dependency. An injection payload stored, searched for and read back must come out byte-identical with the document count unchanged. Tool descriptions must contain no imperative phrasing, search results must be labelled as data, and the database file and every sidecar — including the undo journal, which holds the plaintext of whatever is being overwritten — have restricted permissions applied when they are created. |
Checked against a second implementation
A test that only asks the code whether it agrees with itself will pass while both the writer and the reader of a format share the same misunderstanding. Four places therefore carry an independent reference, and the check is a comparison rather than an assertion.
- The storage engine is run against a plain dictionary model over a long randomised sequence of writes, deletes and reads on a small key space, with the bucket count forced low so that hash collisions are guaranteed rather than hoped for.
- The index is compared with brute-force substring search over a set of the project's own long-form text files, using randomly cut substrings as queries. Both sides must return the same set of documents.
- Nearest-neighbour search is compared with a pure-Python cosine similarity over the same vectors, because an encoder and a search routine written together can share a mistake.
- The record hash is compared with an independent implementation of the published algorithm. A constant that is internally consistent but wrong passes every test that only uses it to look things up.
The order of the steps is part of the check
Every check that measures a property of the file follows the same four steps, in this order.
1 clean ..... start from an empty path; no leftovers from an earlier run
2 load ...... n documents and relations, counted, not estimated
3 confirm ... search for a known document and get it back
4 measure ... only now read the size, the ratio, the free space
step 3 is what stops "broken, therefore small"
from being read as "efficient"
Checks that are meant to fail
When a check fails because of a known defect that has not been fixed yet, it is left failing. It is not weakened, marked as expected, or deleted, because it is the only machine-readable record that the defect exists. That means a red check in this suite should be read as an open problem with a name, not as a broken test.
What we do not test.
The items below are gaps, not omissions from this page. Each one is something a reader might reasonably assume has been covered, and has not been.
| Not covered | What that means here |
|---|---|
| Other operating systems and architectures | Every check described here has only ever run on Windows x64. The storage engine has never been compiled or run on macOS, Linux or ARM. The POSIX build script is compared against the Windows one by reading both, which catches a missing source file but nothing that only appears when a compiler runs. Nothing is claimed for those platforms, and the package for them does not exist yet. |
| Other Python versions | Everything on this page was run on Python 3.14. The declared version floor is asserted by the suite but has never been exercised on an older interpreter, so a claim of support for anything below the version actually run would be a claim about untested ground. |
| Long-running operation | No database in any check lives longer than the check itself. Audit-log rotation and expiry are exercised by supplying dates, not by waiting for them, so nothing is known about a file that has been written to continuously for months, about fragmentation accumulated over a year, or about a process that stays open for weeks. |
| Concurrency at scale | The file takes an exclusive lock, and what is checked is that a second process opening it is refused. Inside one process, a handful of simultaneous tool calls and simultaneous log-maintenance passes are covered. Beyond that there is nothing: no many-client load, no sustained contention, no measurement of what happens at the lock boundary under pressure. Concurrent writers sharing one audit file can lose lines. |
| Production-scale data | The largest corpus ever indexed for a published measurement is 5,183 documents (BEIR SciFact test split, Windows x64, Python 3.14, measured 2026-08-27), and the storage growth curve was measured up to 2,000 documents (single bulk load, Windows x64, Python 3.14, measured 2026-08-27). Nothing is known about a hundred thousand documents, a million, or a file of tens of gigabytes, and a growth curve measured at small sizes is not evidence about them. |
| Hardware and filesystem faults | An abnormal exit is simulated by killing the process, which exercises the undo journal. A disk that loses a write it has already acknowledged, a full volume, a failing device and a filesystem that reorders writes are not simulated. Randomised testing exists at the interface level with a fixed seed; there is no fuzzer that mutates the file format at random, and the corruption cases are specific, hand-constructed mutations. |
| Independent audit | There has been no third-party security audit, no external certification, and no published penetration test. Everything described on this page was written by the same people who wrote the code. That is the ordinary situation for a project at this stage; it is stated here so that it is not mistaken for something else. |
| Paths that are not connected | Vector search is not reachable from the tools in this build, so it is not tested end to end; the checks that exercised it against the previous storage layer were removed rather than left skipped, because a skipped check reports nothing while looking like coverage. Backup and dump are in the same position while they are rebuilt for the current file format. |
What the current tests establish.
Everything below holds on Windows x64 with Python 3.14, which is the only configuration that has been run. Each one corresponds to checks described above.
- A committed write survives termination of the Hadano process while the operating system keeps running, and a partial write does not. Verified by killing a process mid-transaction and reopening the file. This is not a claim about sudden power loss: the operating system may still hold written bytes in its cache, and that case has not been tested.
- An unclean shutdown is reported, not swallowed. Reopening rebuilds what is derived from the documents and reports the count, and leaves the documents and the relation graph as they were.
- Deleting a document deletes everything derived from it — chunks, index entries and edges — in the same transaction. Not guaranteed: that the bytes are gone. Freed space is reused, not overwritten, so the old content can remain in the file until something else takes the space. Disk encryption is the answer to that, not this software.
- The same query against the same database returns the same result, including the same chunk ids and the same ordering.
- A database path containing non-ASCII characters works, and the file exists under the name that was asked for rather than a silently transliterated one.
- The server opens no listening socket, accepts no path argument, and imports
no document or image parser. All three are enforced by checks that fail the
build. The socket check covers listening sockets: it intercepts
bindand reads the process's listening ports. Outbound connections are constrained by the import allowlist rather than by a check of their own, so “makes no outbound connection” is not something this page claims. - Stored text is never executed by this software. Not guaranteed: that stored text is safe to act on. Results are returned as data with their provenance, and it remains the client's responsibility not to treat retrieved content as instructions.
- Only one process writes at a time. The file is held exclusively while open, and a location on a network share or a cloud-sync folder is refused when it is opened rather than corrupted later — refused by default, and only reachable by passing an explicit override.
Not guaranteed
- Any behaviour on macOS, Linux or ARM.
- Any behaviour on a Python version other than the one everything here was run on.
- Vector search, backup and restore, which are not connected in this build.
- Anything about corpora substantially larger than those measured, or about databases that have been in continuous use for a long time.
- Durability against hardware that loses acknowledged writes.
- Availability or support commitments of any kind. Nothing here is published as a service.
How results are recorded.
A number without its conditions is not a result, so numbers are published only alongside what produced them: the dataset and its size, the parameters, the platform and the date. Benchmark output carries the environment — language version, engine version, index format, embeddings mode, concurrency mode, platform — in the result file itself, so a figure can always be traced back to the implementation that produced it, and checks enforce the discipline that would otherwise decay: a metric that was not measured is recorded as null with a note rather than as zero, a measured metric must be a positive number, a tail percentile may not fall below the median, and the counts reported for a dataset must be the counts actually loaded. Figures measured on the storage layer that has since been replaced are not reprinted as current, a measurement still running appears as in progress rather than as a provisional number, and a result that rests on one model or one synthetic corpus is published with that limit attached instead of generalised.
| Published number | Recorded with it |
|---|---|
| nDCG@10 0.5800 BEIR SciFact |
5,183 documents, 300 queries, test split, top_k=50, lexical search only with embeddings off. 0 of 300 queries returned nothing. Windows x64, Python 3.14. Measured 2026-08-27. |
| Search latency current storage layer |
Not published, and no figure is quoted here in its place. The figures that existed were measured on the storage layer that has since been replaced. Repeated runs of the identical build on a machine carrying unrelated load differed from each other by more than the effect anyone would want to read out of them, so a single median cannot carry a latency claim: a timing number needs a distribution over several runs on a quiet machine, not just a stated condition. Until that exists, an old number under a new name would be worse than none. |