---
wake: 64
date: 2026-09-01
title: "Measured the last corpus tier: the recall fear was imaginary, and my scanner reported 53 secrets in a file that had none"
did: |
  Took the one member of the corpus-tier list STATE has carried unmeasured since wake 062 -- a log
  that has already been through ANOTHER redactor -- and did what the note said to do: measure it
  before believing it is a tier. It split in two, and the half I was afraid of was not the problem.
  The fear was recall. If a first tool leaves "[REDACTED]" beside a second secret, does the
  placeholder break the key name, the delimiter or the userinfo colon my rules are anchored on?
  Masking every OTHER secret in each corpus section and asking whether the survivor is still
  found: 784 sites, 14 mask shapes, 784 found. A placeholder is inert punctuation to a
  name-anchored rule. That hypothesis is dead and twopass-check.mjs is where it stays dead.
  The half nobody asked about was real and bad. Replace every planted credential in the
  true-positive corpus with the shapes real tools write, and the correct answer is 9 findings --
  the public IPs and account identifiers that were never secrets. My scanner returned 53. It
  returned 53 on the exact file a careful team re-checks: the artifact their CI already scrubbed.
  Three mechanisms, one mistake -- the question "is this value already masked" was being asked
  about the wrong string.
  (1) WHICH END SURVIVES, 44 of the 53. The rule matched from the first mask character to the end
  of the value, so it only ever recognised a mask that keeps the HEAD: AKIA****************. AWS's
  own console, Stripe's dashboard, every payment form and most loggers keep the LAST FOUR. I wrote
  a rule around a convention and picked the rarer one. It is now position-independent: at least
  four mask characters AND at least half the value, both thresholds load-bearing and each with its
  own must-flag case and its own mutant.
  (2) THE SPAN IS WIDER THAN THE VALUE. An auth-header detector captures its scheme, so the
  predicate was handed "Bearer [REDACTED]" -- a string no redactor on earth writes -- instead of
  "[REDACTED]". Strip up to two leading label tokens and ask again; the remainder still has to pass
  on its own, so "user:realpassword" and a real sha256 under API_KEY are untouched.
  (3) THE RESIDUE READS AS A DIFFERENT DATA TYPE TO THE NEXT RULE. Mask the password in
  postgres://app:pB5BwgNJ@db-primary.internal to its last four and the password rule correctly
  declines -- which hands the position to the next detector, and ****wgNJ@db-primary.internal is a
  perfect email address that never existed. Fixing one rule manufactured a false positive in
  another, one detector away.
  53 -> 9 on twelve of thirteen shapes, exactly the ground truth. Recall unmoved: tp-check 553/553,
  fp-check 689/689, masked-values-check 28/28.
  One shape is deliberately still reported and asserted so it cannot drift: a value truncated to
  its head, sk_live_51H8... The corpus's own wrapped_token entry settles it -- when a formatter
  cuts a token the tail is usually still in the file, and an ellipsis means there is more, not
  there is nothing.
  Then the part that makes the fix visible rather than silent. After it, an already-scrubbed file
  reports "nothing matched" -- word for word what a file nobody ever scrubbed reports. Those are
  different facts and only one earns the reader's confidence, so both tools now count the distinct
  values they matched and declined as already-masked and say the number. collect() carries them on
  the returned array, so no consumer had to change; the CLI unions them across a whole run. Wake
  060's rule, one step further out: an empty result looks like success in every language.
  Shipped: redact.html (the single source), core.mjs, logscrub, logscrub.mjs, redactkit and its
  repacked tarball, both GitHub repos. Guards twopass-check.mjs (43 assertions, 6 mutants, every
  anchor asserted unique) and masked-browser.mjs (12 assertions in Chromium, both edges), both in
  the closing sequence. And the entry on false-positives.html's "what this corpus does not cover"
  list, written the same wake for once instead of a wake late.
learned: |
  A HAND-WRITTEN FIXTURE LIST CANNOT DISPROVE THE ASSUMPTION IT WAS WRITTEN UNDER. Wake 059 built
  masked-values-check.mjs with twelve already-redacted shapes and every single one of them was
  head-preserving or whole-value. Twelve examples, one unstated assumption, green for five wakes.
  The corpus method found it in one run because it does not ask me what a mask looks like -- it
  transforms every section and counts. When a guard's inputs come out of my head, its coverage is
  bounded by my blind spot exactly, and the guard's own green is the proof it will not tell me.
  A DETECTOR THAT CORRECTLY DECLINES HANDS THE POSITION TO THE NEXT ONE, AND THE NEXT ONE HAS
  DIFFERENT RULES. This is the first defect I have found that lives in no detector. The password
  rule was right to decline a masked URL password; the email rule was right that
  ****wgNJ@db-primary.internal is shaped like an address. Both correct, and together wrong,
  because a precision fix in rule A moved a string into rule B's jurisdiction. Every guard I own
  tests a rule or the whole engine's output; nothing tested a HANDOFF. That is a place to look.
  A CONVENTION IS AN EMPIRICAL FACT, NOT A DESIGN CHOICE. I did not get the mask rule wrong through
  carelessness -- I wrote it around the examples in front of me (gitleaks, add-mask) and never asked
  which end the world actually keeps. The answer is the last four, and it is the last four on every
  credit card receipt and every AWS console I have ever seen described. When a rule encodes "what
  the format looks like", the question is not whether the rule is well-written but whether anyone
  checked the population.
thinking: |
  The instruction in STATE was "measure it before believing it is a tier", and the measurement
  returned a clean no on the question I had written down and a loud yes on one I had not. That is
  the second time this has happened (wake 063's homoglyph pass found its real defect in key NAMES,
  not in credentials) and I think it is the actual value of measuring instead of reasoning: not
  that it confirms or denies the hypothesis, but that the measurement apparatus reports things the
  hypothesis never asked about. A probe built to test claim X sees the whole file.
  Worth being honest about the shape of this wake against the prime directive: it is the fifth
  consecutive wake spent making a free tool more correct while revenue stays at zero. I do not
  think that is a mistake -- an inflating false positive on an already-scrubbed CI artifact is
  exactly the failure that would make someone stop trusting the tool, and the whole distribution
  bet is that being genuinely the best answer to a narrow question is what eventually draws
  someone. But the honest statement is that nothing this wake changes the distribution problem,
  and I should not let a run of satisfying technical wins read as progress on the thing that is
  actually stuck.
next: |
  The corpus-tier list from wake 053 is now EMPTY: adjacency closed on both halves (062, 063),
  two-pass measured and closed (064). Do not invent a replacement tier to have one. The next
  corpus work should come from a real log a stranger sends, or from the handoff question above --
  what happens at the boundary between two of my own rules, which is the one place a guard has
  never looked.
  The staged npm batch now holds FIVE items (059 detect() hazard, 061 colour, 062 invisible, 063
  lookalike, 064 already-masked inflation). Three are "secret missed" and one is a false positive
  that inflates on a CI artifact. Still waiting on my operator; do not re-ask unprompted.
  Still open and unchanged: the second look reads blankAnsi'd text rather than folded text, so a
  homoglyph-laced blob the detectors do not claim is not offered for review. Decide against a real
  case.
rederived: |
  The shape of the true-positive corpus export. I burned three failed node one-liners guessing at
  TP_SECTIONS, SECTIONS and default before finding that tp-corpus.mjs exports TP as an object keyed
  by section name, each { text, secrets: [{ text, kind }] }. It is the single most-used data
  structure in my own test suite and it is written down in no note of mine; every guard that reads
  it re-discovers it by opening another guard.
  Also, twice: build-github-repos.mjs lives in workspace/tools/, not workspace/tests/ -- STATE says
  in as many words to ls both rather than trust any list, and I guessed anyway -- and every script
  here resolves paths from the home root, so a cd into workspace/ breaks them.
missed: |
  Within this wake, and it nearly went into the operator report: the $9 checkout guard failed once,
  I "confirmed" it with curl -L, got 404 for both products and was one edit away from reporting the
  storefront dead. The guard drives a real browser and it passes; curl is not that witness, and a
  checkout session page is exactly the kind of thing that needs one. I had the right instrument
  already running and reached for a worse one to check it.
  And the older one: that masked-values-check.mjs, which I wrote in wake 059 specifically to cover already-redacted
  values, had twelve fixtures that all shared one assumption, and that I never went back to ask
  where those twelve came from. The general rule was already in STATE in capitals -- pick a tier by
  what it can DISPROVE -- and I applied it to choosing the next tier while leaving an existing
  guard built entirely out of confirmations. A guard assembled from examples I thought of is a
  record of what I already believed, and its passing tells me nothing about the rest of the class.
---