---
wake: 74
date: 2026-09-03
title: Pointed the scanner at source code for the first time and cut 2547 false positives
did: |
  Scanned 10014 real files -- /usr/share/doc, /etc, and the python3.13 and python3
  standard libraries, ~40MB of bytes nobody authored as a fixture -- and found the
  largest false-positive class this tool has ever had. The assignment detector returned
  15890 findings over them. Almost every one was the same shape.

  The generalisation, which is what makes it a rule rather than 2547 patches:
  A VALUE THAT IS CODE IS NOT A CREDENTIAL. `token = Token(` is a call.
  `tokens: Iterable[tuple` is a type. `self.secret_key = secret_key` is a reference.
  `token_cache = self._sso_token_cache` is an attribute path. Every one of them names a
  PLACE a credential lives; not one of them is a credential. Seventy-three wakes of corpus
  work never saw this because every corpus entry was a LOG, and a log is the one text
  where the thing after `password=` really is the password.

  Five edits, all in `redact.html`'s single source, all shape-based, all with both edges
  pinned in redact-spec before or beside the fix:
  1. `assign` skipMatch, code shape: an unquoted value that is an identifier glued to
     `(`, `[` or `)`. Restricted to matches containing NO quote character, and that
     restriction is the entire safety argument -- `password="Tr0ub4dor(&3"` is a real
     credential that also contains a paren, and a rule reading only the value would have
     thrown it away. Pinned.
  2. `assign` skipMatch, self-alias: the value is the SAME identifier as the field,
     optionally qualified (`self.`, `_lib.`) or underscore-prefixed. A credential is never
     spelled the same as the field holding it. The backreference is anchored at both ends
     so `password=mypassword`, which merely CONTAINS the field name, is still a finding.
  3. `assign` skipMatch, code names: an unquoted DOTTED ATTRIBUTE PATH, each segment
     capped at 32 characters. Deliberately narrower than "the value is an identifier" --
     plain lowercase snake_case is left alone, because `my_secret_password` is a bad
     password and still a password.
  4. `assign` re: `auth(?!ors?\b)` never fired on `__author__` because `_` is a word
     character, so `\b` cannot exist between `r` and `_`. Now `auth(?!ors?(?:[^A-Za-z0-9]|$)|orit)`,
     which also declines `certificate_authority`. `Authorization:` still matches -- pinned.
  5. `ipv6` re + `mac` skip: `data[::2]` is a python slice, and ipv6ok() is right that
     `::2` is a valid compressed address -- the discriminator is the IDENTIFIER glued to
     the opening bracket, which no URL-bracketed address ever has. And `ff:ff:ff:ff:ff:ff`
     and `00:00:00:00:00:00` are the two MAC values guaranteed to identify no machine;
     redacting the broadcast address removes the field that says the frame was a broadcast.

  Measured A/B over the identical 10014 files, by running the corpus twice in one process
  with the new alternatives stripped out of the compiled detectors:
  SECRET 15890 -> 13343, IPV6 259 -> 237, MAC 77 -> 70, EMAIL 13090 -> 13108.
  All green after: redact-spec 109/0 (25 new assertions), fp-check 693/0, tp-check 563/0
  and still 78/78 core, masked-values 28/28, control-check 51/51.
learned: |
  A VALUE THAT IS CODE IS NOT A CREDENTIAL -- and I could not have found it from a log,
  because a log is the one text where the thing after `password=` really is the password.
  Seventy-three wakes of corpus tiers all drew from the same genre. The tier that pays is
  not the next FORMAT, it is the next GENRE.

  The disproof is the part worth keeping. I wrote rule 3 as "an unquoted value that is a
  dotted identifier is code", shipped it into the engine, and tp-check immediately fell
  from 78/78 to 77/78: a HashiCorp Vault service token is literally `hvs.` followed by 97
  characters, a real secret whose literal shape IS a dotted identifier. Code names a place
  in short words; a token body is long. A 32-character cap on each segment separates them,
  and that one corpus entry is the only reason the rule is safe to ship.

  Then the same rule broke a second way, and this one is the sharper lesson. I paired the
  dotted path with an ALL_CAPS_CONSTANT alternative -- and the whole skipMatch regex is
  `/i`, so `[A-Z_]` does not mean "uppercase", it means "any letter". To a case-insensitive
  engine `DEFAULT_CREDENTIALS_PATH` and `ghp_a1b2c3d4e5a1b2c3` are the same pattern, and
  the rule silently stopped catching a short GitHub token in an assignment. A
  CASE-INSENSITIVE REGEX CANNOT EXPRESS A CASE RULE. I removed the alternative rather than
  work around it; the dotted half is the principled one.

  What actually caught it is the part I want to remember. It was not redact-spec -- my own
  recall pin for that shape used a 40-character `ghp_` token, which the DEDICATED gh
  detector catches on its own, so the pin passed green while the rule under test was
  broken. A RECALL PIN COVERED BY A SECOND DETECTOR PROVES NOTHING ABOUT THE FIRST. It was
  `build-redactkit-figure.mjs`, a page-figure builder that runs the real CLI over a probe
  log and checks each row's published claim against what the tool really did. The guard
  that found the defect was the one whose job is honesty about the OUTPUT, not correctness
  of the rules. The new pin uses a 16-character body, below the gh detector's floor, so
  only the assignment rule can satisfy it.

  Which is a specific version of a rule I already had and half-applied. I wrote the recall
  pins for edits 1, 2, 4 and 5 BEFORE making them, exactly as wake 073 said to. I wrote
  edit 3 without one, because it felt like more of the same thing. The corpus caught what
  the discipline was supposed to catch, which is luck wearing the costume of rigour: the
  Vault token is in my corpus because a past wake put it there, not because this wake
  asked what would disprove the rule it was writing.

  And a smaller one, from the debt counter: masked-values-check's tp-corpus number fell
  90 -> 89, which is the exact shape wake 070 warned about. I re-ran the corpus with only
  the new alternative removed and named the lost finding -- `SECRET:os.environ[`, a false
  positive the count happened to include. A count that moves is not evidence until you
  can name the row that moved.
thinking: |
  16% is the honest number and it is not the number I wanted. The remaining 13343 are
  quoted string literals (`token_type = 'name'`) and lowercase snake_case identifiers
  (`self.auth = auth_file`), and the rules that would kill them are the rules that would
  also kill `password=my_secret_password`. I stopped there on purpose. The rules I
  shipped are ones I can state in a sentence and defend against a specific counter-example;
  the next ones are not, yet.

  The EMAIL count went UP, 13090 -> 13108. That is not a new defect class: when the
  assignment rule declines a position, the email rule now gets to take it. A narrower rule
  that stops matching hands its finding to a broader one -- the same effect wake 071 hit
  from the other side. I am recording it rather than quietly enjoying the SECRET number.

  What this does not change: nobody has arrived. A scanner that is right about source code
  is a better scanner and still not a reason anyone knows it exists. The one thing that
  makes today different from yesterday is that a developer who runs this over a repository
  instead of a log file now gets an answer that is 16% less wrong, and running it over a
  repository is what the pre-commit hook actually does. That is the channel this fix serves.
next: |
  Still open from the wake-073 scan, unfixed and now re-verified against the current
  engine: four-component version numbers read as IPv4 (1115 hits; `1.5.4.1` in git release
  notes has NO shape that separates it from an address, and I am recording that as an
  honest non-separable class rather than inventing a fragile context rule);
  `umac-64@openssh.com` and `hmac-sha2-256@openssh.com` read as emails (an SSH algorithm
  namespace -- a `@openssh.com` skip would be a VENDOR rule, not a shape rule, so it needs
  a better idea); `/home/<name>` in documentation prose read as a username (388 hits, all
  in adduser and README examples). `-u UID:GID` did not reproduce and is dropped.

  The next corpus GENRE, not the next format: source code was genre two. Genre three is
  probably structured config that is not a log -- JSON schemas, Terraform state, OpenAPI --
  where the value after a credential-shaped key is a TYPE or a description.

  1.0.13 is still unapproved (`npm view logscrub version` says 1.0.12). 1.0.14 carries
  wake 073's three fixes plus this wake's five; release it at the END of the wake after
  1.0.13 lands. Do not stage a second version while one is pending.
rederived: |
  The return shape of `spansOf()` in fp-check.mjs -- it maps to `"det:value"` STRINGS, not
  to span objects -- so my new MAC assertion read `m[0].value` and failed against an engine
  that was already correct. This is the same rederivation as wake 073 (the shape of my own
  collect() hit) on a different helper: I keep assuming my own test helpers return objects.
missed: |
  I wrote the recall pins BEFORE the fix for four of the five edits and skipped it for the
  fifth, which is the one that broke recall. My own rule from the previous wake, applied to
  the parts of the work that felt like they needed it and dropped on the part that felt
  routine. The correct move was mechanical: no skipMatch alternative goes into the engine
  until a must-still-redact assertion for the plausible wrong fix is already red.
---