Field guide

How to design an API token

You are building an API and you need to hand your users a secret string. The format you pick decides one thing above all: when that string leaks, does anybody notice? This page is the design I would use, why each part earns its place, and a reference implementation in JavaScript and Python that you can copy.

The anatomy · The prefix · The random part · The checksum · Try it · The code · Mistakes · Shipping it

01The anatomy

A good token is three parts glued together, and each part exists for a different reader.

acme_cvKWVJ69KOFmbZXOVmWoPrNLM3PSPt3XITui

Read by a scanneracme_ Turns an anonymous run of base62 into a string that grep can find. Without it, the leak happens and nobody learns of it.

Read by the attackerthe random part The only segment doing cryptographic work. Everything else here is logistics; this is the part that has to be unguessable.

Read by the scanner, againchecksum Lets the scanner be sure before it wakes somebody. Without it every long string is a maybe, and a maybe gets muted.

The random part is the only part doing cryptographic work. The other two exist so that the secret is findable when it escapes — and a secret that escapes without being noticed is the expensive kind. Almost every token format shipped by a large provider in the last few years converged on roughly this shape, and it is not a coincidence.

02The prefix, and why it is not decoration

An unprefixed token is 36 characters of base62. So is a session ID, a build hash, an upload key, a cache bust and half the opaque strings in any log file. Nothing can tell them apart, which means nothing can find yours. A prefix buys you four things at once:

• Automated scanning. Repository scanners, log scrubbers and CI checks work by pattern. acme_ is a pattern. 36 characters of base62 is not — writing a rule for it would flag every hash in the repository.
• Someone can revoke it without knowing what it is. A stranger who finds acme_… in a public gist can search that prefix and reach you. A stranger who finds an anonymous blob does nothing at all.
• Your own logs become searchable. One grep tells you whether a token has ever been written where it should not have been.
• Support stops guessing. "Paste your key" turns into a string that says what it is, so nobody debugs an hour of the wrong credential.

Choosing one

• Make it unique enough to search. api_ and key_ are useless; every project on earth has them. Use your product name.
• Separate it with an underscore. Underscore is a word character, so \b in a regex sits at the start of the prefix and not in the middle of it, and the whole token stays one double-click selection. A hyphen splits the token in two for word-boundary purposes and in most editors for selection purposes as well.
• Encode the kind and the environment in it. acme_sk_live_ versus acme_sk_test_ versus acme_pk_ lets a scanner assign severity, not just identity. A leaked test key and a leaked live key deserve very different alarms, and only the prefix can tell them apart before anything is looked up.
• Never put anything meaningful after it. No user ID, no timestamp, no tenant name. The rest of the token is random or it is a hint.

03The random part

This is the part that has to survive someone guessing at it. Two decisions: the alphabet and the length.

Use base62 — 0-9A-Za-z. Not base64: +, / and = need escaping in URLs, break word boundaries in every scanner regex, and get mangled by anything that touches the string on its way through a form. Not hex: it works, it is just half the density for no benefit. Base62 survives URLs, shell arguments, JSON, CSV, double-click selection and copy-paste out of a terminal, which is the whole job.

128 bits is the floor. Below that you are relying on rate limiting to save you, and rate limiting is a thing you can misconfigure. Here is what 128 bits costs in each alphabet:

The reference implementation below uses 30 base62 characters, which is 178 bits. That is more than the floor on purpose: the extra eight characters cost nothing, and it makes the total token length a memorable, fixed 36 characters after the prefix, so the scanner regex is an exact quantifier instead of a range.

The bias nobody notices

The obvious way to turn random bytes into base62 characters is byte % 62. It is subtly wrong: 62 does not divide 256, so bytes 0–7 map to the first six characters of the alphabet twice as often as the rest. You lose a fraction of a bit and you gain a statistical fingerprint. The fix is rejection sampling — discard any byte of 248 or more (248 = 4 × 62) and draw again. It costs about 3% more random bytes and nothing else. I got this wrong in my own first draft of the code below; the test that checks the alphabet is uniform is what caught it.

04The checksum

This is the part most designs leave out, and it is the one that makes the format genuinely better rather than merely tidy.

The last six characters are a CRC-32 of the random part, written in base62. It is not a security feature — anyone can compute it — and it is not there to stop forgery. It is there so that anything holding the token can tell, offline and instantly, whether the string is real. That buys three things:

• Scanners stop crying wolf. A secret scanner that matches acme_[0-9A-Za-z]{36} will hit example values, test fixtures, docs snippets and random hashes. With a checksum it can discard all of them without a network call. The odds of an arbitrary 36-character string passing are 1 in 62⁶ — 1 in 56,800,235,584. A scanner with a false-positive rate that low is one people leave switched on, and a scanner people switch off protects nobody.
• Typos and truncation are caught at the door. A token that lost its last character to a copy-paste, or gained a stray one, fails the check before it costs anyone a round trip or an unhelpful 401. I verified this exhaustively: every single-character change, at every position, to every other character in the alphabet — 43,920 mutations — is rejected. That is a property of CRC-32, not luck.
• You can reject garbage without touching the database. Malformed tokens die in constant time at the edge, and never become a lookup an attacker can time or flood.

05Try it

This runs the exact code printed in the next section, in your browser. Nothing is sent anywhere — there is no server to send it to. The generated tokens are real random values from your own machine's CSPRNG, so do not use one as a password and expect me to have kept a copy.

06The reference implementation

Both files are public domain: copy them, rename them, no attribution needed. They are not illustrations — the JavaScript below is the code the widget above is running, injected into this page from the same file the test suite imports, so what you read is what I tested. The two implementations are cross-checked against each other on a thousand random inputs, and the CRC-32 is checked against the standard "123456789" → 0xCBF43926 vector, so it agrees with zlib.crc32, binascii.crc32 and every other stock CRC-32 you might verify against in a third language.

JavaScript

// tokenkit — reference implementation for prefixed, checksummed API tokens.
// This exact file is the source of the JavaScript shown on token-design.html:
// build-token-page.mjs injects it, and tokenkit-spec.mjs tests it. If you edit
// one, the other two follow automatically. Public domain — copy it.

const B62 = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz";

// CRC-32/ISO-HDLC, the same one zlib.crc32 and Python's binascii.crc32 compute.
let TABLE = null;
function crc32(str) {
  if (!TABLE) {
    TABLE = new Int32Array(256);
    for (let i = 0; i < 256; i++) {
      let c = i;
      for (let k = 0; k < 8; k++) c = c & 1 ? 0xedb88320 ^ (c >>> 1) : c >>> 1;
      TABLE[i] = c;
    }
  }
  let crc = -1;
  for (let i = 0; i < str.length; i++) {
    crc = (crc >>> 8) ^ TABLE[(crc ^ str.charCodeAt(i)) & 0xff];
  }
  return (crc ^ -1) >>> 0;
}

// A CRC-32 is 32 bits, and 62^6 > 2^32 > 62^5, so six base62 characters hold
// any checksum exactly, with no truncation and no wasted room.
function toBase62(n, width) {
  let out = "";
  do { out = B62[n % 62] + out; n = Math.floor(n / 62); } while (n > 0);
  return out.padStart(width, "0");
}

// 62 does not divide 256, so `randomByte % 62` is biased toward the first six
// characters of the alphabet. Rejection sampling removes the bias: 62*4 = 248,
// so discard any byte of 248 or more and draw again. Costs ~3% more bytes.
function randomBody(len) {
  let out = "";
  const buf = new Uint8Array(len);
  while (out.length < len) {
    crypto.getRandomValues(buf);
    for (let i = 0; i < buf.length && out.length < len; i++) {
      if (buf[i] < 248) out += B62[buf[i] % 62];
    }
  }
  return out;
}

export const BODY_LEN = 30;   // 30 * log2(62) = 178 bits of entropy
export const SUM_LEN = 6;

export function generate(prefix) {
  if (!/^[a-z][a-z0-9_]{0,15}$/.test(prefix)) {
    throw new Error("prefix must be lowercase letters, digits and underscores, starting with a letter");
  }
  const body = randomBody(BODY_LEN);
  return prefix + "_" + body + toBase62(crc32(body), SUM_LEN);
}

export function verify(token, prefix) {
  const head = prefix + "_";
  if (!token.startsWith(head)) return false;
  const rest = token.slice(head.length);
  if (rest.length !== BODY_LEN + SUM_LEN) return false;
  if (!/^[0-9A-Za-z]+$/.test(rest)) return false;
  const body = rest.slice(0, BODY_LEN);
  const sum = rest.slice(BODY_LEN);
  return toBase62(crc32(body), SUM_LEN) === sum;
}

export { crc32, toBase62, B62 };

Python

# tokenkit — reference implementation for prefixed, checksummed API tokens.
# This exact file is the source of the Python shown on token-design.html.
# tokenkit-spec.mjs cross-checks it against the JavaScript version: same body
# string in, same checksum out, for a thousand random cases. Public domain.

import re
import secrets
from binascii import crc32

B62 = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz"
BODY_LEN = 30   # 30 * log2(62) = 178 bits of entropy
SUM_LEN = 6     # 62**6 > 2**32, so six chars hold any CRC-32 exactly

_PREFIX_RE = re.compile(r"^[a-z][a-z0-9_]{0,15}$")


def to_base62(n: int, width: int) -> str:
    out = ""
    while True:
        out = B62[n % 62] + out
        n //= 62
        if n == 0:
            break
    return out.rjust(width, "0")


def checksum(body: str) -> str:
    return to_base62(crc32(body.encode("ascii")), SUM_LEN)


def generate(prefix: str) -> str:
    if not _PREFIX_RE.match(prefix):
        raise ValueError("prefix must be lowercase letters, digits and underscores, starting with a letter")
    body = "".join(secrets.choice(B62) for _ in range(BODY_LEN))
    return f"{prefix}_{body}{checksum(body)}"


def verify(token: str, prefix: str) -> bool:
    head = prefix + "_"
    if not token.startswith(head):
        return False
    rest = token[len(head):]
    if len(rest) != BODY_LEN + SUM_LEN or not rest.isalnum() or not rest.isascii():
        return False
    return checksum(rest[:BODY_LEN]) == rest[BODY_LEN:]

07Mistakes worth avoiding

08Shipping it

1. Publish the regex. Put \bacme_[0-9A-Za-z]{36}\b in your security documentation, in plain sight. Every scrubber, scanner and CI rule that will ever protect you is written by someone who needed to find that line. Publish the checksum rule next to it so they can filter their own false positives.
2. Register with the scanning programs. The large code hosts run partner programs that will match your prefix across public repositories and notify you directly. Look up the current process for the hosts your users push to — this is the single highest leverage hour in the whole project, because it turns other people's mistakes into a webhook you receive instead of a breach you read about.
3. Auto-revoke on that webhook. When the notification arrives, kill the key and tell its owner. Do not queue it for review; the scrapers are already faster than you.
4. Make rotation boring. Overlapping keys, no downtime, one button. If rotating is a maintenance window, nobody rotates, and every leak becomes permanent.
5. Log the prefix, never the token. acme_…4bTq is enough to debug with and useless to steal.