spec/whymark-v1.md

whymark v1 — a diff, marked with why

File extension: .whymark · Media type: text/vnd.whymark · Encoding: UTF-8, LF

whymark is a plain-text file format for reviewing code changes. The left side of a whymark file is an ordinary unified diff. The right side is a set of structured annotations addressed to specific line ranges of that diff: why the change was made, what evidence backs it, how it was verified, and what is still uncertain.

The name states the format's one rule. A diff already shows what changed, so a mark that repeats it is worth nothing; every mark here exists to say why.

It exists because reading an AI-authored diff is cheap but trusting it is expensive. A whymark file carries the reasoning and the receipts next to the code they belong to, so a reviewer spends their time judging decisions instead of reconstructing them.

Three properties drove the design:

  1. An agent can write it directly. No column alignment, no escaping rules, no JSON string-quoted source code. Annotations are addressed by line number, not by position on a rendered page — a model cannot get the alignment wrong.
  2. A tool can generate the skeleton. The diff half comes straight from git diff, so a whymark file can be produced for unstaged, staged, branch, or commit changes and then annotated.
  3. Claims are checkable. Verification claims name the exact command they came from, and file sections carry git blob hashes, so a reader can re-run the claims and detect a review that has gone stale.

1. Document structure

┌─ YAML frontmatter ─────────── metadata, summary, global checks
├─ @note …                      document-level annotations (optional)
└─ @file …                      one section per changed file
   ├─ @@ … @@                   unified diff hunk
   │  + - and context lines
   └─ @note …                   annotations addressed to lines in this file

A minimal but complete document:

---
whymark: 1
title: Cache the user lookup in the session middleware
author: claude-opus-5 (cursor)
date: 2026-09-15T12:04:00Z
scope: staged
base: main@47abc20
summary: |
  Session middleware hit the database on every request. This memoises the
  lookup for the lifetime of a request.
---

@file src/middleware/session.ts modified +6 -1 newsha=9f2a1c4
@@ -12,7 +12,12 @@ export async function loadSession(req: Request) {
   const token = req.headers.get("authorization");
   if (!token) return null;
-  return db.users.findByToken(token);
+  const cached = perRequestCache.get(req);
+  if (cached) return cached;
+  const user = await db.users.findByToken(token);
+  perRequestCache.set(req, user);
+  return user;
 }

@note +15..18 kind=intent risk=low confidence=0.9
why: `loadSession` is called by four separate handlers per request, so the
  same token was resolved up to four times against the database.
source: file:src/handlers/*.ts — four call sites confirmed by grep
source: file:src/lib/per-request-cache.ts:8 — existing WeakMap cache, reused
verify: cmd `npm test -- session` => pass (12 assertions)
verify: manual "4 handlers, 1 query" => pass (checked query log locally)
The cache is keyed on the `Request` object via `WeakMap`, so entries are
released with the request. It deliberately does **not** cache negative
lookups — see the note below.

@note +17 kind=assumption confidence=0.5
why: A `null` user is stored and returned as a cache hit. If a token is
  granted mid-request this returns stale `null`.
source: inference — no requirement found either way in the codebase.
verify: none
todo: confirm with the auth owner whether mid-request grants are possible.

1.1 Line kinds

Every line in a whymark document is exactly one of:

LineMeaning
---Frontmatter fence (first line of file, and its closing fence)
@file …Starts a file section
@@ … @@Starts a diff hunk
, +, -, \ prefixedDiff content, only inside a hunk
@note …Starts an annotation
@check …A document-level verification run
key: valueA field, only inside an annotation
# at column 0Comment, ignored (only outside hunks)
emptySeparator, or a context line inside a hunk (§3.3)

Directives are recognised only at column 0, and diff content always carries a one-character prefix, so a diff of a whymark file cannot be misparsed.


2. Frontmatter

Strict YAML between two --- fences at the very start of the file.

KeyReqTypeNotes
whymark✔intFormat version. 1.
title✔stringOne line, imperative. What the change does.
authorstringModel and/or human. claude-opus-5 (cursor).
dateISO 8601When the review was written.
scopeenumunstaged staged worktree branch commit manual
basestring<ref>@<sha> the diff is against.
headstring<ref>@<sha> or working-tree.
repostringRepository name or URL.
summarymarkdownThe change in a paragraph. Written for a reviewer who has not seen the branch.
tagsstring[]Free-form, e.g. [auth, perf].
checksCheck[]Whole-change verification runs (§2.1).
reviewReviewHuman review state (§2.2).
statsobjectOptional cache of files/added/removed; recomputed by tools.

Unknown keys are preserved and reported as warnings by whymark validate.

2.1 checks

Commands that were run against the whole change, as opposed to a claim about one line range.

checks:
  - cmd: npm run typecheck
    status: pass
  - cmd: npm test
    status: fail
    detail: "1 failing: rate-limit clears window on tick"
    ran: 2026-09-15T12:03:11Z

status is pass | fail | unknown | skipped. whymark verify re-runs every cmd and rewrites status, detail, and ran with what actually happened.

2.2 review

review:
  status: pending        # pending | approved | changes-requested
  by: alex
  at: 2026-09-15T13:00:00Z

3. File sections

@file <path> [status] [+A] [-R] [from=<old path>] [oldsha=<sha>] [newsha=<sha>] [lang=<id>] [binary]

path is repo-relative and may be quoted if it contains spaces. Everything after it is order-independent. status is one of added modified deleted renamed context; context marks a file included only for background, with no changes of its own.

newsha is the git blob hash of the file's new content (git hash-object). whymark validate compares it against the working tree and reports the review as stale when they differ — the change moved on and the annotations may no longer describe it. oldsha does the same for the base side.

lang overrides syntax-highlight detection, which otherwise comes from the extension.

3.1 Hunks

Standard unified diff hunk headers:

@@ -<oldStart>,<oldLines> +<newStart>,<newLines> @@ [section heading]

The counts are used to bound the hunk body, but a parser MUST fall back to reading until the next directive when they disagree with the content, emitting a warning rather than failing. @@ with no ranges is accepted for hand-written files; line numbers then continue from the previous hunk, or start at 1.

3.2 Diff content lines

PrefixMeaning
+Added — exists on the new side only
-Removed — exists on the old side only
(space)Context — exists on both sides
\\ No newline at end of file

3.3 Empty lines in a hunk

A context line for an empty source line should be a single space, but editors and models strip trailing whitespace. An empty line inside a hunk is therefore read as an empty context line, and trailing empty context lines are dropped when the hunk closes. This makes a blank line before a @note safe to write.


4. Annotations

@note <selector> [key=value …]
<field>: <value>
  <continuation, indented>
<field>: <value>

<free-form markdown body>

An annotation ends at the next @note, @file, @@, @check, or end of file.

4.1 Selectors

A selector says which lines the annotation is about. Line numbers are the file's own line numbers as shown in the diff — not offsets into the hunk.

SelectorMeans
+42New-side line 42
+42..57New-side lines 42 through 57
-30Old-side line 30 (something that was removed)
-30..35Old-side range
fileThe file as a whole
docThe whole change (only valid before the first @file)

+42..+57 is accepted as a synonym of +42..57. Ranges are inclusive. A selector that names lines absent from the diff is a validation error — this is the main defence against an annotation that has drifted off its code.

4.2 Header attributes

AttributeValuesMeaning
kindsee §4.3What sort of annotation this is. Default note.
risklow medium highBlast radius if this is wrong.
urgencyinfo normal urgent blockingPriority to act. Accepted on input; writers emit the body field below for older v1 readers.
confidence0–1How sure the author is. Be honest; low confidence is a feature.
idslugStable anchor for deep links. Auto-assigned (n1, n2, …) if absent.

4.3 Kinds

kindUse it for
intentWhy this code exists and what it is for. The default choice for a real explanation.
sourceWhere the approach came from, when provenance is the point.
verifyEvidence that it works.
riskA known hazard.
assumptionSomething taken on faith that a reviewer should confirm.
alternativeA different design considered and rejected, and why.
todoDeliberately unfinished work.
questionA decision the author could not make alone.
securityAn authz/authn/injection/secret-handling consequence.
perfA complexity or allocation consequence.
testWhat the test covers, and what it does not.
generatedMechanically generated or vendored code that was not reasoned about.
noteAnything else.

4.4 Fields

Repeatable fields (source, verify, alt, todo, question, ref) may appear many times in one annotation. A value continues onto following lines while they are indented by at least two spaces.

A field is name: followed by whitespace, or — for the field names in the table below only — name: with no space at all, since why:text is a natural thing to write. Restricting the tight form to known names keeps a body line that starts with https://… from parsing as a field called https.

FieldRepeatMeaning
urgency–info, normal, urgent, or blocking; priority to act, independent of risk. Unknown values are preserved and warned about.
why–Why the code is the way it is. The most important field.
what–What the code does, when it is genuinely non-obvious. Do not narrate.
source✔Evidence and provenance (§4.5).
verify✔Verification claim (§4.6).
alt✔<rejected option> — <reason>
todo✔Follow-up work.
question✔Question for the reviewer.
ref✔Related location, file:path:line or a URL.
impact–What else changes because of this.

4.5 source — evidence

source: <type>:<locator> — <why it matters>

The em dash (or - ) and the trailing note are optional but strongly encouraged: a bare link is not evidence, it is homework for the reviewer.

TypeLocatorExample
filepath[:line[-line]]file:src/config.ts:14 — WINDOW_MS already existed
urlURLurl:https://redis.io/commands/incr — INCR is atomic
docpath or namedoc:docs/auth.md#tokens — token lifetime is 15m
specname/sectionspec:rfc6585#4 — 429 semantics
commitshacommit:a1b2c3d — the bug this fixes was introduced here
pr / issueid or URLissue:#412 — reported by a customer
testpath/nametest:tests/limiter.test.ts:31 — pins the boundary
depname@versiondep:ioredis@5.4.1 — pipeline API used below
conventionpathconvention:src/lib/*.ts — every module exports a factory
promptquoteprompt:"cap it at 100 requests a minute" — the number came from you
inference–inference — no source; pattern-matched from surrounding code

inference is the honest label for "the model made this up from context". It is not a failure — most code is written that way — but it must be visible, and tools surface it separately from sourced claims.

4.6 verify — evidence that it works

verify: <method> [`detail`] => <status> [(comment)]
MethodMeaning
cmdA command that was run. Put it in backticks so it can be re-run.
testA specific test that covers this.
typeThe type checker proves it.
lintA lint rule enforces it.
manualA human or agent checked it by hand; say exactly what was observed.
reviewRead carefully, nothing executed.
noneNot verified. Say so rather than omitting the field.

Status is pass | fail | unknown | skipped.

verify: cmd `npm test -- rate-limit` => pass (8 assertions)
verify: type => pass (no `any` in the public signature)
verify: manual "429 after the 101st request" => pass
verify: none => unknown (needs a load test)

whymark verify re-runs every cmd claim, compares the exit code to the claimed status, and reports each claim as confirmed, contradicted, or unrunnable. A claim of pass on a command that now exits non-zero is the single highest-value signal in the format.


4.7 Urgency and compact rendering

Writers emit urgency: urgent before other fields, rather than urgency=urgent in the header. Older v1 parsers preserve unknown body fields but may drop unknown header attributes. This additive convention retains the value on round trips through older writers. Updated readers also accept header input. Missing urgency remains unspecified; never infer it from risk or kind. Collapsed notes retain kind, urgency, selector and explicitly labeled author confidence (unknown when absent). Multiple annotations may share a selector. Viewers must expose each independently, including when other annotations on that line have been filtered out.

5. @check

A verification run that belongs to the change as a whole, written in the body instead of frontmatter. Same grammar as a verify field:

@check cmd `npm run typecheck` => pass
@check cmd `npm run build` => fail (Type error in src/app/page.tsx:22)

Tools merge @check lines and frontmatter checks into one list.


5.1 Execution evidence extension

Optional frontmatter evidence is an array of version-1 execution records. Existing v1 tools preserve this frontmatter extension; a reader must not promote legacy passing claims to fresh execution results. Each record contains:

  • version: 1, command, noteId (null for a global check), file (nullable), claimed (original claim), status (pass, fail, unavailable).
  • ran (ISO time), durationMs, exitCode (nullable), cwd (repo relative), tool (whymark version), runtime (Node version).
  • source / sourceAfter (nullable SHA-256 source fingerprints), head and base (nullable revisions), reviewHash (SHA-256 of the normalized diff).
  • outputHash (SHA-256), outputArtifact (repo-relative output file), summary.

verify --write records these alongside updated claims. Logs are saved under .artifacts/whymark/<hash>.log; inspect logs before sharing them. Fingerprints cover HEAD, tracked files (including deletions), and non-ignored untracked files, including tests/config/lockfiles and symlink targets as link text. Generated .artifacts/ and review outputs under reviews/ (.whymark, .quality.json) are excluded. Ignored inputs, installed dependency bytes and external services are not fingerprinted; this is bounded evidence, not a reproducible-build attestation. Reviews outside reviews/ may invalidate their own source fingerprint when rewritten.

Hashes bind evidence to bytes; they do not authenticate the author or guarantee truth. Imported records are labeled as such. The local viewer checks current source, normalized diff identity and output hashes before showing a current record. A changed source or diff is stale; missing outputs/fingerprints are unavailable. A passing and failing claim on one note cannot yield unqualified passing coverage. A contradiction is retained even when the original claim is rewritten with the command result. Viewing/importing never executes commands. Explicit verify executes shell commands from the review; inspect them before using it on an untrusted document.

5.2 Quality report extension

Optional frontmatter quality holds one version-1 report. Fields: version, ran, source, sourceAfter, head, clean (whether relevant source was clean), base, reviewHash, provenance (local-run or imported), comparison (available or unavailable), checks, and findings. Each check has id, version, command, status (pass, fail, unavailable), exitCode, outputHash, outputArtifact, and detail. Each finding has id, tool, rule, severity (warning or error), message, nullable repo-relative path, nullable one-based line and endLine, optional HTTP(S) helpUrl, and status (new, existing, resolved, uncompared). Optional suppression contains reason and an ISO expires time. Expired suppressions do not suppress a new scan. IDs derive from tool/rule/path/message plus duplicate occurrence, so shifted lines retain identity; identical duplicate findings can be ambiguous. Rename mappings are taken from Git when a baseline is available.

Quality reports are supplied evidence, never measured accuracy. Failed/missing scanners and missing/mismatched baselines cannot be treated as a clean comparison. Importers bound report size, validate paths/ranges and preserve project-level findings. Unknown extension data remains in frontmatter; malformed known data is diagnosed and must not be interpreted as successful evidence.

6. Derived metrics

Tools compute these; they are never stored in the file.

  • Coverage — the share of added lines (+) covered by at least one annotation. Uncovered added lines are code that arrived with no explanation.
  • Claimed verified coverage (API: verifiedCoverage) — the share of added lines covered by an annotation with at least one verify claim whose status is pass, with no failing claim on that note. This is author-reported coverage, not fresh execution or a correctness score.
  • Sourced share — annotations with at least one source that is not inference.
  • Open items — todo and question fields, plus fail/unknown claims.
  • Risk profile — the count of high/medium risk annotations, and whether each high-risk annotation carries a passing verification.

7. Error recovery

The format is written by language models, so a parser must lose nothing when a document is slightly wrong. Required recoveries, each reported as a warning:

InputBehaviour
Hunk counts disagree with the bodyRead until the next directive; keep the lines
Selector partly outside the diffKeep the annotation, warn; error only if no named line exists
risk: if the CDN rule is removed… — prose where a fixed value belongsKeep the text as a plain field. Never drop it
A field after the free-form bodyResume field parsing at any documented field name
Blank line inside a hunkEmpty context line; trailing ones dropped at the hunk's end
Unknown kind, risk, or field nameKeep the value, warn, treat kind as note
Duplicate annotation idRename the later one
No frontmatter, or no whymark:/title:Report an error but still parse the body

Dropping content silently is the one unacceptable outcome: a reviewer cannot notice a sentence that was never displayed.

8. Conformance

A parser must accept every example in this document, address annotations by line number, and perform every recovery in §7.

A writer must emit whymark: 1 and title, must not emit an annotation whose selector names lines absent from the diff, and should emit newsha for every file it takes from git.

A renderer must show each annotation against the lines its selector names, must distinguish the verification statuses from each other, and must show inference sources as distinct from external evidence.


9. Design notes

Why line numbers instead of inline annotation? An inline format (a trailing || comment on each line) forces the writer to align columns and to fragment a paragraph across lines. Line-addressed annotations let one coherent explanation span a range, let several annotations cover the same line for different reasons, and survive a re-generated diff.

Why keep unified diff? Every tool already speaks it, git diff produces it, and a whymark file therefore degrades gracefully: strip the annotations and it is still a patch a reviewer can read.

Why is provenance a first-class field? The reviewer's real question about AI-written code is not "what does this do" but "where did this come from and did anything check it". source and verify answer exactly those, and inference makes the absence of an answer legible instead of silent.