Kritolith

Verify, dedupe, reproduce.

Kritolith checks incoming security reports before a maintainer reads them.
Today it tests every claim against the real code, flags duplicates, and signs the verdict, with the evidence for each.
Reproducing the PoC in a locked-down sandbox is next on the roadmap.

Try it · How it works · Verdicts · Where it stands · Roadmap · Threat model · Contributing · Security

Warning

Kritolith is pre-alpha. kritolith check works today for Go projects: it extracts the claims in a report, checks them against the repository at the claimed commit, and looks for duplicates. It does not run PoC code yet, so it cannot say whether a report reproduces. Read the threat model, including its list of known gaps, before pointing it at a real embargoed report.

Try it

You need Go 1.26 or later and git. No LLM, no container runtime, no configuration file.

go install github.com/ergasterion-dev/kritolith/cmd/kritolith@latest

git clone https://github.com/ergasterion-dev/kritolith && cd kritolith
kritolith check --repo golang/net --ref e1fcd82abba34df74614020343be8eb1fe85f0d9 \
  testdata/corpus/fabricated/fab-001-invented-method/report.md

That report describes an out-of-bounds read in a method called (*Framer).ReadContinuationUnsafe. The method was invented for the test corpus. Kritolith fetches golang/net, looks at the claimed commit, and answers:

Kritolith: GROUNDING_FAILED at e1fcd82abba3
• file http2/frame.go — file exists at e1fcd82abba3
• function (*Framer).ReadContinuationUnsafe — not declared in the repository; closest match: parseContinuationFrame (http2/frame.go:1257)
• version v0.38.0 — found in report text; not yet checked against code
• a claimed file, function or package does not exist at the resolved commit

The first run clones the repository into a local mirror; later runs on the same repository take about half a second. Add --json for machine-readable output.

A real advisory gets a different answer. Run it on testdata/corpus/real/go-2024-3250/report.md (--repo golang-jwt/jwt --ref 9358574a7a1a2c8d644f22b6e8de627ba85c58d0) and the claimed function is found at parser.go:52; the verdict is INCONCLUSIVE because reproducing the PoC is the part that is not built yet.

---

Why

Maintainers are buried in security reports. Many cite files or functions that don't exist, many are duplicates of each other, and refuting a single fake one can take hours. The harder problem now is volume: too many credible-looking reports and not enough maintainer time.

Kritolith does the tedious first pass so you only spend time on reports that survived it. It never makes the decision for you. Every verdict comes with the evidence behind it, and humans decide what happens next.

How it works

flowchart LR
    A["📥 Intake<br/>GitHub private reports<br/>.eml · CLI"] --> B["🔎 Extract<br/>files, functions,<br/>lines, versions"]
    B --> C["🌳 Ground<br/>check claims against<br/>git at the claimed ref"]
    C --> D["🧬 Dedupe<br/>past reports +<br/>OSV mirror"]
    D --> E["📦 Sandbox<br/>gVisor · no network<br/>claimed ref + HEAD"]
    E --> F["✍️ Verdict<br/>evidence bundle,<br/>ed25519 signed"]

Loading

Stage	What it does
Intake	Receives reports from GitHub private vulnerability reporting, piped .eml messages, or a local file.
Extract	Pulls out concrete claims: file paths, pkg.Func names, file.go:123 lines, versions, commit SHAs, PoC code. Deterministic parsing first; an optional LLM fills gaps with strictly validated JSON.
Ground	Checks out the claimed commit and verifies each claim with git and Go's own parser. Does http2.parseHeader actually exist at a3f9c1?
Dedupe	Compares against earlier reports (fingerprints + embeddings) and a local mirror of the OSV Go advisory database.
Sandbox	Builds and runs the PoC under gVisor with no network, hard CPU/memory/PID limits, and a control run at HEAD.
Verdict	Writes a short, factual verdict plus an evidence bundle signed with ed25519, so anyone can check it wasn't altered.

Built today: intake from a local file, a raw email message, or one GitHub advisory at a time; extract, ground, dedupe, and signed verdicts. Not built yet: the sandbox, which is what will reproduce a PoC. See Roadmap.

Verdicts

Outcome	Meaning
🟥 REPRODUCED	The PoC triggered the claimed behavior at the claimed commit.
🟧 REPRODUCED_FIXED_AT_HEAD	Reproduces at the claimed commit but not at HEAD. Probably already fixed.
⬜ NOT_REPRODUCED	The PoC ran cleanly and the claimed behavior was not observed.
🟪 GROUNDING_FAILED	A hard claim (a file or function) provably doesn't exist at the claimed commit.
🟦 LIKELY_DUPLICATE	Strong match to an earlier report or a published advisory. Shown alongside the repro result.
🟨 NEEDS_INFO	No runnable PoC, or no identifiable commit or version.
⬛ INCONCLUSIVE	Build failure, timeout, sandbox error, or anything ambiguous.

Important

Kritolith is built never to wrongly reject a real report. Line numbers drift, so a wrong line alone never causes GROUNDING_FAILED. When the evidence is unclear, the answer is INCONCLUSIVE, not a rejection.

Design principles

• Deterministic checks decide. The LLM only extracts claims and drafts replies. Everything it produces is re-verified against git or the sandbox.
• Every report is hostile input. Report text, PoC code and attachments are assumed to contain prompt injection or sandbox escape attempts. The threat model says what is done about it at each stage.
• Evidence, not opinions. Each claim records what was found, where, and at which commit.
• One static binary. Standard library first. The only third-party Go module is a pure-Go SQLite driver. No vendor SDKs.

Privacy and LLMs

Security reports describe unfixed vulnerabilities, so Kritolith is local-first:

• It works with no LLM at all, using deterministic extraction only.
• It supports local models through any OpenAI-compatible server: Ollama, llama.cpp, vLLM, LM Studio.
• Cloud providers (OpenAI, Anthropic, Gemini) are an explicit per-project opt-in, and every cloud call is logged with the report ID.
• No telemetry. Report content never appears in logs above debug level.

Commands

Command	Status	What it does
kritolith check --repo owner/name [--ref sha] [--poc dir] [--json] report.md	works	Verify one report file.
kritolith check --repo owner/name [--ref sha] --ghsa GHSA-xxxx-xxxx-xxxx	works	Verify a report filed through GitHub private vulnerability reporting, fetched with your own token.
kritolith ingest --eml --repo owner/name [--ref sha] < message.eml	works	Verify a report from a raw email message. Attachments are kept as PoC files.
kritolith keygen	works	Create the ed25519 key that signs verdicts. Once it exists, every verdict is signed.
kritolith check ... --bundle file.tar (also ingest)	works	Also write the signed evidence bundle: the verdict and a detached signature.
kritolith check ... --notify (also ingest)	works	Also email the verdict, with its bundle if signed, to the project's notify addresses.
kritolith verify-bundle [--pubkey ed25519:...] bundle.tar	works	Check a bundle's signature and print the verdict it carries.
kritolith serve --config kritolith.json [--notify]	works	Receive GitHub webhooks and verify each new private report as it is filed.
kritolith osv sync	works	Build or refresh the local mirror of the OSV Go advisories used for duplicate detection.
kritolith eval [--corpus dir] [--case id]	works	Run the eval corpus, or chosen cases, and print the scoreboard.
kritolith version	works	Print the version.

Reports from GitHub private vulnerability reporting

A private report on GitHub is a repository security advisory in the triage state. Kritolith can read one directly:

export GITHUB_TOKEN=...   # or GH_TOKEN; never passed as a flag
kritolith check --repo owner/name --ghsa GHSA-xxxx-xxxx-xxxx --ref v1.4.2

• The token must be able to read the repository's security advisories: a classic token with the repo or repository_advisories scope, or a fine-grained token with read access to "Repository security advisories". GitHub shows unpublished reports only to repository administrators, security managers and collaborators on the advisory.
• Kritolith makes one GET to api.github.com (or --github-api-url for GitHub Enterprise Server), refuses redirects, and stores the report in the local data directory like any other. Nothing is written back to GitHub.
• An advisory names versions, not a commit. Pass --ref if you know the affected tag or commit. Without it the verdict is NEEDS_INFO, with the claims still checked against any affected release named on the advisory.
• This has been tested against published advisories. It has not yet been run against a report still in triage; GitHub documents the same endpoint for both.

Reports that arrive by email

kritolith ingest --eml --repo owner/name < message.eml

Works with anything that can hand over a raw message: fetchmail, procmail, a Postfix pipe, or "Show original" saved from a mail client. The report text is the plain-text body (or the HTML body with its markup removed), the subject becomes the title, and attachments are stored as PoC files without being run. The sender's address is not stored. With --config naming exactly one project, --repo can be left out.

Run as a service

kritolith serve listens for GitHub's repository_advisory webhook, and when someone files a private report on one of your projects it fetches the report, checks it and (with --notify) emails you the verdict.

{
  "projects": [{ "repo": "owner/name", "notify": ["security@example.org"] }],
  "github": { "webhook_secret_env": "KRITOLITH_WEBHOOK_SECRET", "token_env": "GITHUB_TOKEN" },
  "smtp": { "addr": "smtp.example.org:587", "from": "kritolith@example.org" }
}

export KRITOLITH_WEBHOOK_SECRET=...   # the secret you set on the webhook, 16+ characters
export GITHUB_TOKEN=...               # can read the projects' security advisories
kritolith serve --config kritolith.json --notify

On GitHub, add a webhook to the repository (or organization) with the payload URL https://your-host/webhook/github, content type application/json, your secret, and the "Repository advisories" event.

• It listens on 127.0.0.1:8787 by default and speaks plain HTTP. Put a reverse proxy that terminates TLS in front of it.
• Every delivery must carry a valid X-Hub-Signature-256. Unsigned or wrongly signed requests get a 401 and nothing else happens.
• A webhook only says which report to look at. The report is fetched from GitHub's API with your token, so nothing in a webhook body is acted on.
• Reports are queued in the database and processed one at a time. GitHub's redeliveries are ignored, failures are retried with a growing delay (one minute up to two hours), and a job interrupted by a restart is picked up again.
• A GitHub App is not required: a webhook and a token are enough. This has been tested end to end against a local stand-in for GitHub, not yet against github.com.

Signed verdicts

kritolith keygen                                   # once; prints the public key
kritolith check --repo owner/name --ref v1.4.2 --bundle verdict.tar report.md
kritolith verify-bundle --pubkey ed25519:... verdict.tar

After keygen, every verdict is signed with ed25519. A bundle is a small tar file with verdict.json and verdict.sig. It says which report the verdict is about (ID, repository, commit, title and a SHA-256 of the report text) but does not contain the report, so it can be passed on without passing on an embargoed report. verify-bundle takes the public key from you, never from the bundle, and prints nothing from a bundle whose signature fails.

A signature shows that the verdict came from the holder of that key and has not been changed. It does not make the verdict right.

Emailing verdicts

Add an smtp section and notify addresses to the config, then pass --notify:

{
  "projects": [{ "repo": "owner/name", "notify": ["security@example.org"] }],
  "smtp": { "addr": "smtp.example.org:587", "from": "kritolith@example.org",
            "username": "kritolith", "password_env": "KRITOLITH_SMTP_PASSWORD" }
}

kritolith ingest --eml --config kritolith.json --notify < message.eml

Mail is sent only when --notify is given. Kritolith requires TLS (STARTTLS, or TLS from the start on port 465) for any server that is not on the same machine, and will not send the verdict or the password without it. If delivery fails the verdict is still stored and printed, and the command exits non-zero.

An LLM is optional and off by default. To use one for claim extraction and embeddings, copy kritolith.example.json and pass it with --config.

Requirements today: git and the Go toolchain. The sandbox, when it lands, will need Linux with gVisor (runsc).

Scope

Version 1 deliberately stays narrow.

✅ In v1	🚫 Not in v1
Go projects	Other languages
GitHub private reports, .eml, CLI intake	HackerOne / Bugcrowd integrations, IMAP
Grounding, dedupe, sandbox repro	LLM-based exploitability scoring
Signed evidence verdicts	CSAF / VEX / CRA reporting
Self-hosted single binary	Web dashboard, hosted multi-tenant service

Roadmap

• Skeleton: core types, SQLite store, kritolith check end to end
• Claim extraction + pluggable local-first LLM layer
• Grounding against git and the Go AST
• Deduplication with embeddings and a local OSV mirror
• gVisor sandbox with hostile-PoC hardening
• Intake from email (ingest --eml) and from a GitHub security advisory (check --ghsa)
• Signed verdicts and evidence bundles (keygen, --bundle, verify-bundle)
• Email delivery of verdicts (--notify)
• Webhook server for GitHub private reports (serve)
• Public threat model (docs/threat-model.md) and fuzz tests for every parser of hostile input
• Install docs, first release

Where it stands

Progress is measured on a public eval corpus in testdata/corpus: 24 real Go advisories and 20 deliberately fabricated reports. make eval prints the scoreboard. Results on 2026-11-11, with no LLM configured:

Target	v1 goal	Now
Real reports wrongly marked GROUNDING_FAILED	0	0 of 24
Real reports wrongly marked LIKELY_DUPLICATE	0	0 of 24
Fabricated reports with fake code caught by grounding	≥ 80%	9 of 15 (60%)
Duplicate top-1 accuracy	≥ 80%	3 of 3
Real reports with runnable PoCs reproduced	≥ 60%	0 of 20 with a PoC: the sandbox is not built

The six fabricated reports that are not caught cite an invented package-level function by bare name, or an invented file by bare name (remote_fetch.go rather than internal/remote_fetch.go). Kritolith answers INCONCLUSIVE for those and shows "not found" in the evidence, instead of risking a wrong rejection: real reports name functions and files from the standard library, a dependency or an upstream project all the time. What it does reject is a claim tied to this repository: a method on one of its types, a file in one of its directories, a package under its own module path. Overall, 14 of 44 cases get exactly the expected verdict; the 24 real advisories are INCONCLUSIVE today because their expected verdict needs the sandbox.

Most corpus reports are written for the corpus, so the build is also run over the text of published security advisories, fetched with kritolith check --ghsa: 200 advisories from 35 Go projects at the last count. Those runs found four ways a genuine advisory could be rejected, all fixed, and each is now a corpus case:

What the advisory did	What went wrong	Case
Linked to pkg.go.dev/...	pkg.go was read as a claimed file	ghsa-mh63-6h87-95cp
Cited a file in the upstream project by bare name	Treated as missing from this repository	ghsa-g6w5-3rhc-c379
Used markdown links with relative targets (/caddy/admin.go#L719)	The target was read as a file path	ghsa-gx7w-56w6-g48x
Told the maintainer to save a new test file	The file "did not exist"	ghsa-9pmm-cxww-rrr7

The second fix is why four fabricated reports that cite a file by bare name are no longer rejected.

How this project is built

Kritolith exists because of AI-generated noise in security inboxes, so it is fair to ask how it was made. AI coding assistants wrote much of the code, tests and documentation here, under the maintainer's direction and review. The design notes and implementation plans they worked from are in docs/superpowers, and CLAUDE.md is the brief they are given.

None of that is part of what the tool decides. A verdict comes from git, Go's parser and fixed rules; with no LLM configured, which is the default, no model is involved at any point. You don't have to take the results on trust either: the eval corpus and make eval reproduce the scoreboard above, and the threat model lists what is known to be weak.

Contributing

Contributions are welcome. The easiest useful one needs no knowledge of the code: add a case to the eval corpus, a short report and the verdict it should get. Read CONTRIBUTING.md first. All commits need a DCO sign-off (git commit -s).

This project follows the Contributor Covenant.

Security

Found a vulnerability in Kritolith itself? Don't open a public issue. Please report it privately. See SECURITY.md.

License

Apache License 2.0