Skip to content

fix(cli): reject invalid check numeric flags as usage errors - #7673

Merged
miguel-heygen merged 3 commits into
heygen-com:mainfrom
bardrop:fix/check-numeric-flag-usage-errors
Nov 13, 2026
Merged

miguel-heygen merged 3 commits into
heygen-com:mainfrom
bardrop:fix/check-numeric-flag-usage-errors

Conversation

@bardrop

@bardrop bardrop commented Nov 13, 2026

Copy link
Copy Markdown
Contributor

What

hyperframes check now rejects a bad --samples, --max-issues, --max-transition-samples or --tolerance value as a usage error (exit 2, usage_error under --json) instead of quietly running with the default.

Why

An invalid value was silently replaced with the default, so the run looked fine but used other settings than the user asked for. parseInt also read --samples 5abc as 5 and parseFloat read --tolerance 2px as 2.

Related work

Fixes #7672

How

positiveInteger and nonNegativeNumber in packages/cli/src/commands/check.ts now take the args and the flag name:

  • flag absent → the default, as before
  • otherwise the value is parsed with Number() (empty string → NaN), so trailing junk is rejected
  • not an integer >= 1 (or not a finite number >= 0 for --tolerance) → CliUsageError("Invalid --<flag>: \"<value>\"; use a positive integer"), same style as Invalid --range

The thrown error goes through the existing reportCheckFailure path, so --json gets the usage_error envelope and the exit code is 2.

Behavior notes:

  • --samples 2.5 is now an error (it was read as 2).
  • --max-transition-samples 0 is now an error. Before, 0 fell into the "no cap" branch by accident; leaving the flag out still means no cap.
  • --tolerance 0 is still valid.

Test plan

  • Unit tests added/updated
    • check.test.ts: 9 cases (--samples 0|ten|5abc|2.5, --max-issues -5, --max-transition-samples abc, --tolerance -1|px|2px) expect exit 2, usage_error, a message naming flag and value, and that the pipeline never runs. All 9 fail on main.
    • check.test.ts: valid values (--samples 3 --max-transition-samples 4 --tolerance 0.5, --tolerance 0) reach the pipeline, and absent flags keep their defaults.
    • cli.agentMode.test.ts: check --samples ten and check --tolerance=-1 added to the "flag checks a command makes itself" table (spawned CLI, exit 2 + usage_error). Both fail on main.
    • vitest run src/commands/check.test.ts: 191 passed. Agent-mode flag checks group: 54 passed.
    • oxlint and oxfmt --check clean on the changed files; tsc --noEmit shows no errors in them.
  • Manual testing performed
  • Documentation updated (if applicable)
  • Comments follow CONTRIBUTING.md "Comments": they say why, not what, and a bug fix says what the code must do and how to reproduce the bug

--samples, --max-issues, --max-transition-samples and --tolerance fell
back to their defaults on any value they could not parse, and parseInt
read 5abc as 5. A bad value now exits 2 with a usage_error that names the
flag and the value. Absent flags keep their defaults.

@miguel-heygen miguel-heygen left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review at 5e5104a6. There were no other reviews when I posted. I reviewed fc4047b7 in full. 5e5104a6 only renames numericFlag to wholeStringNumber and drops the added comment (git diff fc4047b7 5e5104a6), so everything below still holds.

What's right

  • Strict parsing. check.ts:741: wholeStringNumber parses with Number() after a trim and maps an empty string to NaN. So 5abc, 2px and ten are rejected, where they used to be read as 5, 2 and the default.
  • Integer check. positiveInteger (:721) requires Number.isInteger(parsed) && parsed >= 1, so --samples 2.5 fails instead of being read as 2.
  • Error path. The thrown CliUsageError goes through the existing reportCheckFailure path, so the exit code (2) and usage_error match Invalid --range.
  • Defaults are safe.
    • samples, max-issues and tolerance declare string defaults ("9", "80", "2", :189-233), so an absent flag still parses to the default.
    • max-transition-samples has no default, so absent is still undefined, which means no cap.
    • The body's "--max-transition-samples 0 is now an error; absent means no cap" matches :480,487.
  • Tests. The agent-mode rows (cli.agentMode.test.ts:236-237) pin exit 2 and the usage_error envelope through a spawned CLI.

Tests and mutation checks (check.test.ts: 191 pass at both heads, matching the body; mutations run at fc4047b7)

  • The number parser → parseFloat → 2 fail (5abc, 2px).
  • parsed < 1 → parsed < 0 → the --samples 0 case fails.
  • Number.isInteger → Number.isFinite → the --samples 2.5 case fails.
  • --tolerance: < 0 → <= 0 → the "--tolerance 0 stays valid" case fails.
  • Two mutations survive (nits below):
    • dropping the empty-string guard in wholeStringNumber;
    • Number.isFinite → !Number.isNaN for --tolerance.

Nits (non-blocking)

  • Two untested edges on --tolerance.
    • --tolerance= (empty) and --tolerance Infinity aren't covered. Without the text === "" guard, Number("") is 0 and --tolerance= would silently mean zero tolerance. Without isFinite, Infinity would be accepted.
    • Both are handled today. Two more rows in the invalid-values table would pin them.
  • The warm-check daemon can't catch this throw (by reading the code, not tested).
    • warmCheck.ts:137 calls void prelaunchBrowserForCheck(checkArgs).catch(() => undefined). But prelaunchBrowserForCheck (check.ts:510) isn't async, so a throw from parseCheckOptions escapes before .catch is attached.
    • So the detached warm process that bin/hyperframes.mjs:25 spawns after an eligible run with bad flags exits, and the user's corrected next run starts cold.
    • Nothing user-visible breaks (stdio is ignored), and Invalid --range already takes this path. This PR only widens it. Making prelaunchBrowserForCheck async would close it.

CI

  • This is a fork PR, so every workflow was waiting on maintainer approval. I approved the runs for both heads after reading the diff: three files, nothing under .github/ or build scripts.
  • On 5e5104a6, every required check that ran is green: Build, Typecheck, Test, Test reachability, Test: runtime contract, regression, the Windows tests and render, captures and Semantic PR title. Skills: manifest in sync and Studio: edit accuracy gate were skipped because they aren't affected.
  • Comments was red at fc4047b7 because of the one added comment line. 5e5104a6 drops it, so the check is green at this head.

Verdict: APPROVE
Reasoning: Each invalid value now fails as a usage error through the existing envelope, the defaults are unchanged, and the tests catch each mutation of the parsing rules except two minor edges. Required CI is green on this head.

@miguel-heygen
miguel-heygen added this pull request to the merge queue Nov 13, 2026
Make prelaunchBrowserForCheck async so a usage error from
parseCheckOptions becomes a rejection that warmCheck's .catch handles,
instead of a synchronous throw that ends the warm process. Pin
--tolerance= and --tolerance Infinity as usage errors.
@bardrop

bardrop commented Nov 13, 2026

Copy link
Copy Markdown
Contributor Author

Thanks for the careful review and the mutation runs. I took both nits in a18b9f7:

  • --tolerance edges: added --tolerance= (empty) and --tolerance Infinity to the invalid-values table, so the empty-string guard and the isFinite check are both pinned now.
  • Warm daemon: prelaunchBrowserForCheck is now async, so a bad flag turns into a rejection that warmCheck.ts:137's .catch handles, not a sync throw that ends the warm process. New test: calling it with --samples 0 doesn't throw synchronously and the promise rejects with the usage error. It fails on the previous head (sync throw) and passes now.

check.test.ts 194 pass; oxlint, oxfmt and the comment ratchet are clean. The fork CI will need another approval for this head, sorry about that.

Merged via the queue into heygen-com:main with commit 564a597 Nov 13, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

cli: check silently replaces invalid --samples/--max-issues/--tolerance with defaults instead of exiting 2

2 participants