Skip to content

[Roadmap] Numerical determinism, parallelism alignment, and agentic debugging #1

Open
0 of 3 issues completed

Description

@ziming-zh

Direction

Extend OpGuard from paired bitwise investigations toward numerical behavior analysis, alignment across distributed execution layouts, and flexible agent-driven debugging for training and inference.

These are proposed workstreams, not shipped capabilities or date commitments. Detailed scope and acceptance criteria live in the feature issues.

Feature tracks

  1. Numerical determinism — characterize numerical repeatability and deviations using lightweight tensor summaries, context-aware historical baselines, operator constraints, and targeted validation.
  2. More complete alignment support across parallelisms — map logical tensor boundaries across PP/DP and especially TP/CP/SP, with explicit framework adapters and partitioned-execution simulation where necessary.
  3. A more seamless agentic debugging experience — let agents compose bounded capture, local alignment, investigation, replay, and evidence/history tools without repeating an entire fixed workflow.

Suggested sequence

  • Foundation: define observation timing, logical tensor identity, numerical comparison criteria, evidence provenance, and explicit unsupported/inconclusive outcomes.
  • First useful slices: a fixed-model inference numerical profile; a divisible TP=1 versus TP=2 alignment example; and a bounded agent investigation using existing evidence plus selective new capture.
  • Broader coverage: CP/SP and framework adapters, training baselines, dynamic capture policies, and durable investigation history.
  • End-to-end evaluation: demonstrate discovery, localization, and discriminating validation on public or synthetic failures, with benign changes as controls.

The first slices can progress independently once their shared contracts are agreed. More complex topology and dynamic-runtime support should build on measured, inspectable examples.

Shared acceptance principles

  • Preserve the distinction between semantic correspondence, recorded-field agreement, numerical tolerance, statistical anomaly, and a supported causal hypothesis.
  • Record inputs/state, runtime settings, mappings, transformations, capture policy, and experiment provenance needed to interpret results.
  • Keep workload numerical settings unchanged by default; record any intentional intervention.
  • Report missing coverage, insufficient history, invalid mappings, failed captures, and inconclusive experiments explicitly.
  • Measure false alarms, localization quality, capture overhead, storage, and validation/replay cost.
  • Retain the current paired bitwise workflow and compatibility with existing artifacts.

Contribution workflow

Use the OpGuard feature proposal form for concrete slices once its template PR is merged. Include the debugging problem, expected user/agent behavior, relevant workload/topology, evidence contract, first milestone, and acceptance cases.

Open questions remain in the individual feature issues so framework choices, thresholds, hot-attachment feasibility, and implementation details can evolve with evidence.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions