# Contributing: evolving the schema ## What you're changing The data contracts the report renderers, Python API, and MCP tool surface all share. PerfXpert has **three** distinct schema layers — changes usually touch at least two of them, and a change to the JSON contract requires a `schema_version` bump. ## The three layers ### Layer A — Agent I/O (Pydantic) `perfxpert/agents/schemas.py` defines the immutable handoff schemas between agents: - `RootInput` / `RootOutput` — top-level orchestrator. - `AnalysisInput` / `AnalysisOutput` — deterministic analysis agent. - `RecommendationInput` / `RecommendationOutput` — recommendation agent. - `CorrectnessInput` / `CorrectnessOutput` — 5-gate correctness cascade. - `ComputeSpecialistInput` / `ComputeSpecialistOutput`, `MemorySpecialistInput` / `MemorySpecialistOutput`, `LatencySpecialistInput` / `LatencySpecialistOutput`, `DiffSpecialistInput` / `DiffSpecialistOutput` — Layer-2 experts. `DiffSpecialistOutput` packs its per-kernel deltas into a single `kernel_deltas: Dict[str, List[Dict]]` field with `"regressions"` / `"improvements"` keys so the 5-field output cap still holds; the MCP wrapper (`perfxpert.tools.agents.diff.agent_diff_specialist`) flattens them back to top-level keys for the public wire shape. All models inherit from `_FrozenModel` (`frozen=True, extra="forbid"`) and are subject to CI-enforced field caps: **≤10 input fields, ≤5 output fields** (see `tests/test_agents/test_schema_field_caps.py`). ### Layer B — Deterministic payload (dict) `perfxpert/analysis/payload.py::build_analysis_payload` returns a plain dict that every formatter consumes: ```python # SKIP-SAMPLE — illustrative shape returned by build_analysis_payload { "database_path": "...", "time_breakdown": {...}, # kernel/memcpy/overhead %, ns totals "hotspots": [...], # list of kernel dicts "memory_analysis": {...}, # per-direction bandwidth + counts "hardware_counters": {...}, # {has_counters, metrics, counters} "kernel_resources": {...}, # VGPR/SGPR/LDS/scratch + occupancy "api_overhead": {...}, # per-API totals + launch overhead "warmup_issues": {...}, # outlier first-call detection "thread_trace": {...} | None, # Tier-3 ATT "tier0_findings": {...} | None, # source scanner output "recommendations_deterministic": [...], # rule-driven recs "metadata": {...}, } ``` ### Layer C — External JSON doc `perfxpert/formatters/json_fmt.py::_format_as_json` serialises Layer B into the public JSON document, stamped with a top-level `schema_version` field. The full additive chain is **`0.3.0` → `0.3.1` → `0.3.2` → `0.3.3` → `0.3.4` → `0.4.0`** — each step advances as richer data becomes available (see `# CHANGES` below and the "Schema versioning policy" table). Tier-3 ATT pins to `0.4.0` regardless of which other sections are present. `perfxpert/analyze.py::_format_agentic_output` (~line 790) then overlays the agentic brain (`narrative`, `primary_bottleneck`, `warnings`, `tier0_findings`) and re-bumps the version if it was still at `0.1.0` / `0.2.0`. The three layers form a pipeline: ``` agents/schemas.py (Layer A) ──► payload.py (Layer B) ──► json_fmt.py (Layer C) Pydantic handoffs deterministic dict public JSON frozen, field-capped no LLM involvement schema_versioned ``` ## How to evolve an agent schema (Layer A) Example: add `gfx_id: str` to `AnalysisOutput` so downstream agents don't re-derive it. 1. **Extend the model** in `perfxpert/agents/schemas.py`: ```python # SKIP-SAMPLE — illustrative Pydantic field addition class AnalysisOutput(_FrozenModel): primary_bottleneck: BottleneckType confidence: float = Field(..., ge=0.0, le=1.0) time_breakdown: Dict[str, float] hot_kernels: List[Dict[str, Any]] counter_data_available: bool gfx_id: str = "" # NEW — check field cap (≤5 outputs). ``` Watch the `≤5 output fields` cap — the CI test in `tests/test_agents/test_schema_field_caps.py` will fail the build if you exceed it. 2. **Populate it** in the agent's runner — `perfxpert/agents/analysis.py` (or wherever `run_analysis` lives): ```python # SKIP-SAMPLE — illustrative return AnalysisOutput( primary_bottleneck=bottleneck, confidence=conf, time_breakdown=tb, hot_kernels=hk, counter_data_available=has_counters, gfx_id=detect_gfx_id(connection), ) ``` 3. **Thread it downstream.** Either carry through `RootOutput.metadata` (untyped escape hatch, no field-cap cost) or extend a specialist input schema (typed, cap-bound). Both options require test coverage. 4. **Render it in all four formatters** — text, markdown, json_fmt, webview. Skipping a format here creates format parity drift (Layer B has the data, but Layer C loses it on render). The reviewer checklist at the bottom of this doc catches this. 5. **Document in the Python API.** `docs/guides/python-api.md` exposes `perfxpert.api.agent_analysis` — the field surfaces there automatically via Pydantic, but the docstring / field table must be updated by hand. 6. **MCP tool discovery is automatic.** The 8 agent tools (`perfxpert_agent_`) read their JSON schema from the Pydantic model via reflection — no manual MCP descriptor edit is needed. Verify with `perfxpert-mcp --describe` that the new field appears in the tool-list output. 7. **Bump `schema_version`** only if the change reaches Layer C (JSON doc). A field added to `AnalysisOutput` that's consumed internally but never serialised into `_format_as_json`'s output does NOT require a bump. See the versioning policy below. ## How to add a new deterministic-payload section (Layer B) Example: `thread_trace` was added this way. 1. **Add the key to `build_analysis_payload`** in `perfxpert/analysis/payload.py` — populate it inside the `payload: Dict[str, Any] = {...}` literal at ~line 353, and attach a best-effort computation branch with try/except so a missing data source never fails the whole pass: ```python # SKIP-SAMPLE — illustrative new payload section payload["my_new_section"] = {} if optional_precondition_met: try: payload["my_new_section"] = analyze_my_new_section(connection) except Exception: payload["my_new_section"] = {"has_data": False, "reason": "..."} ``` 2. **Render in all four formatters.** - `text.py` — a banner + tabular block inside `format_analysis_output`. - `markdown.py::_format_as_markdown` — an `##` section. - `json_fmt.py::_format_as_json` — a top-level key on the JSON doc (rename if the external contract name differs from the payload key). - `webview.py::_format_as_webview` — a `
` block; wire the template in `perfxpert/formatters/templates/webview.html`. 3. **Assert presence + ordering** in `tests/test_formatters/test_report_structure.py` — the existing four tests show the pattern (`find()` indices + `assert i_a < i_b`). 4. **Document the new key** in `docs/guides/getting-started.md` "Report structure" subsection (§4 of that guide) so users know to look for it. ### Pure-rendering derivatives (no schema change) Some webview visualisations render directly from an existing deterministic-payload key and do **NOT** introduce a new payload section. Examples: - **ATT flame graph** — `perfxpert/formatters/_att_flamegraph.py` consumes the existing `payload["thread_trace"]` block (`kernels`, `top_stalling_instructions`, `stall_category`) and produces an inline SVG for the Thread Trace `.scard` in the webview. The flame graph is webview-only; the Markdown / text / JSON formats already surface the same data via the stall table and the `thread_trace` JSON key, so no Layer-A / Layer-B / Layer-C schema change is required. The renderer is a pure function of the payload — no new MCP tool, no new agent, no new dependency. When adding a pure-rendering derivative, only wire the renderer into `webview.py::_format_as_webview` (via a new `%%…_SECTION%%` template slot inside the existing scard) and add a unit test under `tests/test_formatters/` that asserts the rendered fragment shape on a synthetic payload. Do NOT bump `schema_version` — the versioning policy below only applies to changes in the external JSON document (Layer C). ## Schema versioning policy (Layer C) The JSON doc carries a top-level `schema_version` field. Consumers MUST check it before parsing. Bumps follow semver-lite: | Change | Bump | |--------|------| | Breaking: field renamed, field removed, semantics changed | **Major** (`0.x.y` → `1.0.0`) | | Additive: new top-level key, new sub-field under existing key | **Minor** (`0.3.0` → `0.4.0`) | | Bugfix only: value format corrected, no key-set change | **Patch** (`0.3.0` → `0.3.1`) | The current tree: - `0.1.0` — pre-TraceLens baseline (trace-only reports). - `0.2.0` — tier-0 source-scanner addition (used by `_format_tier0_json`). - `0.3.0` — agentic brain (`narrative`, `primary_bottleneck`, `warnings`) + tier-0 separation + summary section. - `0.3.1` — additive `hotspots[i].source_locations` field cross-referencing each hotspot with its Tier-0 definition + launch site (Confluence row #5) — each entry now also carries `severity` + `severity_label` (HIGH/MEDIUM/LOW/INFO; CRITICAL/HOT/WARM/COOL). Also additive: top-level `trace_diff` section emitted by `perfxpert diff` / `perfxpert analyze --baseline` (Confluence row #7). - `0.3.4` — current: additive top-level `roofline` section emitted by `roofline.plot_points` (Live Roofline chart). Shape: ```json "roofline": { "schema_version": "0.3.x", "arch": "gfx942", "arch_peaks": {"fp32": 1.634e14, "fp16": 1.307e15, "bf16": 1.307e15, "fp64": 8.17e13, "fp8": 2.614e15, "int8": 2.614e15}, "hbm_bandwidth_bytes_per_s": 5.3e12, "dtype": "bf16", "dtype_confidence": "from_kernel_name", "kernels": [ { "name": "gemm_bf16_kernel", "ai": 48.2, "achieved_flops_per_s": 1.12e14, "flops": 9.6e13, "bytes": 2_000_000_000, "duration_ns": 860_000, "duration_pct": 94.3, "fp_type": "bf16", "bottleneck_class": "compute", "confidence": "high" } ], "ridge_point": {"ai": 30.8, "flops_per_s": 1.634e14} } ``` Populated only when Tier-2 hardware counters are available in the trace DB. Drives the webview's **Live Roofline** `.scard` (log-log inline SVG). `_format_as_json` bumps `schema_version` from `0.3.3` -> `0.3.4` when the key is set. ATT (`0.4.0`) still trumps. - `0.3.3` — additive change-impact-prediction fields on each recommendation (`predicted_impact_range`, `predicted_confidence`, `predicted_rationale`, `source_citation`, `roofline_delta`). Emitted by the change-impact-prediction pipeline (`perfxpert.tools.predict_impact`) whenever the specialist-attached or category-mapped technique matches an entry in `knowledge/change_impact_models.yaml`. Shape of the recommendation entry when a prediction fires: ```json { "id": "ROCPD-COMPUTE-001", "priority": "HIGH", "category": "Compute-Bound Kernel", "issue": "...", "suggestion": "...", "predicted_impact_range": [1.15, 1.45], "predicted_confidence": 0.70, "predicted_rationale": "Technique 'vgpr_reduction' applied to ...", "source_citation": "knowledge/proven_optimizations.yaml#vgpr_reduction_compute_bound", "roofline_delta": {"before": null, "after": null} } ``` Rules enforced by `predict_impact.predict_change_impact`: Amdahl guard (kernel < 5% runtime ⇒ no emission), tier-2 gate (missing counters ⇒ no emission), conservative bracket (`hi = catalog_hi × 0.85`), and every rec cites the seed entry in `knowledge/proven_optimizations.yaml` via `source_citation`. `_format_as_json` bumps `schema_version` from `0.3.2` → `0.3.3` when any rec in the document carries `predicted_impact_range != null`. `0.3.4` (Live Roofline) and ATT (`0.4.0`) still trump when their own fields are populated. - `0.3.2` — additive top-level `communication` section emitted by `rccl_analysis.analyze_collectives` (RCCL / NIC communication analysis). Shape: ```json "communication": { "collectives": [ { "op_type": "AllReduce", "msg_bytes": 1048576, "duration_ns": 1000000, "effective_bw_gbps": 1.57, "peak_bw_gbps": 340.0, "efficiency_pct": 0.46, "efficiency_label": "poor", "overlap_ratio": 48.5, "algo_hint": "Ring", "topology_hint": "intra-node", "regime": "algo-dependent", "ranks": 4 } ], "summary": { "op_count": 1, "ranks": 4, "dominant_op": "AllReduce", "peak_bw_gbps": 340.0, "avg_bw_gbps": 1.57, "avg_efficiency_pct": 0.46, "overlap_pct": 48.5, "capture_incomplete": false } } ``` The key is absent (not `null`) when the trace contains no RCCL spans. When set, `_format_as_json` bumps `schema_version` from `0.3.1` -> `0.3.2`. ATT (`0.4.0`) still trumps: a trace with both ATT + RCCL data pins `0.4.0`. - `0.4.0` — bumped automatically by `_format_as_json` when `att_analysis.has_att_data=True` (Tier-3 ATT). The bumps in `_format_as_json` live at the end of that function; the overlay in `_format_agentic_output` (`perfxpert/analyze.py` ~line 790) only upgrades `0.1.0` / `0.2.0` → `0.3.0` by default and conditionally bumps to `0.3.1` when any hotspot carries `source_locations`, so a later ATT-driven bump to `0.4.0` is preserved. Preserve that ordering when you add a new minor / patch bump. ## Cross-format parity guarantee Every top-level key in the JSON output must have a visible representation in `text` + `markdown` + `webview`. If a section is structurally empty (`memory_analysis` on a compute-only trace, `hardware_counters` on a Tier-1 run), formatters render a graceful placeholder, **not** an error. The regression guard in `test_json_has_all_required_keys_and_bumped_schema_version` (`tests/test_formatters/test_report_structure.py`) plus the webview/markdown/text ordering tests enforce this. ## Recommendation subtype — `pragma` Recommendations now carry an optional `subtype` field in the Layer B dict (and Layer C JSON). The only defined value today is `"pragma"`, emitted exclusively by the LLVM loop-hint advanced-recommendations pathway: ```json { "priority": "MEDIUM", "subtype": "pragma", "category": "Loop Hint (clang_loop_unroll_count)", "issue": "...", "suggestion": "Apply `#pragma clang loop unroll_count(N)` at src/hot.hip:42", "actions": ["...", "Verify with: perfxpert diff -i .db -i .db"], "estimated_impact": "1.1x-1.5x", "source_file": "src/hot.hip", "source_line": 42, "pragma_id": "clang_loop_unroll_count", "factor_sweep": [2, 4, 8], "risk": "medium" } ``` Rendering invariants: * The text / markdown / webview formatters render an ``[advanced]`` badge next to the priority label for every rec with ``subtype == "pragma"``. * ``perfxpert analyze`` filters pragma recs out of the rendered output unless ``--advanced`` is passed or ``PERFXPERT_ADVANCED_RECS=1`` is set — the Layer-B payload still includes them (so the JSON-contract version is available to downstream consumers who want the raw list). * Every pragma rec MUST carry a Tier-0 source anchor (``source_file`` + ``source_line``); this is enforced by the fence slice in ``perfxpert/agents/fence/compute_specialist.md``. See ``perfxpert/analysis/recommendations.py::build_pragma_recommendation`` for the canonical constructor. ## Advanced-specialist tool shapes The advanced-specialist MCP tools (kernel fusion, GPU runtime monitor, unified memory, dependency graph) each return a plain dict; none carry `schema_version` because they sit alongside — not inside — the analyze-report JSON. Consumers that want schema-stable outputs should pin the tool's release notes in `docs/integration/mcp-server.md`. ### `kernel_fusion.find_fusion_candidates` (Feature A) Returns `List[Dict]` ranked by `est_speedup_hi` descending: ```json { "pair": ["add_kernel", "add_kernel"], "signature": "9b7a2e4c1ff0", "gap_ns": 120, "est_speedup_lo": 1.08, "est_speedup_hi": 1.25, "confidence": 0.78 } ``` ### `gpu_runtime_monitor.*` (Feature B) `parse_amd_smi_json` / `parse_rocm_smi_json` return a normalised envelope: ```json { "source": "amd-smi", "samples": [ {"gpu_id": 0, "temp_c": 72.5, "power_w": 410.0, "sclk_mhz": 1950, "mclk_mhz": 1600, "gfx_busy_pct": 96} ], "gpu_count": 1 } ``` `analyze_thermal` consumes the envelope and returns: ```json { "max_temp_c": 72.5, "avg_temp_c": 71.2, "p95_temp_c": 72.5, "max_power_w": 410.0, "throttle_events": 0, "power_throttle_suspected": false, "thermal_headroom_c": 32.5, "verdict": "healthy" } ``` ### `unified_memory.analyze_paging` (Feature C) ```json { "paging_events": 3, "cross_die_bytes": 2147483648, "page_faults": 0, "estimated_penalty_ns": 1006632960, "recommendations": ["Pin host-shared buffers ...", "..."] } ``` ### `dependency_graph.reconstruct_dag` (Feature D) ```json { "nodes": [{"id":"k0","name":"matmul","start_ns":0,"end_ns":5000,"duration_ns":5000,"stream":0}], "edges": [{"from":"k0","to":"k1","kind":"stream_order"}], "critical_path": ["k0", "k1"], "bubbles": [{"start":5000,"end":15000,"cause":"idle_gap_stream_0","duration_ns":10000}], "total_bubble_ns": 10000, "sync_event_count": 2 } ``` ## Roofline payload The `roofline` top-level key (added in `schema_version: 0.3.4`) carries the per-kernel live-roofline points produced by `perfxpert.tools.roofline.plot_points`. Full shape and bumping-semantics are documented in the Layer-C version-table above (see **0.3.4** under "Schema versioning policy"). Key points: - Populated only when Tier-2 hardware counters (`SQ_INSTS_VALU` / `FETCH_SIZE` / `WRITE_SIZE`) are available. Absent otherwise (formatters skip the section gracefully). - `arch_peaks` is in FLOPs/s (not TFLOPs) — the formatter divides. - `ridge_point.ai` = dominant-dtype peak / HBM bandwidth (FLOPs/Byte); `ridge_point.flops_per_s` = dominant-dtype peak. - `kernels[i].confidence` is `"low"` when only one side of the byte pair (FETCH_SIZE / WRITE_SIZE) was collected — the other is assumed 0, so AI is an upper bound. - The webview formatter renders the Live Roofline as an inline SVG `
`; the Markdown / text formats render a table with columns (kernel, AI, achieved GFLOPs/s, regime, dtype, confidence); the JSON format is a passthrough of this shape. ## Tier-0 carve-out `tier0_findings` is conditionally included — it's only present when `--source-dir` was supplied. Even then, the instrumentation-advice sub-fields are conditionally **stripped** when a DB is also present (`has_profiling=True`). The exact keys stripped, from `perfxpert/analyze.py` ~line 776: ```json { "suggested_counters": "...", "profiling_plan": "...", "profiling_plan_actions": "...", "suggested_first_command": "..." } ``` The rationale: in combined mode (`-i` + `--source-dir`) the user already has profiling data. Suggesting they run `rocprofv3` again with a hand-picked counter set is noise — the Analysis agent uses the existing DB instead. In source-only mode those sub-fields are the whole point of Tier 0 output, so they stay. Every `_format_tier0_` accepts `has_profiling: bool = False` and gates those sub-fields internally. New tier-0 formatters MUST honour the same gate. ## Reviewer checklist - [ ] Agent-schema change respects field caps (≤10 input, ≤5 output per model) — CI test `tests/test_agents/test_schema_field_caps.py` passes. - [ ] Agent-schema change populated in the agent's `run_*` function, not just declared on the model. - [ ] Rendered in **all four** built-in formatters (text, json_fmt, markdown, webview) OR explicitly documented as internal-only. - [ ] Python-API docstring updated if the field is agent-output (reflected through `perfxpert.api.agent_`). - [ ] `docs/guides/getting-started.md` "Report structure" subsection mentions the new key if it's user-facing. - [ ] `schema_version` bumped iff Layer C (JSON doc) shape changed; use major/minor/patch per the policy above. - [ ] Regression guard added to `tests/test_formatters/test_report_structure.py` (presence + ordering, or `schema_version` assertion). - [ ] `has_profiling` gate respected for any new tier-0 sub-field. - [ ] `scripts/lint.sh --strict`, `scripts/link-checker.py --strict`, `scripts/test-samples.py --strict` all return rc=0. ## See also - [output_formats.md](output_formats.md) — the companion guide for format authors consuming the schema you just evolved. - `perfxpert/agents/schemas.py` — Layer A models. - `perfxpert/analysis/payload.py` — Layer B builder. - `perfxpert/formatters/json_fmt.py` — Layer C serialiser. - `tests/test_formatters/test_report_structure.py` — the regression guards for all three layers' final render. - `tests/test_agents/test_schema_field_caps.py` — the field-cap gate for Layer A.