Phase 1: Enhance plotting functionality in plot_phase1_figures.py #82

Merged

erikinkinen merged 1 commit from 1-detect-and-surface-overlapping-identical-strategy-curves-80 into main

2026-03-06 15:06:54 +01:00

erikinkinen commented

2026-03-06 15:01:15 +01:00

Owner

Summary

Implements Issue #80 by adding deterministic overlap analysis to the Phase 1 figure pipeline so overlapping strategy curves are reported explicitly, not inferred visually.

Closes #80.

What Changed

Extended plotting CLI in tools/figures/phase1/plot_phase1_figures.py:
- --tie-abs-tol (default 1e-9)
- --baseline-strategy (default direct)
- --log-y-cost
- --coincident-display-offset (display-only, default 0.0)
Added reusable tie/equivalence analysis for plotted outputs:
- exact equality groups
- tolerance-based groups
- deterministic pairwise stats (comparable, exact_equal, tolerance_equal, max_abs_diff)
Added sidecar outputs:
- <figure>.tie_groups.json for all multi-strategy line/scatter figures
- <figure>.pairwise_diff.csv for:
  - depth_sensitivity_total_cost
  - fanout_sensitivity_total_cost
  - post_revoke_hot_path_cost
  - residual_risk_vs_revoke_latency
Added delta companion figures (PNG + SVG):
- depth_sensitivity_total_cost.delta
- fanout_sensitivity_total_cost.delta
- post_revoke_hot_path_cost.delta
Added fail-fast delta validation:
- baseline strategy must exist
- x-vectors must match baseline exactly
Improved readability deterministically:
- stable marker map per strategy
- stable linestyle map per strategy
- markers at each x-value
- optional log-scale for cost plots
- optional display-only offset for exactly coincident series
Added per-figure tie summaries to console output during generation.
Updated smoke test expected artifact set.
Updated docs/phase1.md with new flags, overlap semantics, sidecars, delta behavior, and display-offset rules.

Determinism Notes

Tie detection/delta/pairwise use unmodified plotted values.
Display offsets are never used in analysis math.
Sidecar payloads and CSVs are emitted in stable order.
Figure outputs remain deterministic under identical inputs/flags.

Validation

Executed and passing:

ctest --test-dir _build --output-on-failure -R "aes_phase1_figures_tests|aes_phase1_figures_smoke_test"
ctest --test-dir _build --output-on-failure -R "aes_cli_sweep_tests|aes_cli_simulate_tests|aes_metrics_runner_tests|aes_revocation_outcome_metrics_tests"
ctest --test-dir _build --output-on-failure -R "aes_event_log_reader_tests|aes_event_log_replay_tests|aes_revocation_strategy_tests|aes_strategy_equivalence_tests|aes_invalid_event_determinism_tests"

Scope / Non-goals

No simulator semantics changes.
No workload-generation changes.
No new revocation metrics added in this PR.
This PR is analysis/plot interpretability only.

## Summary Implements Issue #80 by adding deterministic overlap analysis to the Phase 1 figure pipeline so overlapping strategy curves are reported explicitly, not inferred visually. Closes #80. ## What Changed - Extended plotting CLI in `tools/figures/phase1/plot_phase1_figures.py`: - `--tie-abs-tol` (default `1e-9`) - `--baseline-strategy` (default `direct`) - `--log-y-cost` - `--coincident-display-offset` (display-only, default `0.0`) - Added reusable tie/equivalence analysis for plotted outputs: - exact equality groups - tolerance-based groups - deterministic pairwise stats (`comparable`, `exact_equal`, `tolerance_equal`, `max_abs_diff`) - Added sidecar outputs: - `<figure>.tie_groups.json` for all multi-strategy line/scatter figures - `<figure>.pairwise_diff.csv` for: - `depth_sensitivity_total_cost` - `fanout_sensitivity_total_cost` - `post_revoke_hot_path_cost` - `residual_risk_vs_revoke_latency` - Added delta companion figures (PNG + SVG): - `depth_sensitivity_total_cost.delta` - `fanout_sensitivity_total_cost.delta` - `post_revoke_hot_path_cost.delta` - Added fail-fast delta validation: - baseline strategy must exist - x-vectors must match baseline exactly - Improved readability deterministically: - stable marker map per strategy - stable linestyle map per strategy - markers at each x-value - optional log-scale for cost plots - optional display-only offset for exactly coincident series - Added per-figure tie summaries to console output during generation. - Updated smoke test expected artifact set. - Updated `docs/phase1.md` with new flags, overlap semantics, sidecars, delta behavior, and display-offset rules. ## Determinism Notes - Tie detection/delta/pairwise use unmodified plotted values. - Display offsets are never used in analysis math. - Sidecar payloads and CSVs are emitted in stable order. - Figure outputs remain deterministic under identical inputs/flags. ## Validation Executed and passing: - `ctest --test-dir _build --output-on-failure -R "aes_phase1_figures_tests|aes_phase1_figures_smoke_test"` - `ctest --test-dir _build --output-on-failure -R "aes_cli_sweep_tests|aes_cli_simulate_tests|aes_metrics_runner_tests|aes_revocation_outcome_metrics_tests"` - `ctest --test-dir _build --output-on-failure -R "aes_event_log_reader_tests|aes_event_log_replay_tests|aes_revocation_strategy_tests|aes_strategy_equivalence_tests|aes_invalid_event_determinism_tests"` ## Scope / Non-goals - No simulator semantics changes. - No workload-generation changes. - No new revocation metrics added in this PR. - This PR is analysis/plot interpretability only.

erikinkinen added this to the Phase 1 milestone

2026-03-06 15:01:15 +01:00

erikinkinen added the

figure

phase-1

labels

2026-03-06 15:01:15 +01:00

erikinkinen self-assigned this

2026-03-06 15:01:15 +01:00

erikinkinen added 1 commit

2026-03-06 15:01:15 +01:00

Enhance plotting functionality in plot_phase1_figures.py (#80 )

ci / smoke (push) Successful in 1m21s

Details

clang-format / check-format (push) Successful in 10s

Details

markdownlint / markdown-lint (push) Successful in 9s

Details

ci / smoke (pull_request) Successful in 1m20s

Details

clang-format / check-format (pull_request) Successful in 10s

Details

markdownlint / markdown-lint (pull_request) Successful in 9s

Details

b97bf29b5e

- Introduced new PlotOptions dataclass to encapsulate plotting options.
- Added support for log scale on y-axis and baseline strategy selection.
- Implemented new marker and linestyle configurations for strategies.
- Enhanced tie reporting with JSON sidecar files for better data handling.
- Improved scatter and line plotting functions to accommodate new options.
- Added delta plotting functionality to visualize differences against a baseline strategy.
- Refactored existing plotting functions to utilize new options and improve readability.
- Updated argument parsing to include new plotting options.

erikinkinen added this to the AES — Active Workboard project

2026-03-06 15:01:15 +01:00

erikinkinen merged commit 7f5a06c94f into main

2026-03-06 15:06:54 +01:00

erikinkinen referenced this pull request from a commit

2026-03-06 15:06:56 +01:00

Phase 1: Enhance plotting functionality in plot_phase1_figures.py (#82)