Strategy-Differentiating Revocation Artifacts and Figure Pipeline

erikinkinen commented

2026-03-06 13:42:19 +01:00

Owner

Current Phase 1 analysis still over-emphasizes workload size and under-exposes strategy trade-offs.
This issue adds end-to-end artifact and plotting support so we can compare strategies on eventual cost, residual risk, and bookkeeping overhead.

Scope

Wire new revocation workloads into aes simulate and aes sweep:
- deep_tree_mid_subtree_revoke
- root_revoke_with_large_unrelated_background
- shared_descendant_revoke
Extend strict sweep config parsing/validation for the new workload schemas and minima.
Emit per-run metrics_v2.json in sweep outputs and include metrics_v2_path in runs.csv.
Update figure generation to consume sweep outputs plus per-run artifacts (metrics_v2.json, event_log.jsonl) and generate deterministic trade-off plots.
Update canonical cost/completeness sweep configs to include new strategy-separating workloads.
Update docs and tests for CLI, sweep, and figure pipeline behavior.

Expected Figure Outputs (PNG + SVG)

immediate_vs_total_cost_by_strategy
completeness_vs_total_cost_by_workload
residual_risk_vs_revoke_latency
metadata_overhead_vs_completeness
post_revoke_hot_path_cost
depth_sensitivity_completeness
depth_sensitivity_total_cost
fanout_sensitivity_completeness
fanout_sensitivity_total_cost

Acceptance Criteria

New workloads are accepted by CLI with strict flag validation and deterministic behavior.
Sweep runs persist event_log.jsonl, metrics_v1.json, and metrics_v2.json per run.
runs.csv includes metrics_v2_path and remains deterministic.
tools/regen-figures.sh produces the full figure set reproducibly.
Existing revocation/sweep/metrics regression tests remain green.

Current Phase 1 analysis still over-emphasizes workload size and under-exposes strategy trade-offs. This issue adds end-to-end artifact and plotting support so we can compare strategies on eventual cost, residual risk, and bookkeeping overhead. **Scope** 1. Wire new revocation workloads into `aes simulate` and `aes sweep`: - `deep_tree_mid_subtree_revoke` - `root_revoke_with_large_unrelated_background` - `shared_descendant_revoke` 2. Extend strict sweep config parsing/validation for the new workload schemas and minima. 3. Emit per-run `metrics_v2.json` in sweep outputs and include `metrics_v2_path` in `runs.csv`. 4. Update figure generation to consume sweep outputs plus per-run artifacts (`metrics_v2.json`, `event_log.jsonl`) and generate deterministic trade-off plots. 5. Update canonical cost/completeness sweep configs to include new strategy-separating workloads. 6. Update docs and tests for CLI, sweep, and figure pipeline behavior. **Expected Figure Outputs (PNG + SVG)** 1. `immediate_vs_total_cost_by_strategy` 2. `completeness_vs_total_cost_by_workload` 3. `residual_risk_vs_revoke_latency` 4. `metadata_overhead_vs_completeness` 5. `post_revoke_hot_path_cost` 6. `depth_sensitivity_completeness` 7. `depth_sensitivity_total_cost` 8. `fanout_sensitivity_completeness` 9. `fanout_sensitivity_total_cost` **Acceptance Criteria** 1. New workloads are accepted by CLI with strict flag validation and deterministic behavior. 2. Sweep runs persist `event_log.jsonl`, `metrics_v1.json`, and `metrics_v2.json` per run. 3. `runs.csv` includes `metrics_v2_path` and remains deterministic. 4. `tools/regen-figures.sh` produces the full figure set reproducibly. 5. Existing revocation/sweep/metrics regression tests remain green.