erikinkinen/AES
1
0
Fork 0
Code Issues 1 Pull requests Projects 1 Activity Actions

Experiment hygiene and threats-to-validity checks #39

New issue
Closed
opened 2026-01-31 17:18:25 +01:00 by erikinkinen · 0 comments
erikinkinen commented 2026-01-31 17:18:25 +01:00
Owner
Copy link

Goal
Prevent trivial or misleading conclusions.

Steps

  • Enforce comparable workloads across strategies
  • Add multi-seed confidence intervals
  • Add sanity checks before long runs
**Goal** Prevent trivial or misleading conclusions. **Steps** * [ ] Enforce comparable workloads across strategies * [ ] Add multi-seed confidence intervals * [ ] Add sanity checks before long runs
erikinkinen added this to the Phase 1 milestone 2026-01-31 17:18:25 +01:00
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
1-experiment-hygiene-and-threats-to-validity-checks-39
1-detect-and-surface-overlapping-identical-strategy-curves-80
1-strategy-differentiating-revocation-artifacts-and-figure-pipeline
1-plot-and-figure-generation-pipeline
1-batch-runner-for-phase-1-sweeps
1-revocation-outcome-metrics
1-revocation-focused-workloads
1-strategy-equivalence-and-sanity-checks
1-strategy-d-epoch-indirection-revocation
1-strategy-c-lineage-based-revocation-basic
1-strategy-b-lazy-invalidate-on-use-revocation
1-strategy-a-eager-traversal-revocation
1-revocation-strategy-interface
1-revocation-cost-accounting-framework
1-ground-truth-revocation-oracle
1-revocation-event-types-and-api
1-capability-use-semantics
1-phase1-specification-and-invariants
0-phase0-documentation
0-determinism-and-invariant-test-suite
0-cli-simulate-and-export
0-metrics-framework
0-residual-authority-detection-baseline
0-ground-truth-reachability-analysis
0-snapshot-hashing-and-equality
0-replay-and-snapshot-reconstruction
0-event-log-format-v0
0-workloads
0-workload-generator-interface
0-deterministic-simulation-engine
0-graph-delta-format
0-event-application-api
0-event-taxonomy-v0
0-graph-indexing-and-integrity
0-capability-edge-representation
0-subject-and-object-node-storage
0-authority-model-specification
0-fix-formatting
0-stable-id-types-and-utilities
0-repository-scaffold-and-conventions
No results found.
Labels
Clear labels   
bug

   
cli

   
core

   
docs

   
event

   
experiment

   
figure

   
invariant

   
metrics

   
oracle

   
phase-0

   
phase-1

   
phase-2

   
phase-3

   
phase-4

   
phase-5

   
phase-6

   
provenance

   
revocation

   
tests

   
workload
No labels
bug
cli
core
docs
event
experiment
figure
invariant
metrics
oracle
phase-0
phase-1
phase-2
phase-3
phase-4
phase-5
phase-6
provenance
revocation
tests
workload
Milestone
Clear milestone
No items
No milestone
Phase 1
Projects
Clear projects
No items
No project
AES — Active Workboard
Assignees
Clear assignees
No assignees
erikinkinen
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
erikinkinen/AES#39
No description provided.