Phase 6: Fit and freeze all predictive models and thresholds #518

Open
opened 2026-10-20 15:05:15 +02:00 by erikinkinen · 0 comments
Owner

Plan ID: P6-28-E
Record type: Gate leaf
Scientific protocol: 2.2 · Backlog revision: 2.2.1
Specification: sections 10, 11.1, 17.1, 17.2; parent requirements and common section-11/17.4 gates apply.
Source: AES_PHASES_2_6_PLAN.md, issue heading at line 9056.
Plan SHA-256: 8acdaeb52d6a94fcad7de64d64521cd465f608344d63560b45f9202f5af71212
Protocol snapshot: Download the complete governing plan.
Tracking parent: P6-28 (#513).

Goal

Complete production analysis before expanding any draft into a full article.

Applicable parent requirements

  • P6-28.R1: Execute all pilot/precision/qualification stages; generate 192*(1600+M) internal traces, freeze fits, then execute 64M external AES, 16M prototype and 24*M application traces.

Prerequisites

Steps

  • Fit the declared models using the 1200 training seeds. Perform the specified hyperparameter and control-parameter selection through five grouped training-only folds, fitting preprocessing inside each fold and refitting on all training data after selection.
  • Use the 400 validation seeds to select alert thresholds and the C6b operational comparator under the fixed false-alarm budget and tie rules. Do not use this partition to tune model hyperparameters or fit preprocessing.
  • Sign fit-freeze with the training-fold selection records, validation decisions and final model, preprocessing, feature, threshold and baseline hashes.

Definition of Done

Selection records demonstrate the training/validation roles specified in sections 10.4, 10.5 and 10.8; all hashes are fixed before any test evaluation or external production tape, and test labels/outcome summaries remain sealed throughout fitting and selection.

Execution and acceptance contract

Apply the governing semantic/data/statistical contracts in sections 3–5 and the relevant phase section, plus the section-11 checks applicable to this task. Section 17.4 defines the required acceptance evidence. Store the outcome and immutable prerequisite/input/output hashes at

artifacts/backlog/2.2.1/P6-28-E/acceptance.json.

Only explicit dependency and parent-closure edges govern readiness; numbering adds no predecessor dependency. Checkboxes are work/acceptance items, not commit quotas. Keep implementation changes independently reviewable using the existing PR template. Execution, review and gate tasks close with their evidence/decision records and a PR only when tracked artifacts change.

An unsupported scientific effect is a valid result. Missing correctness validation, invalid inference or absent required review is not completion. Do not change endpoints, matrices, margins, inference rules or sample sizes outside the protocol change-control process.

<!-- aes-plan-id: P6-28-E; backlog-revision: 2.2.1 --> **Plan ID:** `P6-28-E` **Record type:** Gate leaf **Scientific protocol:** 2.2 · **Backlog revision:** 2.2.1 **Specification:** sections 10, 11.1, 17.1, 17.2; parent requirements and common section-11/17.4 gates apply. **Source:** `AES_PHASES_2_6_PLAN.md`, issue heading at line 9056. **Plan SHA-256:** `8acdaeb52d6a94fcad7de64d64521cd465f608344d63560b45f9202f5af71212` **Protocol snapshot:** [Download the complete governing plan](https://git.erikinkinen.fi/attachments/31cb768b-4788-4e38-aba9-9d7f0d1ee16d). **Tracking parent:** [P6-28 (#513)](https://git.erikinkinen.fi/erikinkinen/AES/issues/513). ## Goal Complete production analysis before expanding any draft into a full article. ## Applicable parent requirements - **P6-28.R1:** Execute all pilot/precision/qualification stages; generate 192*(1600+M) internal traces, freeze fits, then execute 64*M external AES, 16*M prototype and 24*M application traces. ## Prerequisites - [P5-32 (#427)](https://git.erikinkinen.fi/erikinkinen/AES/issues/427) - [P6-28-D (#517)](https://git.erikinkinen.fi/erikinkinen/AES/issues/517) ## Steps - [ ] Fit the declared models using the 1200 training seeds. Perform the specified hyperparameter and control-parameter selection through five grouped training-only folds, fitting preprocessing inside each fold and refitting on all training data after selection. - [ ] Use the 400 validation seeds to select alert thresholds and the C6b operational comparator under the fixed false-alarm budget and tie rules. Do not use this partition to tune model hyperparameters or fit preprocessing. - [ ] Sign fit-freeze with the training-fold selection records, validation decisions and final model, preprocessing, feature, threshold and baseline hashes. ## Definition of Done Selection records demonstrate the training/validation roles specified in sections 10.4, 10.5 and 10.8; all hashes are fixed before any test evaluation or external production tape, and test labels/outcome summaries remain sealed throughout fitting and selection. ## Execution and acceptance contract Apply the governing semantic/data/statistical contracts in sections 3–5 and the relevant phase section, plus the section-11 checks applicable to this task. Section 17.4 defines the required acceptance evidence. Store the outcome and immutable prerequisite/input/output hashes at `artifacts/backlog/2.2.1/P6-28-E/acceptance.json`. Only explicit dependency and parent-closure edges govern readiness; numbering adds no predecessor dependency. Checkboxes are work/acceptance items, not commit quotas. Keep implementation changes independently reviewable using the existing PR template. Execution, review and gate tasks close with their evidence/decision records and a PR only when tracked artifacts change. An unsupported scientific effect is a valid result. Missing correctness validation, invalid inference or absent required review is not completion. Do not change endpoints, matrices, margins, inference rules or sample sizes outside the protocol change-control process.
erikinkinen added this to the Phase 6 milestone 2026-10-20 15:05:15 +02:00
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Reference
erikinkinen/AES#518
No description provided.