Research schemas accept trailing line terminators in identities and SHA-256 strings #646

Closed
opened 2026-10-20 23:30:01 +02:00 by erikinkinen · 0 comments
Owner

Discovered during P2-09 (#100) independent tape-schema validation against accepted main e36da8b557.

The shared identity pattern and SHA-256 patterns end with $. JSON Schema regex matching permits that anchor immediately before a final line terminator. With jsonschema 4.23.0 / Draft 2020-12, a valid event changed only to event_id = "1\n" or state_sha256 = <64 lowercase hex characters> + "\n" still passes the checked-in research-event-v4 schema. The v4 decoder correctly rejects both. The checkpoint schema uses the same identity definition and permissive hash patterns.

A baseline valid event passes both schema and decoder. The two one-field mutations pass schema but return error from aes_research_event_codec_tests --classify-lines. Local preserved reproduction: /tmp/aes-phase2-execution/schema-terminator-probe/ (accepted schema copies, three JSONL fixtures and probe-results.json).

Impact: schema-only validation can misclassify malformed provenance identifiers/digests as valid. The decoder remains strict; this is a schema/codec contract mismatch, not evidence that malformed events can execute. The new intent tape schema inherits the identity definition and must receive the correction before P2-09 acceptance.

  • Correct shared identity and digest schemas to require the entire string, without altering valid canonical encodings.
  • Preserve regression probes for trailing LF, CR, CRLF and Unicode line separators; require schema/decoder agreement and retain valid numeric/hash boundaries.
  • Validate the exact source in Debug, Release and Clang ASan/UBSan, record evidence, and merge the dedicated signed PR only after all required CI checks pass.

Milestone: Phase 2. Labels: bug, event, invariant, tests, phase-2. This issue blocks P2-09 acceptance; it does not change the frozen scientific protocol.

Discovered during P2-09 (#100) independent tape-schema validation against accepted main e36da8b5577510d101eff4755278cc075d3dc5c4. The shared identity pattern and SHA-256 patterns end with `$`. JSON Schema regex matching permits that anchor immediately before a final line terminator. With jsonschema 4.23.0 / Draft 2020-12, a valid event changed only to `event_id = "1\n"` or `state_sha256 = <64 lowercase hex characters> + "\n"` still passes the checked-in research-event-v4 schema. The v4 decoder correctly rejects both. The checkpoint schema uses the same identity definition and permissive hash patterns. A baseline valid event passes both schema and decoder. The two one-field mutations pass schema but return `error` from `aes_research_event_codec_tests --classify-lines`. Local preserved reproduction: `/tmp/aes-phase2-execution/schema-terminator-probe/` (accepted schema copies, three JSONL fixtures and probe-results.json). Impact: schema-only validation can misclassify malformed provenance identifiers/digests as valid. The decoder remains strict; this is a schema/codec contract mismatch, not evidence that malformed events can execute. The new intent tape schema inherits the identity definition and must receive the correction before P2-09 acceptance. - [x] Correct shared identity and digest schemas to require the entire string, without altering valid canonical encodings. - [x] Preserve regression probes for trailing LF, CR, CRLF and Unicode line separators; require schema/decoder agreement and retain valid numeric/hash boundaries. - [x] Validate the exact source in Debug, Release and Clang ASan/UBSan, record evidence, and merge the dedicated signed PR only after all required CI checks pass. Milestone: Phase 2. Labels: bug, event, invariant, tests, phase-2. This issue blocks P2-09 acceptance; it does not change the frozen scientific protocol.
erikinkinen added this to the Phase 2 milestone 2026-10-20 23:30:01 +02:00
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Reference
erikinkinen/AES#646
No description provided.