Reject leading-zero numbers and non-JSON whitespace in legacy logs #633

Merged
erikinkinen merged 1 commit from phase2/bug-632-jsonl-numeric-grammar into main 2026-10-20 19:51:31 +02:00
Owner

Task

Closes #632.

Summary

The legacy decoder accepted "v":03 and treated vertical tab/form feed as JSON whitespace. Restrict numeric digits and whitespace to JSON's ASCII grammar and reject a leading zero before another digit. Valid legacy records retain their canonical encoding.

Scope

Change only the legacy parser's lexical boundary and add a regression test covering versions 0–3. The test checks nested numeric fields, unsigned overflow, all four valid whitespace characters, invalid whitespace positions, zero and the maximum u64 value. Version-4 codec implementation remains P2-07-B.

Design intent

Return ParseError for malformed input rather than accepting a partial or silently normalized record. Preserve existing successful event semantics and archived legacy fixture bytes.

Phase discipline

This independently reproduced parser bug was found during preparation for the Phase 2 strict research codec. It has its own issue, branch and PR. P2-07-A is already merged; no scientific protocol or data contract changes are included.

Verification

Before the fix the new regression CTest ran and failed with 61 lexical acceptance failures. After the fix it passes 44 valid canonical/whitespace round trips and rejects all 117 malformed cases. The standalone probe changes from leading_zero_accepted=1 to leading_zero_accepted=0 while the canonical record still succeeds.

All 59 CTests pass at signed source a0130fac20efeb36721d3d351e86e9ff7d39c767 in Debug, Release and Clang ASan/UBSan, including the archived legacy fixture suite. All repository hooks pass. Before/after logs, probe source, JUnit and toolchain evidence.

All eight final-head Forgejo checks must pass before the three issue checklist entries are checked once and the PR is merged.

## Task Closes #632. ## Summary The legacy decoder accepted `"v":03` and treated vertical tab/form feed as JSON whitespace. Restrict numeric digits and whitespace to JSON's ASCII grammar and reject a leading zero before another digit. Valid legacy records retain their canonical encoding. ## Scope Change only the legacy parser's lexical boundary and add a regression test covering versions 0–3. The test checks nested numeric fields, unsigned overflow, all four valid whitespace characters, invalid whitespace positions, zero and the maximum u64 value. Version-4 codec implementation remains P2-07-B. ## Design intent Return ParseError for malformed input rather than accepting a partial or silently normalized record. Preserve existing successful event semantics and archived legacy fixture bytes. ## Phase discipline This independently reproduced parser bug was found during preparation for the Phase 2 strict research codec. It has its own issue, branch and PR. P2-07-A is already merged; no scientific protocol or data contract changes are included. ## Verification Before the fix the new regression CTest ran and failed with 61 lexical acceptance failures. After the fix it passes 44 valid canonical/whitespace round trips and rejects all 117 malformed cases. The standalone probe changes from `leading_zero_accepted=1` to `leading_zero_accepted=0` while the canonical record still succeeds. All 59 CTests pass at signed source `a0130fac20efeb36721d3d351e86e9ff7d39c767` in Debug, Release and Clang ASan/UBSan, including the archived legacy fixture suite. All repository hooks pass. [Before/after logs, probe source, JUnit and toolchain evidence](https://git.erikinkinen.fi/attachments/83d7b827-a16e-4778-bf79-93c26c208745). All eight final-head Forgejo checks must pass before the three issue checklist entries are checked once and the PR is merged.
erikinkinen added this to the Phase 2 milestone 2026-10-20 19:46:28 +02:00
Reject non-JSON number and whitespace syntax (fixes #632)
All checks were successful
markdownlint / markdown-lint (pull_request) Successful in 19s
clang-format / check-format (pull_request) Successful in 24s
ci / sanitizers (pull_request) Successful in 4m54s
ci / smoke (pull_request) Successful in 4m58s
markdownlint / markdown-lint (push) Successful in 18s
clang-format / check-format (push) Successful in 18s
ci / smoke (push) Successful in 2m47s
ci / sanitizers (push) Successful in 2m52s
a0130fac20
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
erikinkinen/AES!633
No description provided.