[BUG] Native realm-admission diagnostic exits before its success marker #66

Closed
opened 2026-09-19 12:22:56 +02:00 by erikinkinen · 0 comments
Owner

Summary

Resolved: the shell's private script intake lacked realm-operation dispatch. Signed Launchd ca9e7f534e26023a6c011b5cb9a8e256fd56c2d6 adds the shared handler after the original active-envelope and native-owner checks, preserving held reply custody and delegating no sender. The matching local VM and both original corrected-source CI runs pass the admission scenario. The original Exsh 0xe5/Rootd 0xdc failures and investigation evidence remain below. The separate ext3 timeout and full realm-lifecycle gates remain open.

Reproduction

Build signed Exsh revision 183bcb9e9b4e37ecfab65bd0baa0cde53d02c728 with exsh-runtime,exsh-integration-realm-admission. Use the initial-shell start/exit disk-image fixture, runtime Rootd, tests/runtime-configs/runtime.dev.vm.toml, release profile and one CPU. Integration 76411ed6d96356b20ee0a4c21ad219dacc56340d publishes the explicit initial_exsh_realm_admission_probe = true selector and appliance-disk-image-realm-admission-positive scenario in PR 12. The original failed run used the identical candidate scenario/code and component graph before the roadmap status update.

One original run has been observed. It requires the unique ERIX_EXSH:REALM_ADMISSION:VERIFIED marker and ordinary successful shell exit, with the existing 120-second hard deadline and 45-second progress watchdog. No unchanged rerun has been used to claim success.

Expected and actual behavior

Expected: actual BEGIN/READ requests, full-width stale-generation rejection, missing-scope PREPARE refusal, acknowledged ABORT, record reuse and rejection of a stale ABORT, followed by the unique diagnostic marker and normal shell exit.

Actual: the image reaches Launchd readiness, final Rootd readiness and the version banner. Exsh exits 0xe5 without its diagnostic marker; Rootd exits 0xdc. The progress watchdog terminates QEMU after 45 seconds without serial progress. The scenario runner returns status 1. This is a runtime failure, not a successful negative test.

Environment and identity

  • Integration base: c479813423fb71c606fa30e5cf7d4dfb16f557d0, with the explicit diagnostic scenario under review.
  • Exsh: 183bcb9e9b4e37ecfab65bd0baa0cde53d02c728; Launchd: 5837873167c5e6ed99045af7052004affb5c28ff; Rootd: 3babf21e4531f563559e0eda05378a7e69f8c3c2.
  • x86-64 QEMU/OVMF, release disk image, Rust 1.97.1, one CPU and the existing explicit development VM policy.
  • Retained post-VM writable disk SHA256: d725f36d069a391f7e73d3c1b282215259a2763ce3bf93552ba58225fd28cd14. The earlier packaging checksum is retained separately.
  • The diagnostic uses Exsh's existing authenticated Launchd SEND|GRANT route and transfers no capabilities. It must not acquire another route or weaken caller/cleanup checks to pass.
  • No known passing native revision of this newly introduced diagnostic.

Evidence and prior checks

All 106 original appliance artifacts, complete serial/QEMU logs, actual process statuses and source selections are retained with hashes. Serial SHA256: 5bd1e76c8b086112fdb837ff48bc3925556cf4a4a7b11781b8731b0cb523ad59 (55,762 bytes). Image construction reports no build warnings.

Exsh's eight strict 966-test configurations and native builds pass. Integration's 169 helper commands, twelve route/scenario controls and four strict 320/321-test configurations pass, including formatting, strict host/native Clippy and private rustdoc. Those host results do not establish native execution. Exsh's complete frame proof remains an independent open gate.

Investigation is adding request/error telemetry through the existing stdout descriptor while preserving fail-stop behavior. A changed-input diagnostic run is required before attributing a cause. Preserve the failed run, all original timeouts, marker requirements and authority checks.

Tracking

Phase 6 master, Integration implementation, Exsh implementation, Exsh PR 3, Integration PR 12. Complete mediator preparation, configuration, sealing, client I/O and both builds inside EriX remain required.

Signed reproducer — 19 September 2026: Integration 76411ed6d96356b20ee0a4c21ad219dacc56340d publishes the explicit scenario and selector. With the documented image signing inputs, run bash scripts/run-scenario.sh tests/scenarios/appliance-disk-image-realm-admission-positive.toml components.toml. The original failed runtime uses the same scenario, implementation and component graph; only roadmap status changed after validation. Its VM result remains FAIL. All 169 helpers and four strict Rust configurations pass without warnings. Original CI 1673 and 1674 are queued. Issue 66 retains the failure. No acceptance credit or complete realm execution is claimed.

Changed diagnostic observation — 19 September 2026: signed Exsh e81f0cf0aed0b5a3360d2ee2dac6d3f656c90d3f passes all eight strict 967-test configurations, native builds, fmt, host/native Clippy and private rustdoc without warnings. Six native frame observations remain incomplete (97/63/100 runtime/all/diagnostic observations). Its bounded failure output uses existing stdout and identifies request 1, BEGIN_REALM, as InvalidReply. The actual telemetry-only VM retains original Launchd 5837873167c5e6ed99045af7052004affb5c28ff, the original authority policy, 120-second hard limit and 45-second progress watchdog. It still fails and all 106 artifacts are retained; serial SHA256 520999cc511e72ccb0bcb138c107f52ff629e2e3b21aa4d116ff4ad9aa402000 (55,858 bytes). No unchanged retry or successful runtime acceptance is claimed.

Source review shows that shells receive private script senders, while the initial realm dispatch recognizes these operations only on the public receiver. A Launchd change is under strict validation to enter the shared realm handler after the existing active-envelope and native-owner proof, preserving original held-reply custody and adding no sender. A matching fixed-server VM is required to establish the correction. Integration issue 66 retains both failures. Original Exsh CI 267 and 268 are under observation.

Signed private-route correction — 19 September 2026: Launchd ca9e7f534e26023a6c011b5cb9a8e256fd56c2d6 dispatches realm operations through the shell's existing private script receiver after active-envelope and original native-owner authentication. It repeats Running-job proof in the shared realm handler and preserves the original held reply across both ingress classes. No new sender is delegated. All six strict configurations pass 322 library and 55 runtime tests each, six native builds, fmt, strict host/native Clippy, private rustdoc and Markdown without warnings. Original CI 143 and 144 are under observation. A matching fixed-server VM is still required; Integration issue 66 remains open with both failed predecessor observations.

Corrected native caller VM — 19 September 2026: signed Integration 9139c6c5fa38c139e92520f6d410626b4cf1e4aa selects Launchd ca9e7f534e26023a6c011b5cb9a8e256fd56c2d6, Exsh 9c0f7ab851d527dd9dd10161d6a98e1fdc14598e and Docs ce8a1538ab2220f1b346e1a051265f58f83f47fc. The actual VM passes all eleven admission calls over the shell's existing private script route: reservation, reads, full-width stale-generation rejection, missing-scope refusal, acknowledged retirement, record reuse and stale-abort rejection. Exactly one ERIX_EXSH:REALM_ADMISSION:VERIFIED precedes ERIX_ROOTD:INITIAL_EXSH:EXITED_OK. The original 120-second hard limit and 45-second progress watchdog are unchanged; scenario status is 0 and no build warning is present.

All 106 actual appliance artifacts and complete logs are retained. The 55,738-byte serial log has SHA256 bbf41401e975cb0b39c6d62ca32f8e612f4f751a505a7dbb113fe1f05c6415f7; the post-VM writable disk has SHA256 8f7faf83e8923de675bf5d030458f7bf7e8065dc4fdb95ec98714dfebc5f7938. The earlier packaging checksum is retained separately. Independent review verifies all 73 original component revisions, the packaged diagnostic executable and the signed image's exact 72-byte LCH1 version-3 capacity record (four realm records; native arena 102,400 bytes). All 169 helper commands, twelve route/scenario controls and four strict 320/321-test Rust configurations pass, including native builds, fmt, strict host/native Clippy and private rustdoc; all 394 host streams are warning-free. Both failed predecessor images remain evidence in issue 66.

The first BEGIN failed because the shell's private script intake lacked realm dispatch. The correction retains active-envelope and original native-owner checks and original reply custody, without delegating a new sender. Exsh's uncertain-disposal path makes no stdout IPC before terminal failure. Original Integration CI 1675 and 1676 are queued. Successful preparation, mediator configuration/readiness/sealing, client byte I/O, complete source/effect/frame proof, native Rust/LLVM rebuilding and both full EriX builds remain required. This partial lifecycle acceptance adds no whole checklist item.

Original corrected-source CI checkpoint — 19 September 2026: at signed Integration 9139c6c5fa, original push CI 1675 passes all 488 VM scenarios. Original PR CI 1676 passes 487/488 and fails only subsystem-e2fs-fat-ext3-htree-positive at the unchanged 120-second QEMU limit. Both pass appliance-disk-image-realm-admission-positive; both Rust and Markdown jobs pass. Six complete hashed logs contain 27,276,890 bytes and no warnings. No rerun or cancellation supplies either result. The ext3 timeout remains tracked in issue 20; its cause is not inferred from the passing sibling run. The private script-route admission defect is corrected and natively demonstrated, while full realm lifecycle, complete frame proof and guest builds remain open.

Historical pre-correction CI reconciliation — 19 September 2026: original Integration run 1673 and 1674, at 76411ed6d96356b20ee0a4c21ad219dacc56340d, are terminal failures. Each passes 487/488 catalog scenarios and fails only appliance-disk-image-realm-admission-positive, reporting the unchanged 45-second serial-progress watchdog. Later native and interactive workflow gates are not reached. Both Rust and Markdown jobs pass. Six complete hashed logs total 27,087,569 bytes with zero warning candidates. Neither workflow was cancelled or rerun.

These runs precede the retained correction at 9139c6c5fa38c139e92520f6d410626b4cf1e4aa; both later original runs 1675/1676 passed that realm-admission scenario. Issue 66 retains the historical evidence and its existing closed state. This does not establish a new current-source regression, full lifecycle acceptance or full guest builds. Current Integration 1689/1690 remains queued at its second observation; acceptance stays 15/460, 3.48% weighted.

## Summary Resolved: the shell's private script intake lacked realm-operation dispatch. Signed Launchd ca9e7f534e26023a6c011b5cb9a8e256fd56c2d6 adds the shared handler after the original active-envelope and native-owner checks, preserving held reply custody and delegating no sender. The matching local VM and both original corrected-source CI runs pass the admission scenario. The original Exsh 0xe5/Rootd 0xdc failures and investigation evidence remain below. The separate ext3 timeout and full realm-lifecycle gates remain open. ## Reproduction Build signed Exsh revision `183bcb9e9b4e37ecfab65bd0baa0cde53d02c728` with `exsh-runtime,exsh-integration-realm-admission`. Use the initial-shell start/exit disk-image fixture, runtime Rootd, `tests/runtime-configs/runtime.dev.vm.toml`, release profile and one CPU. Integration `76411ed6d96356b20ee0a4c21ad219dacc56340d` publishes the explicit `initial_exsh_realm_admission_probe = true` selector and `appliance-disk-image-realm-admission-positive` scenario in [PR 12](https://git.erikinkinen.fi/erix/integration/pulls/12). The original failed run used the identical candidate scenario/code and component graph before the roadmap status update. One original run has been observed. It requires the unique `ERIX_EXSH:REALM_ADMISSION:VERIFIED` marker and ordinary successful shell exit, with the existing 120-second hard deadline and 45-second progress watchdog. No unchanged rerun has been used to claim success. ## Expected and actual behavior Expected: actual BEGIN/READ requests, full-width stale-generation rejection, missing-scope PREPARE refusal, acknowledged ABORT, record reuse and rejection of a stale ABORT, followed by the unique diagnostic marker and normal shell exit. Actual: the image reaches Launchd readiness, final Rootd readiness and the version banner. Exsh exits 0xe5 without its diagnostic marker; Rootd exits 0xdc. The progress watchdog terminates QEMU after 45 seconds without serial progress. The scenario runner returns status 1. This is a runtime failure, not a successful negative test. ## Environment and identity - Integration base: `c479813423fb71c606fa30e5cf7d4dfb16f557d0`, with the explicit diagnostic scenario under review. - Exsh: `183bcb9e9b4e37ecfab65bd0baa0cde53d02c728`; Launchd: `5837873167c5e6ed99045af7052004affb5c28ff`; Rootd: `3babf21e4531f563559e0eda05378a7e69f8c3c2`. - x86-64 QEMU/OVMF, release disk image, Rust 1.97.1, one CPU and the existing explicit development VM policy. - Retained post-VM writable disk SHA256: `d725f36d069a391f7e73d3c1b282215259a2763ce3bf93552ba58225fd28cd14`. The earlier packaging checksum is retained separately. - The diagnostic uses Exsh's existing authenticated Launchd SEND|GRANT route and transfers no capabilities. It must not acquire another route or weaken caller/cleanup checks to pass. - No known passing native revision of this newly introduced diagnostic. ## Evidence and prior checks All 106 original appliance artifacts, complete serial/QEMU logs, actual process statuses and source selections are retained with hashes. Serial SHA256: `5bd1e76c8b086112fdb837ff48bc3925556cf4a4a7b11781b8731b0cb523ad59` (55,762 bytes). Image construction reports no build warnings. Exsh's eight strict 966-test configurations and native builds pass. Integration's 169 helper commands, twelve route/scenario controls and four strict 320/321-test configurations pass, including formatting, strict host/native Clippy and private rustdoc. Those host results do not establish native execution. Exsh's complete frame proof remains an independent open gate. Investigation is adding request/error telemetry through the existing stdout descriptor while preserving fail-stop behavior. A changed-input diagnostic run is required before attributing a cause. Preserve the failed run, all original timeouts, marker requirements and authority checks. ## Tracking [Phase 6 master](https://git.erikinkinen.fi/erix/integration/issues/65), [Integration implementation](https://git.erikinkinen.fi/erix/integration/issues/1), [Exsh implementation](https://git.erikinkinen.fi/erix/exsh/issues/1), [Exsh PR 3](https://git.erikinkinen.fi/erix/exsh/pulls/3), [Integration PR 12](https://git.erikinkinen.fi/erix/integration/pulls/12). Complete mediator preparation, configuration, sealing, client I/O and both builds inside EriX remain required. Signed reproducer — 19 September 2026: Integration `76411ed6d96356b20ee0a4c21ad219dacc56340d` publishes the explicit scenario and selector. With the documented image signing inputs, run `bash scripts/run-scenario.sh tests/scenarios/appliance-disk-image-realm-admission-positive.toml components.toml`. The original failed runtime uses the same scenario, implementation and component graph; only roadmap status changed after validation. Its VM result remains FAIL. All 169 helpers and four strict Rust configurations pass without warnings. Original [CI 1673](https://git.erikinkinen.fi/erix/integration/actions/runs/1673) and [1674](https://git.erikinkinen.fi/erix/integration/actions/runs/1674) are queued. [Issue 66](https://git.erikinkinen.fi/erix/integration/issues/66) retains the failure. No acceptance credit or complete realm execution is claimed. Changed diagnostic observation — 19 September 2026: signed Exsh `e81f0cf0aed0b5a3360d2ee2dac6d3f656c90d3f` passes all eight strict 967-test configurations, native builds, fmt, host/native Clippy and private rustdoc without warnings. Six native frame observations remain incomplete (97/63/100 runtime/all/diagnostic observations). Its bounded failure output uses existing stdout and identifies request 1, BEGIN_REALM, as InvalidReply. The actual telemetry-only VM retains original Launchd `5837873167c5e6ed99045af7052004affb5c28ff`, the original authority policy, 120-second hard limit and 45-second progress watchdog. It still fails and all 106 artifacts are retained; serial SHA256 `520999cc511e72ccb0bcb138c107f52ff629e2e3b21aa4d116ff4ad9aa402000` (55,858 bytes). No unchanged retry or successful runtime acceptance is claimed. Source review shows that shells receive private script senders, while the initial realm dispatch recognizes these operations only on the public receiver. A Launchd change is under strict validation to enter the shared realm handler after the existing active-envelope and native-owner proof, preserving original held-reply custody and adding no sender. A matching fixed-server VM is required to establish the correction. [Integration issue 66](https://git.erikinkinen.fi/erix/integration/issues/66) retains both failures. Original [Exsh CI 267](https://git.erikinkinen.fi/erix/exsh/actions/runs/267) and [268](https://git.erikinkinen.fi/erix/exsh/actions/runs/268) are under observation. Signed private-route correction — 19 September 2026: Launchd `ca9e7f534e26023a6c011b5cb9a8e256fd56c2d6` dispatches realm operations through the shell's existing private script receiver after active-envelope and original native-owner authentication. It repeats Running-job proof in the shared realm handler and preserves the original held reply across both ingress classes. No new sender is delegated. All six strict configurations pass 322 library and 55 runtime tests each, six native builds, fmt, strict host/native Clippy, private rustdoc and Markdown without warnings. Original [CI 143](https://git.erikinkinen.fi/erix/launchd/actions/runs/143) and [144](https://git.erikinkinen.fi/erix/launchd/actions/runs/144) are under observation. A matching fixed-server VM is still required; [Integration issue 66](https://git.erikinkinen.fi/erix/integration/issues/66) remains open with both failed predecessor observations. Corrected native caller VM — 19 September 2026: signed Integration `9139c6c5fa38c139e92520f6d410626b4cf1e4aa` selects Launchd `ca9e7f534e26023a6c011b5cb9a8e256fd56c2d6`, Exsh `9c0f7ab851d527dd9dd10161d6a98e1fdc14598e` and Docs `ce8a1538ab2220f1b346e1a051265f58f83f47fc`. The actual VM passes all eleven admission calls over the shell's existing private script route: reservation, reads, full-width stale-generation rejection, missing-scope refusal, acknowledged retirement, record reuse and stale-abort rejection. Exactly one `ERIX_EXSH:REALM_ADMISSION:VERIFIED` precedes `ERIX_ROOTD:INITIAL_EXSH:EXITED_OK`. The original 120-second hard limit and 45-second progress watchdog are unchanged; scenario status is 0 and no build warning is present. All 106 actual appliance artifacts and complete logs are retained. The 55,738-byte serial log has SHA256 `bbf41401e975cb0b39c6d62ca32f8e612f4f751a505a7dbb113fe1f05c6415f7`; the post-VM writable disk has SHA256 `8f7faf83e8923de675bf5d030458f7bf7e8065dc4fdb95ec98714dfebc5f7938`. The earlier packaging checksum is retained separately. Independent review verifies all 73 original component revisions, the packaged diagnostic executable and the signed image's exact 72-byte LCH1 version-3 capacity record (four realm records; native arena 102,400 bytes). All 169 helper commands, twelve route/scenario controls and four strict 320/321-test Rust configurations pass, including native builds, fmt, strict host/native Clippy and private rustdoc; all 394 host streams are warning-free. Both failed predecessor images remain evidence in [issue 66](https://git.erikinkinen.fi/erix/integration/issues/66). The first BEGIN failed because the shell's private script intake lacked realm dispatch. The correction retains active-envelope and original native-owner checks and original reply custody, without delegating a new sender. Exsh's uncertain-disposal path makes no stdout IPC before terminal failure. Original [Integration CI 1675](https://git.erikinkinen.fi/erix/integration/actions/runs/1675) and [1676](https://git.erikinkinen.fi/erix/integration/actions/runs/1676) are queued. Successful preparation, mediator configuration/readiness/sealing, client byte I/O, complete source/effect/frame proof, native Rust/LLVM rebuilding and both full EriX builds remain required. This partial lifecycle acceptance adds no whole checklist item. Original corrected-source CI checkpoint — 19 September 2026: at signed Integration 9139c6c5fa38c139e92520f6d410626b4cf1e4aa, original [push CI 1675](https://git.erikinkinen.fi/erix/integration/actions/runs/1675) passes all 488 VM scenarios. Original [PR CI 1676](https://git.erikinkinen.fi/erix/integration/actions/runs/1676) passes 487/488 and fails only `subsystem-e2fs-fat-ext3-htree-positive` at the unchanged 120-second QEMU limit. Both pass `appliance-disk-image-realm-admission-positive`; both Rust and Markdown jobs pass. Six complete hashed logs contain 27,276,890 bytes and no warnings. No rerun or cancellation supplies either result. The ext3 timeout remains tracked in [issue 20](https://git.erikinkinen.fi/erix/integration/issues/20); its cause is not inferred from the passing sibling run. The private script-route admission defect is corrected and natively demonstrated, while full realm lifecycle, complete frame proof and guest builds remain open. Historical pre-correction CI reconciliation — 19 September 2026: original [Integration run 1673](https://git.erikinkinen.fi/erix/integration/actions/runs/1673) and [1674](https://git.erikinkinen.fi/erix/integration/actions/runs/1674), at `76411ed6d96356b20ee0a4c21ad219dacc56340d`, are terminal failures. Each passes 487/488 catalog scenarios and fails only `appliance-disk-image-realm-admission-positive`, reporting the unchanged 45-second serial-progress watchdog. Later native and interactive workflow gates are not reached. Both Rust and Markdown jobs pass. Six complete hashed logs total 27,087,569 bytes with zero warning candidates. Neither workflow was cancelled or rerun. These runs precede the retained correction at `9139c6c5fa38c139e92520f6d410626b4cf1e4aa`; both later original runs 1675/1676 passed that realm-admission scenario. [Issue 66](https://git.erikinkinen.fi/erix/integration/issues/66) retains the historical evidence and its existing closed state. This does not establish a new current-source regression, full lifecycle acceptance or full guest builds. Current Integration 1689/1690 remains queued at its second observation; acceptance stays 15/460, 3.48% weighted.
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
erix/integration#66
No description provided.