[BUG] Process-event decoder rejects valid realm mediator terminal replies #3

Open
opened 2026-09-24 19:26:15 +02:00 by erikinkinen · 0 comments
Owner

Summary

The authenticated process-event decoder admits only ordinary LaunchChild terminal replies even though the runtime dispatcher also owns RealmMediator children. A valid realm-mediator terminal reply is rejected before it reaches that dispatcher.

Correction to the initial report: the observed VM exit 0xE5 does not identify Launchd or establish that this decoder defect caused the startup failure. Exsh uses the same exit status. The recorded exit endpoint is 0x520, whereas Launchd is bound to 0x280; these are different processes. The actual startup failure remains unresolved and no Posixd startup or readiness is accepted.

Reproduction

The source-bound decoder trigger is a successful, correctly correlated WAIT_PROCESS_EVENT reply with a nonzero process identity and generation, a valid terminal state/event pair, zero flags/receipts, and the RealmMediator role. At the selected Launchd revision, the explicit role check rejects it. This is an independently identified decoder defect; a regression test and correction are in progress.

The associated native VM observation used Integration ffedb3630895b2bc547ab58e4900f750d84cd64b and its selected component graph, including Launchd 0bb56cc229fe4563da6c77ebe6ce9c178b903f26 and Procd d04311abff67e7355bd92bb2e48fb6161e0ec674, then run the maintained scenario:

bash scripts/run-scenario.sh tests/scenarios/appliance-disk-image-realm-startup-positive.toml components.toml

This is one original local native VM observation, not a successful CI receipt or an unchanged-input retry. The original 120-second scenario deadline and 45-second progress watchdog were retained. Failure frequency and a last known working revision are unestablished.

Expected and actual behavior

Launchd should process authenticated terminal events for its original ordinary-child and realm-mediator owners, then remain available for the actual private startup sequence. The scenario requires two actual Posixd mediators, authenticated readiness, independent retirement, stale-handle rejection and successful shell exit.

The serial receipt instead records, in order:

ERIX_EXSH:REALM_ADMISSION:VERIFIED
ERIX_EXSH:REALM_PREPARATION:VERIFIED
ERIX_KERNEL:PROCESS_EXIT_TCB=0000000000000022
ERIX_KERNEL:PROCESS_EXIT_EP=0000000000000520
ERIX_KERNEL:PROCESS_EXIT_CODE=00000000000000E5
ERIX_EXSH:REALM_ADMISSION:FAILED request=00000012 kind=0000000b first=000006d1 second=00000003
ERIX_KERNEL:PROCESS_EXIT_TCB=0000000000000001
ERIX_KERNEL:PROCESS_EXIT_CODE=00000000000000DC

The recorded TCB 0x22 exits through endpoint 0x520, which differs from Launchd's bound endpoint 0x280. Both Launchd and Exsh define an 0xE5 failure status, so that value alone does not establish which process exited or identify a Launchd process-event failure. Exsh's diagnostic kind=0x0B means a valid typed rejected reply, not transport loss: its first PREPARE for the actual Posixd sequence, request 0x12 / operation 0x6D1, returns NOT_FOUND (3). Rootd then exits 0xDC. The 45-second progress watchdog terminates QEMU after progress stops; the scenario command returns 1. The required REALM_STARTUP:VERIFIED marker is absent. That watchdog failure is preserved as a failed run, not a warning-free success.

Environment and identity

  • Release native x86-64 QEMU/OVMF image, development runtime configuration, one virtual CPU, Rust 1.97.1; actual packaged service binaries and shared objects.
  • Integration driver: ffedb3630895b2bc547ab58e4900f750d84cd64b; Launchd: 0bb56cc229fe4563da6c77ebe6ce9c178b903f26; Procd: d04311abff67e7355bd92bb2e48fb6161e0ec674.
  • Kernel: 5e285aa91a1423523b2a094d4ce0449ada39b16c; Rootd: 2f1c7256eb77b6b29f1dad7d9b5028fdf8a7e527; Exsh: 2239d5faabc0a52091af921484bee6bb2d558c0b; Posixd: 809647d60544c4c6121868c99133f30d054965c9.
  • The pinned shared orchestration component uses Integration a787946077c918e9426a8eac741afe2d4e51372a. It is distinct from the selected scenario driver above.
  • Packaged disk SHA-256 receipt: e9a66bf5887f1d0fdb746080e0b6b10e5bd7923b02c8e2306e0d0586acf937b0. The writable post-VM disk is retained separately; no byte-identity inference substitutes for runtime acceptance.
  • Authority remains the selected cwd scope, launch route and private realm capabilities. No global route, additional root, weakened identity check or timeout change was used to bypass the failure.

Evidence and prior checks

The retained evidence inventory covers 107 files. The complete serial log is 56,013 bytes, SHA-256 a1132d376e101c370f06f7522f45139d34703e8d385847e848f6d4f84c43ae72; its hash and length verify. Image construction has no recorded warnings. QEMU stderr retains the original progress-watchdog termination. Existing admission/preparation markers only prove those preceding checks; they do not establish actual Posixd startup, readiness or retirement.

Source inspection identifies a concrete contract gap: validate_procd_wait_process_event_reply admits only the LaunchChild role. The runtime dispatcher already checks realm ownership before ordinary jobs, but a valid RealmMediator event cannot reach it through that decoder. This source-level defect is independent of the VM exit attribution. The serial log does not expose a rejected WAIT_PROCESS_EVENT reply and does not prove that the decoder branch caused the PREPARE rejection. The selected Exsh runtime also defines 0xE5; its realm diagnostic encodes typed rejection as kind 11. Source-bound decoder regression tests and separate diagnosis of the native PREPARE failure remain required.

Tracking

The same checkpoint fixes the independently reproduced WAIT decoder defect by accepting exactly ordinary-child and realm-mediator terminal roles before matching the retained process generation. Invalid identity, framing, state and authority remain rejected. All six default/all-feature/production development/release configurations pass 329 library and 66 runtime tests each, strict host/native Clippy, native builds, private rustdoc, formatting and documentation checks, with zero warnings. Authored Rust files remain below 1,000 lines. Final documentation corrections preserve the tested Rust behavior.

The original failed VM remains retained. A newly selected image must pass the unchanged actual startup/readiness/retirement scenario before native acceptance; configuration/seal, four clients, byte I/O and guest toolchain/EriX builds remain open. Private preparation issue 4 and independent event issue 3 remain distinct. WIP PR 2 and Phase 6 completion track the remaining acceptance.

Track the correction in Launchd PR 2, Launchd authority and lifecycle audit, Integration validation and Phase 6 completion.

  • Cover valid realm-mediator terminal events and malformed, mismatched or surplus-authority replies with regression tests while preserving exact original-owner identity and generation checks.
  • Complete strict host/native validation for the correction and preserve ordinary-child event behavior.
  • Pass the original actual startup scenario with unchanged authority, output, retirement and timing requirements before accepting native startup or closing this report.
## Summary The authenticated process-event decoder admits only ordinary `LaunchChild` terminal replies even though the runtime dispatcher also owns `RealmMediator` children. A valid realm-mediator terminal reply is rejected before it reaches that dispatcher. Correction to the initial report: the observed VM exit `0xE5` does **not** identify Launchd or establish that this decoder defect caused the startup failure. Exsh uses the same exit status. The recorded exit endpoint is `0x520`, whereas Launchd is bound to `0x280`; these are different processes. The actual startup failure remains unresolved and no Posixd startup or readiness is accepted. ## Reproduction The source-bound decoder trigger is a successful, correctly correlated WAIT_PROCESS_EVENT reply with a nonzero process identity and generation, a valid terminal state/event pair, zero flags/receipts, and the `RealmMediator` role. At the selected Launchd revision, the explicit role check rejects it. This is an independently identified decoder defect; a regression test and correction are in progress. The associated native VM observation used Integration [`ffedb3630895b2bc547ab58e4900f750d84cd64b`](https://git.erikinkinen.fi/erix/integration/commit/ffedb3630895b2bc547ab58e4900f750d84cd64b) and its selected component graph, including Launchd `0bb56cc229fe4563da6c77ebe6ce9c178b903f26` and Procd `d04311abff67e7355bd92bb2e48fb6161e0ec674`, then run the maintained scenario: ```sh bash scripts/run-scenario.sh tests/scenarios/appliance-disk-image-realm-startup-positive.toml components.toml ``` This is one original local native VM observation, not a successful CI receipt or an unchanged-input retry. The original 120-second scenario deadline and 45-second progress watchdog were retained. Failure frequency and a last known working revision are unestablished. ## Expected and actual behavior Launchd should process authenticated terminal events for its original ordinary-child and realm-mediator owners, then remain available for the actual private startup sequence. The scenario requires two actual Posixd mediators, authenticated readiness, independent retirement, stale-handle rejection and successful shell exit. The serial receipt instead records, in order: ```text ERIX_EXSH:REALM_ADMISSION:VERIFIED ERIX_EXSH:REALM_PREPARATION:VERIFIED ERIX_KERNEL:PROCESS_EXIT_TCB=0000000000000022 ERIX_KERNEL:PROCESS_EXIT_EP=0000000000000520 ERIX_KERNEL:PROCESS_EXIT_CODE=00000000000000E5 ERIX_EXSH:REALM_ADMISSION:FAILED request=00000012 kind=0000000b first=000006d1 second=00000003 ERIX_KERNEL:PROCESS_EXIT_TCB=0000000000000001 ERIX_KERNEL:PROCESS_EXIT_CODE=00000000000000DC ``` The recorded TCB `0x22` exits through endpoint `0x520`, which differs from Launchd's bound endpoint `0x280`. Both Launchd and Exsh define an `0xE5` failure status, so that value alone does not establish which process exited or identify a Launchd process-event failure. Exsh's diagnostic `kind=0x0B` means a **valid typed rejected reply**, not transport loss: its first PREPARE for the actual Posixd sequence, request `0x12` / operation `0x6D1`, returns `NOT_FOUND` (`3`). Rootd then exits `0xDC`. The 45-second progress watchdog terminates QEMU after progress stops; the scenario command returns 1. The required `REALM_STARTUP:VERIFIED` marker is absent. That watchdog failure is preserved as a failed run, not a warning-free success. ## Environment and identity - Release native x86-64 QEMU/OVMF image, development runtime configuration, one virtual CPU, Rust 1.97.1; actual packaged service binaries and shared objects. - Integration driver: `ffedb3630895b2bc547ab58e4900f750d84cd64b`; Launchd: `0bb56cc229fe4563da6c77ebe6ce9c178b903f26`; Procd: `d04311abff67e7355bd92bb2e48fb6161e0ec674`. - Kernel: `5e285aa91a1423523b2a094d4ce0449ada39b16c`; Rootd: `2f1c7256eb77b6b29f1dad7d9b5028fdf8a7e527`; Exsh: `2239d5faabc0a52091af921484bee6bb2d558c0b`; Posixd: `809647d60544c4c6121868c99133f30d054965c9`. - The pinned shared orchestration component uses Integration `a787946077c918e9426a8eac741afe2d4e51372a`. It is distinct from the selected scenario driver above. - Packaged disk SHA-256 receipt: `e9a66bf5887f1d0fdb746080e0b6b10e5bd7923b02c8e2306e0d0586acf937b0`. The writable post-VM disk is retained separately; no byte-identity inference substitutes for runtime acceptance. - Authority remains the selected cwd scope, launch route and private realm capabilities. No global route, additional root, weakened identity check or timeout change was used to bypass the failure. ## Evidence and prior checks The retained evidence inventory covers 107 files. The complete serial log is 56,013 bytes, SHA-256 `a1132d376e101c370f06f7522f45139d34703e8d385847e848f6d4f84c43ae72`; its hash and length verify. Image construction has no recorded warnings. QEMU stderr retains the original progress-watchdog termination. Existing admission/preparation markers only prove those preceding checks; they do not establish actual Posixd startup, readiness or retirement. Source inspection identifies a concrete contract gap: [`validate_procd_wait_process_event_reply`](https://git.erikinkinen.fi/erix/launchd/src/commit/0bb56cc229fe4563da6c77ebe6ce9c178b903f26/src/process_lifecycle.rs#L463) admits only the `LaunchChild` role. The [runtime dispatcher](https://git.erikinkinen.fi/erix/launchd/src/commit/0bb56cc229fe4563da6c77ebe6ce9c178b903f26/src/runtime/dispatcher_process_event.rs) already checks realm ownership before ordinary jobs, but a valid `RealmMediator` event cannot reach it through that decoder. This source-level defect is independent of the VM exit attribution. The serial log does not expose a rejected WAIT_PROCESS_EVENT reply and does not prove that the decoder branch caused the PREPARE rejection. The selected [Exsh runtime](https://git.erikinkinen.fi/erix/exsh/src/commit/2239d5faabc0a52091af921484bee6bb2d558c0b/src/runtime/entry_platform.rs) also defines `0xE5`; its [realm diagnostic](https://git.erikinkinen.fi/erix/exsh/src/commit/2239d5faabc0a52091af921484bee6bb2d558c0b/src/runtime/launch_transport/realm_probe.rs) encodes typed rejection as kind 11. Source-bound decoder regression tests and separate diagnosis of the native PREPARE failure remain required. ## Tracking The same checkpoint fixes the independently reproduced WAIT decoder defect by accepting exactly ordinary-child and realm-mediator terminal roles before matching the retained process generation. Invalid identity, framing, state and authority remain rejected. All six default/all-feature/production development/release configurations pass 329 library and 66 runtime tests each, strict host/native Clippy, native builds, private rustdoc, formatting and documentation checks, with zero warnings. Authored Rust files remain below 1,000 lines. Final documentation corrections preserve the tested Rust behavior. The original failed VM remains retained. A newly selected image must pass the unchanged actual startup/readiness/retirement scenario before native acceptance; configuration/seal, four clients, byte I/O and guest toolchain/EriX builds remain open. [Private preparation issue 4](https://git.erikinkinen.fi/erix/launchd/issues/4) and [independent event issue 3](https://git.erikinkinen.fi/erix/launchd/issues/3) remain distinct. [WIP PR 2](https://git.erikinkinen.fi/erix/launchd/pulls/2) and [Phase 6 completion](https://git.erikinkinen.fi/erix/integration/issues/65) track the remaining acceptance. Track the correction in [Launchd PR 2](https://git.erikinkinen.fi/erix/launchd/pulls/2), [Launchd authority and lifecycle audit](https://git.erikinkinen.fi/erix/launchd/issues/1), [Integration validation](https://git.erikinkinen.fi/erix/integration/issues/3) and [Phase 6 completion](https://git.erikinkinen.fi/erix/integration/issues/65). - [ ] Cover valid realm-mediator terminal events and malformed, mismatched or surplus-authority replies with regression tests while preserving exact original-owner identity and generation checks. - [ ] Complete strict host/native validation for the correction and preserve ordinary-child event behavior. - [ ] Pass the original actual startup scenario with unchanged authority, output, retirement and timing requirements before accepting native startup or closing this report.
erikinkinen changed title from [BUG] Launchd exits while decoding realm mediator terminal events during native startup to [BUG] Process-event decoder rejects valid realm mediator terminal replies 2026-09-24 19:27:56 +02:00
erikinkinen added this to the Alpha 1 milestone 2026-10-30 09:34:31 +01:00
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
erix/launchd#3
No description provided.