[BUG] Private Provision assumes an endpoint SEND source is moved #7

Closed
opened 2026-09-24 20:52:56 +02:00 by erikinkinen · 0 comments
Owner

Summary

Launchd treats the configuration SEND passed to private Provision as a moved capability, but native owned invocation copies ordinary endpoint bindings. After a successful Provision result and caller retirement, Launchd queries its original SEND source, finds the expected retained copy, and fails with SourceAuthority. Public START then returns INTERNAL before native child Start.

A bounded hardware-breakpoint diagnostic of the exact failing image identifies this boundary. The host transport fixture incorrectly removes the SEND source at SUBMIT and therefore misses the actual native behavior.

Reproduction

At Integration 8f9b7d752c4769e3017aa4116ad0a542f54f488b and Launchd cf6d4a70d5423dca65bbab5e8092fd374184c4ed, run:

bash scripts/run-scenario.sh tests/scenarios/appliance-disk-image-realm-startup-positive.toml components.toml

The retained original execution completes admission and guarded preparation, then rejects request 0x13, operation 0x6D4, with result 5 (INTERNAL). A separate changed-debugger execution of the unchanged image captures Launchd's first failure while retaining a successful private Provision reply (0x433, result zero, Provisioned). This diagnostic is not an acceptance run or a failure-frequency estimate.

A deterministic host regression should preserve the ordinary configuration SEND source when SUBMIT succeeds, return the matching Provision acknowledgment and finish the original caller's retirement. The present implementation then fails its post-Provision source query. Unique InstallGrant and RevocationGrant transfers must continue using their existing MOVE behavior.

Expected and actual behavior

After validating Provision and retiring its original caller, Launchd must explicitly dispose of its own configuration SEND copy and independently prove that exact source slot absent before constructing native Start. The receiving child retains its intended SEND descendant; dropping a local copied binding must not be confused with revoking the descendant. Any disposal refusal, failed absence query or retained source must prevent Start and preserve cleanup ownership.

Instead, advance_call calls verify_column(2) without a preceding local drop because it assumes native transfer consumed the source. The actual query succeeds, producing SourceAuthority; the public mapper returns INTERNAL. The exact diagnostic capture is still in phase Provision with no previous failure, a valid 224-byte Provision reply, and the original configuration source marked owned. No native Start, Posixd readiness or running retirement is established.

Environment and identity

  • Integration 8f9b7d752c4769e3017aa4116ad0a542f54f488b; Launchd cf6d4a70d5423dca65bbab5e8092fd374184c4ed; Procd 4f3b01bf21932e86ac542f07e6ade05138cc0473; Kernel 5e285aa91a1423523b2a094d4ce0449ada39b16c; Posixd 5f54a55a5f189559352c7d0e6a2af55aaa37c44f.
  • Release native x86-64 full-service QEMU/OVMF image, one virtual CPU, Rust 1.97.1. Original guest limits remain 120 seconds hard and 45 seconds without serial progress.
  • Image SHA-256: 396e15c2ba4f81bf2d39a86eb11d6b536588b32ad671c02a724ced602906c65b. Both the retained original and diagnostic copy remain unchanged after diagnosis.
  • Only the existing scoped preparation, private configuration route and exact child generation are involved. No additional guest endpoint, process grant, fallback route or log authority was introduced.
  • Corrected full startup passes at Integration 66ddc78da31dd03dd554d0677cdc2af86588d525.

Evidence and prior checks

The maintained appliance-disk-image-realm-startup-positive scenario passes unchanged: both actual Posixd processes reach authenticated PrivateReady; A retires while B stays ready; B then retires; both stale handles are rejected; the initial shell exits successfully. The original required, unique, ordered and forbidden markers, expected guest exit 0x10, 120-second guest hard limit and 45-second serial-progress limit all pass.

This exercises counted minimum startup, actual capability census, setup-grant consumption and copied SEND disposal before Start, followed by native child registration, exact-origin readiness and the controller challenge. The packaged Posixd alias selects the same authenticated executable; no ordinary sidecar, extra child endpoint or fallback is introduced. Source-bound exercised gates are the evidence; no separate global inventory dump is claimed. Posixd's Ready caller remains its separately owned obligation until retirement. The successful explicit-retirement path does not exercise Launchd's spontaneous WAIT decoder, final CPU/profiler handoff, repeated row reuse, loss paths or capacity exhaustion.

Launchd's six strict local configurations pass 329 library and 69 runtime tests each; all 48 required commands are warning-free. The corrected native-faithful 14-test group includes COPY source retention, post-Provision caller retirement, separate source disposal/absence verification and failure preservation. Original Launchd CI155/156 also passes: four complete logs, 148,673 bytes, zero warning/failure candidates. Original Integration CI1791/1792 remains queued; earlier failures and regression reports remain preserved. The copied-SEND defect and its native startup regression are resolved at this scoped boundary. Relevant implementation reviews remain Launchd PR2, Procd PR2, Posixd PR5 and Integration PR12.

Configuration/seal/publication, four real clients and mediated byte I/O, complete running retirement and profiler disposition, rebuilding the external Rust/LLVM toolchain inside EriX and both complete EriX guest builds remain required and unproved. The WIP reviews remain unfinished.

Native Start is constructed only after explicit local SEND disposal and independent absence proof.

The native-faithful copied_configuration_source_retires_after_provision_before_start control fails against unchanged production cf6d4a70d5423dca65bbab5e8092fd374184c4ed with Some(SourceAuthority) versus None, exit 101. The corrected 14-test focused group passes, including source retention through Provision/RELEASE, separate disposal/query turns and both disposal/query failures. The original failing control and its source identity remain retained.

All six default/all-feature/runtime-feature development and release selections pass 329 library plus 69 runtime tests each. Formatting, strict host/native Clippy, native builds, warning-denied host/native rustdoc, doctests, dependency and Markdown checks complete with zero warnings. These checks cover the frozen published source. Original CI is monitored separately; no current CI success is inferred here. The subsequent changed-image startup/readiness/independent-retirement acceptance above closes this scoped bug.

Launchd's post-Provision check assumes a successful MOVE consumed its original SEND. The independent local query returns SourceAuthority when that slot still contains a capability. The host fixture instead deletes the source during SUBMIT.

Kernel owned invocation admission uses the native transfer transaction: InstallGrant and RevocationGrant move, while endpoint bindings use copy_cap_between_with_rights. The source remaining present is correct Kernel behavior.

The diagnostic uses hardware breakpoints and a private host Unix socket. Symbol carriers preserve all packaged PT_LOAD bytes and geometry; the running executable-load bytes are also checked against the packaged ELF. The first RetainedRealmStart::fail entry has prior failure None, Provision phase, and SourceAuthority. Exact selected disassembly independently confirms the phase offset and that this error is produced by a successful capability query. The retained reply proves Provision completion arrived; an unhit optimized Procd completion symbol is not evidence that completion failed.

The original acceptance attempt remains failed: command exit 1 after 169.570989 seconds including image construction, 107 retained artifacts, 56,013 serial bytes, zero image-warning candidates and no process-cleanup failure. The diagnostic also ends at the original progress watchdog, exit 1 after 95.831832 seconds. Its GDB index-cache warning and repeated-SIGTERM wrapper traceback are retained as host diagnostic issues; QEMU and GDB are both reaped, and no cleanup error remains. They do not alter the captured first guest failure or count as warning-free diagnostic execution. No raw guest memory or private environment is published.

Tracking

Integration's original INTERNAL failure, Launchd lifecycle audit, Launchd PR 2, Integration PR 12, and Phase 6 completion.

  • Attribute the original failing boundary with exact-image native evidence and independently verified source/disassembly.
  • Reproduce the defect using the actual endpoint COPY semantics in the host fixture; preserve unique-grant MOVE controls.
  • Explicitly dispose and independently verify the original copied SEND before Start, retaining refusal and cleanup behavior.
  • Pass formatting, strict Clippy, unit tests, warning-denied host/native builds and documentation checks.
  • Pass the maintained two-Posixd startup, readiness and independent-retirement scenario after adopting corrected source, without changing its authority or timing gates.
## Summary Launchd treats the configuration SEND passed to private Provision as a moved capability, but native owned invocation copies ordinary endpoint bindings. After a successful Provision result and caller retirement, Launchd queries its original SEND source, finds the expected retained copy, and fails with `SourceAuthority`. Public START then returns INTERNAL before native child Start. A bounded hardware-breakpoint diagnostic of the exact failing image identifies this boundary. The host transport fixture incorrectly removes the SEND source at SUBMIT and therefore misses the actual native behavior. ## Reproduction At Integration `8f9b7d752c4769e3017aa4116ad0a542f54f488b` and Launchd `cf6d4a70d5423dca65bbab5e8092fd374184c4ed`, run: ```sh bash scripts/run-scenario.sh tests/scenarios/appliance-disk-image-realm-startup-positive.toml components.toml ``` The retained original execution completes admission and guarded preparation, then rejects request `0x13`, operation `0x6D4`, with result `5` (INTERNAL). A separate changed-debugger execution of the unchanged image captures Launchd's first failure while retaining a successful private Provision reply (`0x433`, result zero, Provisioned). This diagnostic is not an acceptance run or a failure-frequency estimate. A deterministic host regression should preserve the ordinary configuration SEND source when SUBMIT succeeds, return the matching Provision acknowledgment and finish the original caller's retirement. The present implementation then fails its post-Provision source query. Unique InstallGrant and RevocationGrant transfers must continue using their existing MOVE behavior. ## Expected and actual behavior After validating Provision and retiring its original caller, Launchd must explicitly dispose of its own configuration SEND copy and independently prove that exact source slot absent before constructing native Start. The receiving child retains its intended SEND descendant; dropping a local copied binding must not be confused with revoking the descendant. Any disposal refusal, failed absence query or retained source must prevent Start and preserve cleanup ownership. Instead, `advance_call` calls `verify_column(2)` without a preceding local drop because it assumes native transfer consumed the source. The actual query succeeds, producing `SourceAuthority`; the public mapper returns INTERNAL. The exact diagnostic capture is still in phase Provision with no previous failure, a valid 224-byte Provision reply, and the original configuration source marked owned. No native Start, Posixd readiness or running retirement is established. ## Environment and identity - Integration `8f9b7d752c4769e3017aa4116ad0a542f54f488b`; Launchd `cf6d4a70d5423dca65bbab5e8092fd374184c4ed`; Procd `4f3b01bf21932e86ac542f07e6ade05138cc0473`; Kernel `5e285aa91a1423523b2a094d4ce0449ada39b16c`; Posixd `5f54a55a5f189559352c7d0e6a2af55aaa37c44f`. - Release native x86-64 full-service QEMU/OVMF image, one virtual CPU, Rust 1.97.1. Original guest limits remain 120 seconds hard and 45 seconds without serial progress. - Image SHA-256: `396e15c2ba4f81bf2d39a86eb11d6b536588b32ad671c02a724ced602906c65b`. Both the retained original and diagnostic copy remain unchanged after diagnosis. - Only the existing scoped preparation, private configuration route and exact child generation are involved. No additional guest endpoint, process grant, fallback route or log authority was introduced. - Corrected full startup passes at Integration `66ddc78da31dd03dd554d0677cdc2af86588d525`. ## Evidence and prior checks The maintained `appliance-disk-image-realm-startup-positive` scenario passes unchanged: both actual Posixd processes reach authenticated PrivateReady; A retires while B stays ready; B then retires; both stale handles are rejected; the initial shell exits successfully. The original required, unique, ordered and forbidden markers, expected guest exit `0x10`, 120-second guest hard limit and 45-second serial-progress limit all pass. This exercises counted minimum startup, actual capability census, setup-grant consumption and copied SEND disposal before Start, followed by native child registration, exact-origin readiness and the controller challenge. The packaged Posixd alias selects the same authenticated executable; no ordinary sidecar, extra child endpoint or fallback is introduced. Source-bound exercised gates are the evidence; no separate global inventory dump is claimed. Posixd's Ready caller remains its separately owned obligation until retirement. The successful explicit-retirement path does not exercise Launchd's spontaneous WAIT decoder, final CPU/profiler handoff, repeated row reuse, loss paths or capacity exhaustion. Launchd's six strict local configurations pass 329 library and 69 runtime tests each; all 48 required commands are warning-free. The corrected native-faithful 14-test group includes COPY source retention, post-Provision caller retirement, separate source disposal/absence verification and failure preservation. Original Launchd CI155/156 also passes: four complete logs, 148,673 bytes, zero warning/failure candidates. Original Integration CI1791/1792 remains queued; earlier failures and regression reports remain preserved. [The copied-SEND defect](https://git.erikinkinen.fi/erix/launchd/issues/7) and [its native startup regression](https://git.erikinkinen.fi/erix/integration/issues/89) are resolved at this scoped boundary. Relevant implementation reviews remain [Launchd PR2](https://git.erikinkinen.fi/erix/launchd/pulls/2), [Procd PR2](https://git.erikinkinen.fi/erix/procd/pulls/2), [Posixd PR5](https://git.erikinkinen.fi/erix/posixd/pulls/5) and [Integration PR12](https://git.erikinkinen.fi/erix/integration/pulls/12). Configuration/seal/publication, four real clients and mediated byte I/O, complete running retirement and profiler disposition, rebuilding the external Rust/LLVM toolchain inside EriX and both complete EriX guest builds remain required and unproved. The WIP reviews remain unfinished. Native Start is constructed only after explicit local SEND disposal and independent absence proof. The native-faithful `copied_configuration_source_retires_after_provision_before_start` control fails against unchanged production `cf6d4a70d5423dca65bbab5e8092fd374184c4ed` with `Some(SourceAuthority)` versus `None`, exit 101. The corrected 14-test focused group passes, including source retention through Provision/RELEASE, separate disposal/query turns and both disposal/query failures. The original failing control and its source identity remain retained. All six default/all-feature/runtime-feature development and release selections pass 329 library plus 69 runtime tests each. Formatting, strict host/native Clippy, native builds, warning-denied host/native rustdoc, doctests, dependency and Markdown checks complete with zero warnings. These checks cover the frozen published source. Original CI is monitored separately; no current CI success is inferred here. The subsequent changed-image startup/readiness/independent-retirement acceptance above closes this scoped bug. [Launchd's post-Provision check](https://git.erikinkinen.fi/erix/launchd/src/commit/cf6d4a70d5423dca65bbab5e8092fd374184c4ed/src/realm_start_progress.rs#L136) assumes a successful MOVE consumed its original SEND. [The independent local query](https://git.erikinkinen.fi/erix/launchd/src/commit/cf6d4a70d5423dca65bbab5e8092fd374184c4ed/src/realm_start_route.rs#L110) returns `SourceAuthority` when that slot still contains a capability. The [host fixture](https://git.erikinkinen.fi/erix/launchd/src/commit/cf6d4a70d5423dca65bbab5e8092fd374184c4ed/src/runtime/realm_start_tests_io.rs) instead deletes the source during SUBMIT. [Kernel owned invocation admission](https://git.erikinkinen.fi/erix/kernel/src/commit/5e285aa91a1423523b2a094d4ce0449ada39b16c/src/kobj/endpoint/invocation/admission.rs#L81) uses [the native transfer transaction](https://git.erikinkinen.fi/erix/kernel/src/commit/5e285aa91a1423523b2a094d4ce0449ada39b16c/src/kobj/endpoint/transfer_transaction.rs#L76): InstallGrant and RevocationGrant move, while endpoint bindings use `copy_cap_between_with_rights`. The source remaining present is correct Kernel behavior. The diagnostic uses hardware breakpoints and a private host Unix socket. Symbol carriers preserve all packaged PT_LOAD bytes and geometry; the running executable-load bytes are also checked against the packaged ELF. The first `RetainedRealmStart::fail` entry has prior failure None, Provision phase, and `SourceAuthority`. Exact selected disassembly independently confirms the phase offset and that this error is produced by a successful capability query. The retained reply proves Provision completion arrived; an unhit optimized Procd completion symbol is not evidence that completion failed. The original acceptance attempt remains failed: command exit 1 after 169.570989 seconds including image construction, 107 retained artifacts, 56,013 serial bytes, zero image-warning candidates and no process-cleanup failure. The diagnostic also ends at the original progress watchdog, exit 1 after 95.831832 seconds. Its GDB index-cache warning and repeated-SIGTERM wrapper traceback are retained as host diagnostic issues; QEMU and GDB are both reaped, and no cleanup error remains. They do not alter the captured first guest failure or count as warning-free diagnostic execution. No raw guest memory or private environment is published. ## Tracking [Integration's original INTERNAL failure](https://git.erikinkinen.fi/erix/integration/issues/89), [Launchd lifecycle audit](https://git.erikinkinen.fi/erix/launchd/issues/1), [Launchd PR 2](https://git.erikinkinen.fi/erix/launchd/pulls/2), [Integration PR 12](https://git.erikinkinen.fi/erix/integration/pulls/12), and [Phase 6 completion](https://git.erikinkinen.fi/erix/integration/issues/65). - [x] Attribute the original failing boundary with exact-image native evidence and independently verified source/disassembly. - [x] Reproduce the defect using the actual endpoint COPY semantics in the host fixture; preserve unique-grant MOVE controls. - [x] Explicitly dispose and independently verify the original copied SEND before Start, retaining refusal and cleanup behavior. - [x] Pass formatting, strict Clippy, unit tests, warning-denied host/native builds and documentation checks. - [x] Pass the maintained two-Posixd startup, readiness and independent-retirement scenario after adopting corrected source, without changing its authority or timing gates.
erikinkinen added this to the Alpha 1 milestone 2026-10-30 09:34:32 +01:00
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
erix/launchd#7
No description provided.