[FEATURE] Add exclusive ordinary thread join and detach #41

Open
opened 2026-10-12 20:51:53 +02:00 by erikinkinen · 2 comments
Owner

Problem and motivation

Ordinary thread creation retains a genuine native member, but Posixd has no exclusive join/detach transaction, operational departure cleanup or distinction between native quota and a retained joinable result. A generic cancellable ByteCall can discard a completed response, so consuming a thread in one ordinary reply would leave cancelled joins unable to preserve joinability.

Implement an ordinary lifecycle path that waits for actual departure, delivers the full result once, releases original execution storage, and safely handles cancellation/reply loss. Callback publication alone cannot satisfy it.

Proposed behavior and scope

LibPosixABI owns shared authority-free birth and lifecycle codecs. Move the existing NBT1/NBR1/NBF1 codecs from Posixd rather than duplicating them for later libc use; update actual consumers and remove the old codec implementation. Preserve existing birth wire bytes and counted storage requirements.

Posixd adds DescribeCurrent, BeginJoin, CommitJoin, AbortOwnJoin and Detach through its original cap-free byte ingress. A logical handle contains the complete native process/thread pairs and counted mediator row/generation already present in birth results. DescribeCurrent derives that handle from actual successful CLAIM. It performs no service/identity discovery.

NJL1 requests are exactly 80 bytes: magic at 0, operation at 4, nonzero request correlation at 8, process pair at 16, thread pair at 24, row at 32, generation at 40, join epoch at 48 and zero bytes 56..80. Operations are DescribeCurrent=1, BeginJoin=2, CommitJoin=3, AbortOwnJoin=4 and Detach=5. DescribeCurrent/AbortOwnJoin require zero handle/epoch. BeginJoin/Detach require a complete nonzero handle and epoch 0. CommitJoin requires that handle and nonzero Ready epoch.

NJR1 replies are exactly 80 bytes: magic/operation/correlation at 0..16, result handle at 16..48, epoch at 48, full result at 56, flags at 64, errno at 68 and zero word at 72. Success flags are Ready=1, Committed=2, Aborted=4, Detached=8 and Described=16, matched exclusively to the operation with errno 0. Ready/Committed contain the original nonzero join epoch and full result. Aborted returns the new own fence epoch with zero handle/result. Detached echoes its exact handle with epoch/result 0. Described returns the actual admitted current handle with epoch/result 0. Refusal has flags 0, represented nonzero errno, epoch/result 0 and the requested handle, which is zero for description/abort. Mixed flags, reserved bytes, changed extent/correlation and capability-bearing frames refuse.

BeginJoin reserves one exclusive target against the genuine current joiner and server-owned join epoch, then waits on the native source-bound departure subscription. Ready returns the genuine complete result without consuming joinability. CommitJoin requires the same joiner/target/Ready epoch and positive native cleanup; only correlated positive native COMPLETE consumes joinability. AbortOwnJoin selects its actual caller, advances that execution's epoch and releases its own matching target reservation. Detach commits only same-process logical policy, keeps a live member's READ/watch/quota, and never terminates it.

A noncopy low-level JoinOwner retains each original immutable packet and native caller through collection and retirement. Before irreversible commit, its runtime checks deferred cancellation and enters a cancellation-disabled commit region. The owner cannot use generic cancel/discard after commit submission. This establishes the ordinary lifecycle protocol; the future libc wrapper must implement that actual runtime cancellation policy before complete pthread_join conformance is claimed.

Authority, security and reliability

Only successful original native CLAIM, configured client/session and the genuine admitted current ThreadKey authorize an operation. Handle/epoch bytes narrow that held scope and cannot grant membership, select a capability slot or probe another process. Preserve exact native process/thread generation, counted row generation, request/delivery owner and join epoch at every continuation. One original execution has one synchronous join owner; other executions, calls and clients remain independent.

Begin preflights all geometry, result budget and generation links before reserving. Self/cyclic waits return EDEADLK, stale/unknown/foreign handles ESRCH, and detached/already claimed targets EINVAL. Follow actual generation-checked join links in counted T storage without an extra traversal ceiling. Caught signals cannot synthesize exit or an EINTR join result.

The joiner first retires its original Begin caller, then completes/retires AbortOwnJoin before reporting cancellation complete or starting a successor. This works with declared C=1. Actual native RELEASE makes queued work Draining, while CLAIM selects only Queued records; an already claimed delivery is never requeued. Capture the server join epoch immediately after genuine CLAIM and revalidate it before every local reservation/publication. Abort's fence therefore prevents an older already claimed continuation from restoring an old link.

Ready does not detach or consume the target. Before Commit, the runtime performs the deferred-cancellation check and disables cancellation. Commit's immutable target/result/epoch and original caller remain owned until correlated collection and independent caller retirement, or process teardown. It does not expose result bytes to application memory before retirement. Unknown native completion, malformed output or uncertain cleanup freezes that exact continuation; it never resubmits Commit or borrows another caller.

Preflight every logical mutation and encode the full reply before COMPLETE. Correlated positive COMPLETE consumes the logical result once without a later fallible mutation. Known refused completion preserves the reaped joinable result; release only the cancelled reservation after independent serving retirement. A result positively accepted by Kernel remains available through its original caller even if the receiver later departs. Transport cancellation cannot claim application rollback.

Native departure comes only from the genuine original Member watch/READ, or actual Factory-derived initial observation. Store the complete opaque result, including zero/high bits. Close future intake for that departed origin and settle its original serving/provider/mask obligations independently. Retain its active watch through genuine accounting/machine quiescence. Then unregister, recheck original READ/type/serial/result, use existing original DROP and separately query absence. Never DROP an unknown occupant or repeat an uncertain effect. Initial join/reuse additionally requires positive automatic initial private cleanup; it cannot remove the native process anchor or lifetime charge 1.

A departed member frees native/local quota only after positive original disposal and fresh absence. Its retained joinable result still occupies counted T storage. Preserve nonwrapping row generations until join/detach clears the logical row. Native factory capacity remains initial-inclusive. Stack/TLS reuse requires positive original execution-reference cleanup; producer mappings remain actual shared process mappings. This neither grants a mapper nor fabricates an allocator free.

Detach changes policy only after positive COMPLETE; refused completion leaves joinability unchanged. A live detached member retains its wanted observation/storage until actual departure and positive native cleanup, then clears its logical result/row. Genuine member/initial joiner death releases that joiner's reservation without detaching or killing the target. Whole-process retirement invalidates same-process links before factory/receiver cleanup and keeps original member/watch/source uncertainty independently owned.

Initial logical consumption also retains a distinct native scalar cleanup duty. Preflight its complete original factory/serial/process/initial identity, immutable retirement request, original serving intent and counted storage before COMPLETE, but keep it dormant. Correlated positive original CommitJoin/Detach COMPLETE alone consumes policy and arms that duty through infallible assignments. No fallible local mutation or scalar-retirement effect follows COMPLETE in the same transaction. Cancelled, refused or uncertain completion preserves the independently wanted native/cached original result and does not arm cleanup.

A later fair turn uses narrow Factory WRITE to retire the initial scalar only after genuine ThreadExited/quiescence/positive private cleanup. The native operation also removes spent WRITE, preserving wanted READ|MINT. Its read-only original actor/selection reconciliation yields only RESULT_RETIRED metadata, never a scalar shadow. Keep initial admission's exact READ|WRITE|MINT receipt as immutable history and a separately proved reduced runtime phase; later births revalidate READ|MINT plus the original retirement proof rather than restoring or comparing the obsolete WRITE mask. Unrelated factories are unchanged.

Positive join has transferred the full result to Kernel's original caller, whose collection/retirement remains independent. Clear Posixd's consumed cache/result arena when server publication no longer needs it; the native retirement duty contains no result value. Positive live-initial detach retains watch/cleanup duty until actual departure, then erases the unused scalar through the same transition. A closed/source-revoked watch supplies no result or erasure proof. Unknown retirement keeps the exact duty and blocks initial logical reuse/rejoin until genuine retired acknowledgement or acknowledged original whole-process disposal; neither canonical source absence nor copied RESULT_RETIRED bytes settles it. The original initial row remains the retained process anchor and is never recycled into a member row. Born-member rows follow their own independent positive DROP/absence/watch and logical-consumption barriers.

Acceptance criteria

  • Relocate shared birth codecs without wire changes or duplicate/deprecated implementations, and add complete canonical lifecycle codecs with literal positive/negative framing controls.
  • Implement genuine current description, exclusive Begin/Ready/Commit/Abort and Detach through the original ingress, original native CLAIM and counted arenas; reject stale, foreign, replacement, self/cyclic and conflicting targets before effects.
  • Return a full zero/high-bit exit value only from genuine departure. Pending callbacks, absent carriers and last-thread finalization cannot satisfy join.
  • Exercise cancellation before claim and at every reservation/Ready/quiescence/reap/commit boundary, including C=1, old claimed continuation after abort, dead member/initial joiners, positive/refused/uncertain COMPLETE and accepted reply surviving receiver death.
  • Preflight dormant initial-result cleanup before COMPLETE, arm only after correlated positive consumption with no subsequent fallible local mutation, retain results on cancellation/refusal, and settle once-only retirement/mask/watch uncertainty before logical reuse.
  • Keep native registration/removal/DROP uncertainty separate from caller/delivery/result policy. Wrong serial/rights/type/generation, replaced slots and contradictory observations retain independent cleanup rather than touching a successor.
  • Prove initial-inclusive native charge, retained joinable logical row, positive member quota release, detached-live custody, nonwrapping logical reuse, and unchanged permanent initial anchor/charge after initial join/detach while its unwanted scalar and spent WRITE are positively erased.
  • Demonstrate genuine departure/join before reusing the same producer stack/TLS mapping for another member, then demonstrate a detached member's eventual positive cleanup through the actual native interfaces.
  • Extend the real producer-provisioned thread client and maintain its two-realm/two-client/T=4/C=2/38-process/120-second deployment envelopes and every prior byte/signal/close/EOF/process-exit oracle. Preserve the actual Exsh frame gate and distinguish an authored workload from an executed VM.
  • Keep service turns fair and bounded to one native effect, use generation-checked counted storage without hidden pools/limits, and profile the maintained lifecycle workload for actual bottlenecks.
  • Update rustdoc/missing_docs and relevant component/manual documentation. Keep libc allocation, dynamic TLS and the final pthread cancellation consumer explicitly open until they are implemented.

Alternatives and tradeoffs

A single consuming cancellable reply cannot restore joinability after the generic caller discards it. Staged Ready/Commit separates cancellable waiting from an irreversible, cancellation-disabled result transfer. Kernel retains native execution/storage ownership; Posixd owns POSIX exclusivity/detach policy.

Reaping resource custody separately from a retained logical result allows genuine native quota reuse without discarding a joinable exit value. It requires distinct arena states but avoids both leaked native charge and uncounted result storage. Existing DROP supplies the native reaping effect, so no broader authority is needed.

Tracking and rollout

Native prerequisite: departure observation and initial retirement.

This follows the source-bound native exit/watch and initial-private cleanup prerequisite, continues Posixd#34 and Kernel#47, and is tracked by Integration#65. Implement the native prerequisite first, relocate the pure codecs, then complete counted ordinary policy and the real client workload in one coherent signed dependency wave. Full pthread and guest self-hosting acceptance remain separate.

## Problem and motivation Ordinary thread creation retains a genuine native member, but Posixd has no exclusive join/detach transaction, operational departure cleanup or distinction between native quota and a retained joinable result. A generic cancellable ByteCall can discard a completed response, so consuming a thread in one ordinary reply would leave cancelled joins unable to preserve joinability. Implement an ordinary lifecycle path that waits for actual departure, delivers the full result once, releases original execution storage, and safely handles cancellation/reply loss. Callback publication alone cannot satisfy it. ## Proposed behavior and scope LibPosixABI owns shared authority-free birth and lifecycle codecs. Move the existing NBT1/NBR1/NBF1 codecs from Posixd rather than duplicating them for later libc use; update actual consumers and remove the old codec implementation. Preserve existing birth wire bytes and counted storage requirements. Posixd adds DescribeCurrent, BeginJoin, CommitJoin, AbortOwnJoin and Detach through its original cap-free byte ingress. A logical handle contains the complete native process/thread pairs and counted mediator row/generation already present in birth results. DescribeCurrent derives that handle from actual successful CLAIM. It performs no service/identity discovery. NJL1 requests are exactly 80 bytes: magic at 0, operation at 4, nonzero request correlation at 8, process pair at 16, thread pair at 24, row at 32, generation at 40, join epoch at 48 and zero bytes 56..80. Operations are DescribeCurrent=1, BeginJoin=2, CommitJoin=3, AbortOwnJoin=4 and Detach=5. DescribeCurrent/AbortOwnJoin require zero handle/epoch. BeginJoin/Detach require a complete nonzero handle and epoch 0. CommitJoin requires that handle and nonzero Ready epoch. NJR1 replies are exactly 80 bytes: magic/operation/correlation at 0..16, result handle at 16..48, epoch at 48, full result at 56, flags at 64, errno at 68 and zero word at 72. Success flags are Ready=1, Committed=2, Aborted=4, Detached=8 and Described=16, matched exclusively to the operation with errno 0. Ready/Committed contain the original nonzero join epoch and full result. Aborted returns the new own fence epoch with zero handle/result. Detached echoes its exact handle with epoch/result 0. Described returns the actual admitted current handle with epoch/result 0. Refusal has flags 0, represented nonzero errno, epoch/result 0 and the requested handle, which is zero for description/abort. Mixed flags, reserved bytes, changed extent/correlation and capability-bearing frames refuse. BeginJoin reserves one exclusive target against the genuine current joiner and server-owned join epoch, then waits on the native source-bound departure subscription. Ready returns the genuine complete result without consuming joinability. CommitJoin requires the same joiner/target/Ready epoch and positive native cleanup; only correlated positive native COMPLETE consumes joinability. AbortOwnJoin selects its actual caller, advances that execution's epoch and releases its own matching target reservation. Detach commits only same-process logical policy, keeps a live member's READ/watch/quota, and never terminates it. A noncopy low-level JoinOwner retains each original immutable packet and native caller through collection and retirement. Before irreversible commit, its runtime checks deferred cancellation and enters a cancellation-disabled commit region. The owner cannot use generic cancel/discard after commit submission. This establishes the ordinary lifecycle protocol; the future libc wrapper must implement that actual runtime cancellation policy before complete pthread_join conformance is claimed. ## Authority, security and reliability Only successful original native CLAIM, configured client/session and the genuine admitted current ThreadKey authorize an operation. Handle/epoch bytes narrow that held scope and cannot grant membership, select a capability slot or probe another process. Preserve exact native process/thread generation, counted row generation, request/delivery owner and join epoch at every continuation. One original execution has one synchronous join owner; other executions, calls and clients remain independent. Begin preflights all geometry, result budget and generation links before reserving. Self/cyclic waits return EDEADLK, stale/unknown/foreign handles ESRCH, and detached/already claimed targets EINVAL. Follow actual generation-checked join links in counted T storage without an extra traversal ceiling. Caught signals cannot synthesize exit or an EINTR join result. The joiner first retires its original Begin caller, then completes/retires AbortOwnJoin before reporting cancellation complete or starting a successor. This works with declared C=1. Actual native RELEASE makes queued work Draining, while CLAIM selects only Queued records; an already claimed delivery is never requeued. Capture the server join epoch immediately after genuine CLAIM and revalidate it before every local reservation/publication. Abort's fence therefore prevents an older already claimed continuation from restoring an old link. Ready does not detach or consume the target. Before Commit, the runtime performs the deferred-cancellation check and disables cancellation. Commit's immutable target/result/epoch and original caller remain owned until correlated collection and independent caller retirement, or process teardown. It does not expose result bytes to application memory before retirement. Unknown native completion, malformed output or uncertain cleanup freezes that exact continuation; it never resubmits Commit or borrows another caller. Preflight every logical mutation and encode the full reply before COMPLETE. Correlated positive COMPLETE consumes the logical result once without a later fallible mutation. Known refused completion preserves the reaped joinable result; release only the cancelled reservation after independent serving retirement. A result positively accepted by Kernel remains available through its original caller even if the receiver later departs. Transport cancellation cannot claim application rollback. Native departure comes only from the genuine original Member watch/READ, or actual Factory-derived initial observation. Store the complete opaque result, including zero/high bits. Close future intake for that departed origin and settle its original serving/provider/mask obligations independently. Retain its active watch through genuine accounting/machine quiescence. Then unregister, recheck original READ/type/serial/result, use existing original DROP and separately query absence. Never DROP an unknown occupant or repeat an uncertain effect. Initial join/reuse additionally requires positive automatic initial private cleanup; it cannot remove the native process anchor or lifetime charge 1. A departed member frees native/local quota only after positive original disposal and fresh absence. Its retained joinable result still occupies counted T storage. Preserve nonwrapping row generations until join/detach clears the logical row. Native factory capacity remains initial-inclusive. Stack/TLS reuse requires positive original execution-reference cleanup; producer mappings remain actual shared process mappings. This neither grants a mapper nor fabricates an allocator free. Detach changes policy only after positive COMPLETE; refused completion leaves joinability unchanged. A live detached member retains its wanted observation/storage until actual departure and positive native cleanup, then clears its logical result/row. Genuine member/initial joiner death releases that joiner's reservation without detaching or killing the target. Whole-process retirement invalidates same-process links before factory/receiver cleanup and keeps original member/watch/source uncertainty independently owned. Initial logical consumption also retains a distinct native scalar cleanup duty. Preflight its complete original factory/serial/process/initial identity, immutable retirement request, original serving intent and counted storage before COMPLETE, but keep it dormant. Correlated positive original CommitJoin/Detach COMPLETE alone consumes policy and arms that duty through infallible assignments. No fallible local mutation or scalar-retirement effect follows COMPLETE in the same transaction. Cancelled, refused or uncertain completion preserves the independently wanted native/cached original result and does not arm cleanup. A later fair turn uses narrow Factory WRITE to retire the initial scalar only after genuine ThreadExited/quiescence/positive private cleanup. The native operation also removes spent WRITE, preserving wanted READ|MINT. Its read-only original actor/selection reconciliation yields only RESULT_RETIRED metadata, never a scalar shadow. Keep initial admission's exact READ|WRITE|MINT receipt as immutable history and a separately proved reduced runtime phase; later births revalidate READ|MINT plus the original retirement proof rather than restoring or comparing the obsolete WRITE mask. Unrelated factories are unchanged. Positive join has transferred the full result to Kernel's original caller, whose collection/retirement remains independent. Clear Posixd's consumed cache/result arena when server publication no longer needs it; the native retirement duty contains no result value. Positive live-initial detach retains watch/cleanup duty until actual departure, then erases the unused scalar through the same transition. A closed/source-revoked watch supplies no result or erasure proof. Unknown retirement keeps the exact duty and blocks initial logical reuse/rejoin until genuine retired acknowledgement or acknowledged original whole-process disposal; neither canonical source absence nor copied RESULT_RETIRED bytes settles it. The original initial row remains the retained process anchor and is never recycled into a member row. Born-member rows follow their own independent positive DROP/absence/watch and logical-consumption barriers. ## Acceptance criteria - [x] Relocate shared birth codecs without wire changes or duplicate/deprecated implementations, and add complete canonical lifecycle codecs with literal positive/negative framing controls. - [ ] Implement genuine current description, exclusive Begin/Ready/Commit/Abort and Detach through the original ingress, original native CLAIM and counted arenas; reject stale, foreign, replacement, self/cyclic and conflicting targets before effects. - [ ] Return a full zero/high-bit exit value only from genuine departure. Pending callbacks, absent carriers and last-thread finalization cannot satisfy join. - [ ] Exercise cancellation before claim and at every reservation/Ready/quiescence/reap/commit boundary, including C=1, old claimed continuation after abort, dead member/initial joiners, positive/refused/uncertain COMPLETE and accepted reply surviving receiver death. - [ ] Preflight dormant initial-result cleanup before COMPLETE, arm only after correlated positive consumption with no subsequent fallible local mutation, retain results on cancellation/refusal, and settle once-only retirement/mask/watch uncertainty before logical reuse. - [ ] Keep native registration/removal/DROP uncertainty separate from caller/delivery/result policy. Wrong serial/rights/type/generation, replaced slots and contradictory observations retain independent cleanup rather than touching a successor. - [ ] Prove initial-inclusive native charge, retained joinable logical row, positive member quota release, detached-live custody, nonwrapping logical reuse, and unchanged permanent initial anchor/charge after initial join/detach while its unwanted scalar and spent WRITE are positively erased. - [ ] Demonstrate genuine departure/join before reusing the same producer stack/TLS mapping for another member, then demonstrate a detached member's eventual positive cleanup through the actual native interfaces. - [ ] Extend the real producer-provisioned thread client and maintain its two-realm/two-client/T=4/C=2/38-process/120-second deployment envelopes and every prior byte/signal/close/EOF/process-exit oracle. Preserve the actual Exsh frame gate and distinguish an authored workload from an executed VM. - [ ] Keep service turns fair and bounded to one native effect, use generation-checked counted storage without hidden pools/limits, and profile the maintained lifecycle workload for actual bottlenecks. - [ ] Update rustdoc/missing_docs and relevant component/manual documentation. Keep libc allocation, dynamic TLS and the final pthread cancellation consumer explicitly open until they are implemented. ## Alternatives and tradeoffs A single consuming cancellable reply cannot restore joinability after the generic caller discards it. Staged Ready/Commit separates cancellable waiting from an irreversible, cancellation-disabled result transfer. Kernel retains native execution/storage ownership; Posixd owns POSIX exclusivity/detach policy. Reaping resource custody separately from a retained logical result allows genuine native quota reuse without discarding a joinable exit value. It requires distinct arena states but avoids both leaked native charge and uncounted result storage. Existing DROP supplies the native reaping effect, so no broader authority is needed. ## Tracking and rollout Native prerequisite: [departure observation and initial retirement](https://git.erikinkinen.fi/erix/kernel/issues/52). This follows the source-bound native exit/watch and initial-private cleanup prerequisite, continues [Posixd#34](https://git.erikinkinen.fi/erix/posixd/issues/34) and [Kernel#47](https://git.erikinkinen.fi/erix/kernel/issues/47), and is tracked by [Integration#65](https://git.erikinkinen.fi/erix/integration/issues/65). Implement the native prerequisite first, relocate the pure codecs, then complete counted ordinary policy and the real client workload in one coherent signed dependency wave. Full pthread and guest self-hosting acceptance remain separate.
Author
Owner

Posixd now imports the shared birth and lifecycle framing from LibPosixABI at 2e058266a5e764c79b7c2f98470733bfdb1d5143. The duplicate codecs and forwarding module are removed. Native join/detach ownership, observation and result retirement are being implemented next.

Posixd now imports the shared birth and lifecycle framing from LibPosixABI at `2e058266a5e764c79b7c2f98470733bfdb1d5143`. The duplicate codecs and forwarding module are removed. Native join/detach ownership, observation and result retirement are being implemented next.
Author
Owner

The shared birth and lifecycle codecs are complete, with the former private codec removed. The staged client join owner and service policy are signed, including exclusive join reservations, cancellation fencing, initial-result retirement and detached-member cleanup.

The workload now performs joined, detached and joined births on the same guarded stack and TLS recipe. Its documented acceptance boundary still requires the complete image and genuine guest run; libc cancellation remains open.

The shared birth and lifecycle codecs are complete, with the former private codec removed. The staged client join owner and service policy are signed, including exclusive join reservations, cancellation fencing, initial-result retirement and detached-member cleanup. The workload now performs joined, detached and joined births on the same guarded stack and TLS recipe. Its [documented acceptance boundary](https://git.erikinkinen.fi/erix/integration/src/branch/feature/posix-compat/docs/realm-thread-witness.md) still requires the complete image and genuine guest run; libc cancellation remains open.
erikinkinen added this to the Alpha 1 milestone 2026-10-30 09:35:08 +01:00
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
erix/posixd#41
No description provided.