[FEATURE] Retain independent native invocations within one acquisition scope #27
Labels
No labels
alpha-1
bug
ci
docs
duplicate
enhancement
help wanted
invalid
performance
phase-6
question
refactor
security
wontfix
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
erix/vfsd#27
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Problem and motivation
Safe overlapping compatibility calls require Close and independent filesystem operations to progress while another thread waits for an advisory lock. Posixd currently selects one acquisition-scope endpoint per client. Vfsd's exact serving path retains one invocation and one lock waiter in the scope's row: while owned_pending is true it advances that owner and does not claim another request. Removing mediator exclusion alone therefore leaves a blocked Wait occupying the route needed for Close.
This is an explicit remaining concurrency limitation. The current mediator still serializes same-process intake. The delayed candidate-retirement defect is separately repaired in Vfsd#26.
Proposed behavior and scope
Retain independently counted original serving invocations under one admitted acquisition scope. A blocking lock Wait must not prevent intake and progress of Close or independent original-object operations on that scope. Bind each admitted invocation to the original receiver/scope generation and retain every selected pin or reservation until its actual effect/publication/cancellation cleanup settles.
Separate receiver/scope ownership from per-call serving custody using actual deployment-counted storage. Preserve the existing scope authority and original candidate selection. Audit lock reservations, mutation journals, complete/cancel transitions, terminal drain and legacy dispatch together. Coordinate with Posixd's per-call source holds and pending process-close policy; that coordinator must protect uncertain ReleaseProcess retries from later process-lock mutations without retaining a barrier for an entire blocking wait.
Affected components are Vfsd and Posixd, with Docs/manual and Integration source/layout adoption where needed. Any wire or target-layout change must update its actual producers and consumers coherently. This remains part of the whole local-stream/concurrent-call requirement; it does not replace libc, native threads/TLS, imported consumers or complete guest-build acceptance.
Authority, security and reliability
The existing native receiver and admitted scope confer authority. Client PIDs, fd numbers, copied inode values and payload thread IDs do not. Bind invocation keys to original receiver/scope generations; stale completion, cancellation and reused storage must never select a newer owner. Retain original pins through suspended native operations and use positive original cleanup evidence before freeing custody.
Explicit process Close must cancel appropriate old reservations and settle its own retained policy before later mutations publish. Candidate disposal remains narrow and must not replay any-close policy. Retiring a scope must fence intake and drain every original serving/caller/wait/journal owner before provider disposal or receiver reuse. No hidden allocator, arbitrary waiter ceiling or reconstructed capability is permitted. Preserve bounded work per service turn and real activity wakeups.
Detailed scope and contracts
Fair original native intake and Close reply custody — 4 October 2026
Retiring scopes and removed metadata stop intake; no separate parallelism cap, heap allocator, per-thread scope grant or packet thread-ID authority is added.
One-shot receivers remain exclusive through refused completion and original retirement. Final ordinary settlement yields the existing preferred legacy RECV turn. Refused COMPLETE remains attempted; draining and receiver-disposal refusal retain original custody. Caller retirement cannot prove semantic cleanup; private revocation may suppress Close's reply while cleanup still finishes. Final scope clearing requires every original call and positive receiver absence. The association is descriptive and adds no capability; actual updated type size/alignment drive exported target layout and runtime storage without a wire or authority ABI change.
They cover real peer-blocked Wait with provider work/query, multiple original waits and process cleanup, Close after older Wait drain, refused Close/draining/delayed disposal, private revocation/provider refusal, counted saturation/recovery, one-shot exclusivity, final-settlement legacy opportunity, valid/contradictory Close associations, original child disposal through provider refusal/caller loss/private abandonment, and receiver-generation refusal before provider retry. Full Posixd uncertain process-close coordination and overlapping intake, per-call masks/TLS, complete libc/imported consumers, same-process native overlap workload sources remain required. Native caller retirement cannot prove semantic cleanup or permit old cleanup to release newer process locks. The whole socket feature remains open.
Completed source criteria are marked despite deferred execution; missing implementation remains open. Both complete EriX guest generations and external Rust/LLVM guest rebuilds remain mandatory and unperformed.
Independent native custody and fair blocked-Wait intake are source-complete; native serving has five completed source criteria and three open criteria: fd/process-lifetime coordination, the retained Posixd close coordinator, and complete same-process native VM workload sources. Recalculated phase progress is 21.11% weighted; 9.61% whole acceptance; 32/460 parents and 495/965 children across 167 decomposed parents.
Original first-submission key order — 4 October 2026
Fair Vfsd intake must dispatch each newly claimed request before yielding so later claims cannot overtake the monotonic admission fence. The existing candidate and Wait replay/absence fences remain intact; this adds no scope mutex, grant, tombstone ledger, allocator or independent waiter cap. Commit/Cancel and uncertain submission keep the original key and normalized intent; original native retirement precedes idempotent cleanup and cannot itself prove semantic rollback.
Six controls are authored and unrun: delayed EOF with reordered Wait admission; a newer actual first submission while another Wait stays pending; pre-submission cancellation without consuming a key; actual packet and EOF-coordinate refusal before issuance; uncertain original submission and cleanup after a newer key; and full-width final-key exhaustion.
Fair overlap, one-shot/legacy lifetimes, full terminal drain integration and uncertain process-close coordination remain unfinished in independent native invocations and local stream sockets. Both features remain open with all eight whole criteria unchecked. Per-call masks/TLS, complete libc frontends and imported-tool closure remain required in the same whole socket scope.
Both complete EriX guest generations and external Rust/LLVM rebuilding inside EriX remain mandatory and unperformed.
Deferred execution does not prevent marking completed implementation; unfinished whole serving and guest-build requirements remain open.
Original advisory provider-pin custody — 4 October 2026
Provider disposal refuses active uses before effects. This creates no additional provider acquisition, native grant, RPC, allocator or per-process concurrency cap.
Original generations, both reservation links and captured intent must agree. Stale saved success and foreign terminal correlation refuse; attempted COMPLETE history remains unchanged without replay.
Eight controls are authored and unrun: immediate publication and uncertain completion; readiness/Commit transfer; prepared cancellation and stale-result refusal; original pin disposal preserving another pin's newer locks; quarantine preserving published before-images; quarantine of an immediate acquisition; no-row cleanup ticket order; and contradictory scope generation before effects. Existing retirement controls now require independent positive cancellation instead of stale success.
Public pre-submission key ordering, independent intake, one-shot/legacy fairness and uncertain process-close coordination remain unfinished in independent native invocations and local stream sockets. Both owning features remain open with all eight whole criteria unchecked. Complete libc frontends, per-call masks/TLS and imported-tool closure remain required in the same whole socket scope.
Both complete EriX guest generations and external Rust/LLVM rebuilds inside EriX remain mandatory and unperformed.
Implementation completion is not gated on those deferred executions; incomplete serving and guest-build requirements remain open.
Whole-receiver native reply abandonment — 4 October 2026
Actual reference count, links, tail, progress cursor and original non-reused generations must agree.
Later progress cannot dispatch the abandoned request, reconstruct a result, attach a new reservation or resume provider arbitration. An earlier accepted or refused COMPLETE keeps its original decision and is never replayed. Provider and receiver refusal retain their own original cleanup barriers.
The cached lock-success defect, Vfsd#28, is closed as source-complete using bug_report with bug/phase-6 metadata and all four criteria checked under the user testing waiver. The original serial serving path could roll back an advisory lock, refuse provider retirement and leave its cached Complete result eligible for native delivery.
Seven new controls are authored and unrun: full roster abandonment preserving an independent receiver and an unclaimed association; malformed roster refusal before any saved result changes; permanent abandonment across movement/live probes; accepted/refused native completion history; multiple-call native/receiver drain; private scope cleanup through provider/receiver refusal; and the minimal serial cached-success regression.
Suspended-operation pins, public pre-submission key ordering, independent intake and one-shot/legacy fairness remain unfinished in Vfsd#27. All eight native-serving and all eight whole socket criteria remain unchecked; no artificial acceptance credit is added.
Native threads/TLS, complete libc frontends and imported consumers remain required. Both full EriX guest generations AND external Rust/LLVM rebuilding inside EriX remain mandatory and unperformed.
Native mutation admission — 4 October 2026
Disjoint requests inside one original before-image still depend on that journal so rollback cannot overwrite newer state. Capacity refusal and duplicate original change admission preserve all rows before any effect.
Public continuation token zero cannot select these native queues.
Cancellation rolls back effects while retaining the original typed result through actual completion or positive retirement; it never redirects cleanup into a newer provider pin or repeats attempted COMPLETE.
Nine new controls are authored and unrun: independent objects/owners and separately settled intervals; shared before-images and merge neighbors; atomic capacity refusal; original Unlock delivery; process-close publication; ordered native cleanup calls; original pin retirement preserving newer locks; and counted queue exhaustion preserving the original caller and effect.
Suspended-operation pins, public pre-submission key ordering, independent native intake, generalized all-call terminal drain and one-shot/legacy fairness remain unfinished in Vfsd#27. All eight native-serving and all eight whole socket criteria remain unchecked; no artificial acceptance credit is added.
Native threads/TLS, complete libc frontends and imported consumers remain required. Both full EriX guest generations AND external Rust/LLVM rebuilding inside EriX remain mandatory and unperformed.
Independent advisory reservation custody — 4 October 2026
Exhaustion refuses before consuming the client's continuation key or installing a call link.
Ordinary progress follows the actual link; copied snapshots cannot replace custody. Commit and Cancel select only the original scope's continuation key and require fully correlated intent. Contradictory generations propagate refusal before arbitration or cleanup. Exact Commit requires this positive readiness boundary and attaches its own original delivery; early or repeated Commit preserves the reservation.
An unsubmitted readiness or Commit result becomes a correlated refusal; an attempted COMPLETE is never replayed. A terminal reservation cannot resume provider arbitration after pin retirement. Scope reuse and call clearing refuse retained reservation custody.
Admission based on actual conflicting mutations remains required. Production intake still progresses an existing call before claiming another; interleaved native intake, suspended-operation pins, generalized all-call terminal drain and one-shot/legacy fairness remain unfinished in Vfsd#27. All eight native-serving and all eight whole socket criteria remain unchecked; no artificial acceptance credit is added.
Native threads/TLS, complete libc frontends and imported consumers remain required. Both full EriX guest generations AND external Rust/LLVM rebuilding inside EriX remain mandatory and unperformed.
Original native request dispatch — 3 October 2026
Dispatch copies the retained packet into scratch; scratch contents cannot substitute another request. Unclaimed allocation, malformed native intake, endpoint substitution, replay and abandoned or settled delivery cannot supply dispatch proof. Malformed application framing consumes readiness before refusal.
Legacy advisory mutations, including Commit, Cancel and process-close cleanup, refuse even while another native call is pending on the scope. Queued waits retain their original call identity; readiness and native settlement cannot borrow another delivery.
Six new controls are authored and unrun: unusable intake and abandonment; one-use dispatch across movement/refusal; legacy mutations during an unrelated native query; accepted/refused Try and Commit from spare call rows; spare-call readiness followed by Commit; and unclaimed/malformed request replay.
Independent multi-waiter custody, exact reservation lookup, cancellation drain, conflict-journal admission, suspended-operation pins and fair all-call terminal drain remain unfinished in Vfsd#27. All eight native-serving and all eight whole socket criteria remain unchecked; no artificial acceptance credit is added.
Native threads/TLS, complete libc frontends and imported consumers remain required. Both full EriX guest generations AND external Rust/LLVM rebuilding inside EriX remain mandatory and unperformed.
Counted native call allocation — 3 October 2026
No separate per-process call cap, new buffer, allocator or native grant is introduced.
Scope clearing refuses retained receiver associations. Stale call keys, replacement generations and contradictory neighbors refuse before native progress or unlinking. Native submission also refuses a substituted endpoint index or generation before CLAIM.
Four controls are authored and unrun: counted exhaustion and roster unlinking with receiver retention after metadata removal; stale call keys and replacement receiver generations; idle CLAIM cleanup through the real serving path; and rejection of substituted endpoint coordinates before native submission.
Explicit invocation-versus-scope dispatch identity, independent reservation ownership after readiness, exact Commit/Cancel lookup, conflict-journal admission, suspended-operation pins and all-call terminal drain remain unfinished in Vfsd#27. All eight native-serving and all eight whole socket criteria remain unchecked; no artificial acceptance credit is added.
Native threads/TLS, complete libc frontends and imported consumers remain required. Both full EriX guest generations AND external Rust/LLVM rebuilding inside EriX remain mandatory and unperformed.
Native call custody — 3 October 2026
COMPLETE and provider/receiver cleanup use that retained endpoint independently of the call's current storage location; replacement generations refuse before native progress or cleanup. Copied endpoint coordinates confer no authority without the retained native owner.
Moving a call preserves its prepared bytes, completion-attempt history, native acceptance/retirement and original advisory settlement. A refused COMPLETE cannot be retried after movement. Clearing now refuses until positive native settlement and consumption of the original journal decision; refusal preserves both custody and bytes. No extra buffer, allocator, wire operation or grant is introduced.
The other moves a refused delivery and requires positive original cancellation retirement plus journal consumption before clearing, without a repeated COMPLETE.
Independent allocation, original receiver/scope holds, multi-waiter reservation lookup and cancellation, conflicting-journal admission, fair progress and all-call terminal drain remain unfinished in Vfsd#27. The process's single admitted acquisition-scope route can still be occupied by a blocked native Wait; a mediator-only fence cannot make Close progress there. All eight native-serving criteria and all eight whole socket criteria remain unchecked; no artificial acceptance credit is added.
The completed candidate-retirement repair remains selected: delayed native pin disposal stays narrow and cannot restore retired custody from rollback records. Complete native thread/TLS semantics, libc frontends and imported consumers remain required. Both full EriX guest generations AND external Rust/LLVM rebuilding inside EriX remain mandatory and unperformed.
Acceptance criteria
Checklist
Alternatives and tradeoffs
A mediator-only mutex does not free the native scope's occupied serving slot. Cancelling unrelated waits simply to admit ordinary operations changes their semantics and can cause starvation. Per-thread scope clones introduce separate native ownership and process-lock policy questions; they are not interchangeable with the already admitted process scope. Repeated polling of ready state would need a defined wake contract and must not replace original serving custody with guessed completion.
The concrete design must distinguish original scope authority from invocation custody, preserve journal publication semantics and select bounded fair scheduling from counted storage. Keep implementation decisions grounded in those requirements; do not add a fixed per-process parallelism limit merely to simplify admission.
Tracking and rollout
Supports local stream sockets and safe concurrent calls and Phase 6 completion. Related reviews: Vfsd, Posixd, Docs, Integration.
Implement native scope serving custody and the mediator policy coordinator before enabling same-process overlapping intake. Preserve the completed narrow candidate-retirement repair. Remove superseded single-call assumptions only after replacement ownership and cleanup paths are concrete. Mark source completion based on implementation under the current execution waiver; do not claim runtime, CI or measured performance evidence that has not been obtained.
Detailed scope and contracts
Original process-close continuation contract — 4 October 2026
Before overlapping mediator intake, replace blind uncertain process release with one original keyed native cleanup intent. Reuse the existing fixed-width candidate-lock request and existing scope-local monotonic continuation domain. Nonzero-key ReleaseProcess prepares whole-object process cleanup with kind absent and report PID zero; Ready leaves published intervals unchanged. Exact Commit publishes once. Retain a counted, grant-free committed receipt through caller loss and repeated exact requests; Cancel with that cleanup shape acknowledges the receipt without releasing locks. Ordinary acquisition Cancel retains its rollback meaning. Original scope/candidate/key and full request echo must correlate every stage.
Pending cleanup retains the original noncopy provider-pin use and actual object/process-owner dependency. It fences only related mutations, not queries, reads, independent objects or description owners. Close cancels older acquisition waits before queueing its original cleanup; it must not wait behind an external lock holder. Refused readiness or Commit retains the original intent for exact continuation; it cannot create a fresh release. Pin disposal preserves historical identity and cannot rebind cleanup to a newer object. Whole-scope teardown disposes its own retained histories. Admission failure and fenced absence must have explicit effect-free outcomes, distinct from semantic completion. Posixd issues the key in its first actual submission turn, retains original caller/descriptor-change custody through ambiguity and application cancellation, and retires the final candidate only after positive semantic confirmation and receipt acknowledgment.
Implement producer, consumer, actual storage/layout, independent failure controls and documentation together, with coherent signed immutable adoption. Do not add an object-identity metadata RPC, per-process activity exclusion, copied grant, payload thread authority, allocator or arbitrary capacity. Same-process mediator overlap, per-call native origin/masks/TLS, complete socket consumers and native workloads remain required. Tests and CI observation remain deferred; this section records the implementation contract, not completion or execution evidence.
Earlier keyed process-close source checkpoint — 4 October 2026
Fixed request/response widths and native authority ceilings remain intact; older peers reject the extended cleanup shapes, so producers and consumers adopt coherently.
Refused readiness or Commit rolls back the original journal and moves the same noncopy provider-pin use out of its retired native call, preserving the original key/object/process owner/ticket. Actual storage refusal precedes effects, pin retention and key burn.
Repeated exact cleanup or Commit cannot release newer locks. No copied grant, object-identity metadata RPC, payload thread authority, allocator or arbitrary coordinator cap is introduced.
They cover refused Ready/Commit and rollback, narrow owner dependencies, old Wait cancellation, newer locks preserved across historical retries, premature/mismatched acknowledgment, original pin/scope history disposal, actual counted admission refusal, malformed Commit lookup, malformed acknowledgment and application abandonment.
Same-process mediator overlap, authenticated native per-call origin/source holds, per-call masks/TLS, full socket/native workload and libc/imported consumer closure remain unfinished. Native caller retirement cannot prove semantic cleanup. The whole socket requirement remains open; both complete EriX guest generations and external Rust/LLVM guest rebuilding remain mandatory and unperformed.
Completed implementation source is marked under the waiver; missing implementation remains open.
Native-serving acceptance remains five of eight: its coordinator criterion also requires real Close intake behind blocking Wait, which remains unfinished; fd/process coordination and complete same-process VM workload sources also remain open. No full-feature acceptance is inferred from authored controls or waived execution.
Counted mediator intake and original-call progress
Reserve an actual invocation row, its nonwrapping generation and its original receiver/client association before every native CLAIM. Retain a returned owner before interpreting status. Empty native intake releases only the uncommitted reservation; actual capacity refusal performs no CLAIM. The shared counted pool admits independent calls from the same authenticated native TCB/generation without duplicating receiver grants or process policy.
An intrusive original-call roster gives every retained owner a fair progress turn, independently of receiver intake. Every association and roster link includes the original row generation. Request/result regions and wake stamps belong to individual calls; receiver parking cannot consume their wake observations. Receiver/watch, descriptor, provider-pin and scope disposal wait for every original call, including refused completion and source cleanup. Result inspection names the original owner explicitly rather than guessing one call for a process.
Each readiness call retains its own noncopy temporary-mask observation. Installing or discarding it must not overwrite another call's mask or a later permanent mask change. A selected caught occurrence is consumed once and remains reserved for SignalTake; it cannot interrupt another call or be overwritten by another selection. Cancellation and uncertain handoff preserve the original native owners and terminal policy. These changes do not authenticate payload thread identifiers. Native shared-process thread membership, distinct per-thread masks/TLS and complete consumer adoption remain required; asynchronous calls from the current one-TCB execution identity do not satisfy that boundary.
Author independent native-owner controls for overlapping waits/intake, capacity refusal, origin mismatch, original result isolation, cancellation, receiver lifetime and signal selection. Testing and CI execution remain deferred under the active instruction; source implementation does not claim runtime validation.
Original mediator calls and source acceptance — 4 October 2026
Terminal retirement drains every call, including refused completion and source cleanup, before descriptor, provider, scope, watch or receiver disposal.
Their original target occupation and sole prospective duplicate source survive fd reuse; stale targets refuse without changing a new binding. Original receipt retries cannot release newer locks.
Cancellation cannot overwrite another call or a later permanent mask. Signal selection consumes one actual occurrence and reserves it exclusively for Take; another call cannot reuse it.
These sources introduce no additional endpoint, grant or hidden fd.
Authentic native shared-process thread membership, shared process actions with distinct per-thread masks/TLS, libc/imported consumer closure and profiling measurements remain open. Payload thread identifiers and duplicated client rows cannot satisfy that boundary. Full EriX guest generations and external Rust/LLVM rebuilding inside EriX remain required and unperformed.
Source acceptance follows the active waiver; missing implementation remains open.
Native-serving source acceptance is complete with all eight design criteria. Duplex semantics, scoped named lifecycle, independent controls/native workload sources and maintained socket documentation are source-accepted in the broader socket design; its complete ABI/consumer matrix, complete signal/runtime integration and whole source/performance closure remain open. The phase socket checklist has fourteen children, seven checked, while its parent remains open.
Related records and contracts
erikinkinen referenced this issue from erix/integration2026-10-03 01:43:05 +02:00
erikinkinen referenced this issue from erix/coreutils2026-10-04 11:37:32 +02:00
erikinkinen referenced this issue from erix/extop2026-10-04 11:37:40 +02:00