[FEATURE] Retain final process accounting until acknowledged observation #20
Labels
No labels
bug
ci
docs
duplicate
enhancement
help wanted
invalid
performance
phase-6
question
refactor
security
wontfix
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
erix/kernel#20
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Problem and motivation
Automatic child reclamation can remove the original TCB before Procd queries terminal CPU counters. Destructive event waiting also removes the event before that separate query. Self-hosted job accounting needs final counters bound to the original generation without preserving dead process resources.
Proposed behavior and scope
Native terminal resource disposal and delivery of scalar observations are distinct duties. Preflight reserves queue storage for the event plus its final CPU result before terminal effects. The final result contains the original user/system nanoseconds or an explicit measurement error, never fabricated successful zero. Nonexecuting targets freeze immediately; an outgoing active interval freezes only after its final native charge and before TCB/CSpace/VSpace disposal. Completed event storage owns no capability, mapping, TCB or saved frame. Generation reuse cannot replace this scalar result.
Replace destructive operation 55 with operation 59, OBSERVE_PROCESS_TERMINAL_EVENT. All flags/arguments are reserved zero; the response uses the existing ProcessTerminalEventV1 process/kind and status/generation packing. The next available ready observation is claimed by the actual current native observer identity, including its creation generation. Repeated observation by that same observer returns identical data without dequeue. Another authorized observer cannot steal that claim but can claim an independent queued event. There is no single global outstanding-observation restriction. An unready or unavailable next observation returns RETRY with zero values. Claiming allocates nothing. Observer terminality releases only its own original claim before disposal so another caller holding actual Process authority can observe that original event and counters; numeric identities never authorize operations.
Add operation 60, ACKNOWLEDGE_PROCESS_TERMINAL_EVENT: zero flags, arg0 exact nonzero u32 process ID, arg1 exact nonzero u32 generation, arg2/arg3 zero. Both response values are zero. Only the current original observer may consume its matching ready observation. Wrong identities never consume another event. NOT_FOUND means that original observation is absent, including an already acknowledged retry; acknowledging another observer's claim is denied. Lost observation/query replies preserve the original event; lost acknowledgement replies cannot consume a successor on retry. Consumer death releases its claim without discarding queued evidence or retaining resources from the dead consumer. Queue capacity remains subject to explicit allocator exhaustion and full preflight rollback; there is no new numerical queue cap or expiry that silently discards observations.
Existing operation 47 resolves a retained final snapshot for the exact process/generation before considering a live record, and never resolves an ID's successor. Pending terminal counters return RETRY rather than provisional successful data. Calibration failure is retained as an error independently of successful process cleanup. Once acknowledged, terminal metadata can be released; native resources never wait for observation or acknowledgement. Finalization and acknowledgement require no allocation. Finalization uses the queued original generation, and acknowledgements free scalar entries without changing another process's object state.
Procd acknowledges only after recording the exact terminal state/counters and retaining all required native/provider cleanup duties. Repeated observation after partial handling must not duplicate visible events, reports or effects. Ordinary launched children and mediator metric consumers require explicit terminal counter ownership; releasing a mediator's capability custody must not falsely report missing metrics as zero. A dead metrics consumer must not pin capabilities or native backing. Unknown pre-service events may be deliberately acknowledged after exact cleanup custody is retained. No acknowledgement is sent after a failed metric read or incomplete local transition. Rootd routing tables and all tests/diagnostics migrate to the new selectors; operation 55 and destructive wait helpers are removed, with negative compatibility controls and no fallback.
Authority, security and reliability
The existing held Process/Root control endpoint remains required. Scalar IDs, observations and counters grant no capability. The actual original native caller owns an outstanding observation; caller loss releases its claim without reassigning authority through a reused ID. Snapshot custody is separate from native resources and from provider cleanup. No zero-counter fallback, wall-clock substitute, silent queue expiry, unchanged retry-to-green or relaxed deadline is allowed. No restricted call-retirement reproducer is part of this work.
Acceptance criteria
Alternatives and tradeoffs
Keeping terminal TCBs and mappings alive would prevent required reclamation. Returning zero or consulting a reused ID would corrupt accounting. A destructive wait followed by a separate query cannot provide reliable delivery. The retained scalar queue separates resource disposal from acknowledged observation; actual profiling must assess its lookup/finalization cost without weakening correctness.
Tracking and rollout
Coordinate Kernel lifetime design, Kernel WIP PR 3, Procd WIP PR 2, Integration WIP PR 12 and Phase 6 completion. Publish the wire producer and actual consumers in one coherent original source graph, then validate and retire the old selector without a compatibility fallback. This design does not close a phase acceptance leaf.
Acknowledged terminal accounting — 21 September 2026: Kernel design 20 specifies final scalar CPU evidence retained independently of native resource reclamation, repeatable observations bound to actual original observer generations, independent authorized observers, exact acknowledgement and claim release on observer death. Both ordinary and mediator consumers must commit terminal measurements and cleanup duties before acknowledgement, including consumer-loss handling.
Shared wire commit 0b889095fd04f6f4b1da1d4c0064150551f52122 is signed and pushed. Four wire configurations pass 427 tests each; four syscall-shim configurations pass 20 each with one existing ignore. Eight native builds, strict host/native Clippy, formatting, private rustdoc, Markdown and dependency checks pass without warnings. Native kernel and consumer adoption, actual CPL3/service VM acceptance and original CI observation remain open.
Canonical completion remains 3.48% weighted; 15 of 460 acceptance items. Full upstream Rust/LLVM/runtime rebuilding inside EriX remains mandatory together with both full EriX guest build generations. No acceptance item closes from this representation checkpoint.
Verified native terminal accounting — 21 September 2026: Kernel a9bdf6163813d378e0b4a164bceb839e24fbb6b7 is signed/pushed. Terminal preflight reserves final scalar CPU evidence independently of TCB/CSpace/VSpace reclamation; exact queries preserve final results or explicit errors. Repeat observations belong to the actual original observer until exact acknowledgement, and observer death releases that claim. Independent authorized observers can progress. Selector 55 is retired; checked selectors 59/60 have no destructive fallback.
Four strict 736/760-unit configurations, both standalone controls and thirteen native build/Clippy profiles pass without warnings. Host controls include nonzero final counters after actual reclamation and ID reuse, original observer death, pending-final-charge destruction refusal, malformed requests, wrong callers, immutable errors and lost-acknowledgement reply retry. Original Kernel CI 624/625 and corrected 626/627 all pass from eight complete hashed logs (1,532,848 bytes), zero warnings.
Both maintained lifetime and owned-invocation VMs pass under the unchanged 60-second scenario limits and standard watchdogs, with no build warnings and empty QEMU stderr. Actual guest instructions check repeat observations and CPU queries, exact acknowledgement and absent-acknowledgement retry; executing children require nonzero user and kernel counters after native reclamation. Lifetime retains 2,025 serial bytes (SHA256
921edf5eadfdff61f2d85a63158555666e77e57a1e8aa4254ac30dcd216f8cf9); owned invocation retains 1,587 (SHA256404bc4ecad5074349d9ba45d1caf5439aebe849d344b726e5dec2ee2b9c4d907). Normal stripping exactly matches both packaged kernels to retained original artifacts; all fifteen selected original signatures and clean checkouts verify.Kernel regression 21 retains the first VM's final page-census failure. The corrected layout declares and allocates all six request pages and derives the independent census from that declaration. No unchanged retry or deadline relaxation occurred.
Integration fcd7b4a9608f629a12de78c53da5c3615d906b46 is signed/pushed with the verified isolated catalog. Twenty dependency and 46 immutable-source tests, native policies, Markdown and source checks pass. The unchanged orchestration crate, embedded fixture and original dependency closure retain verified four 320/321-unit and native/Clippy/rustdoc configurations. The full service catalogs retain their separately coordinated revisions; original Integration CI remains under observation.
Procd and Rootd consumer adoption, ordinary and mediator metric retention/consumer loss, manual updates, full service/profiler scenarios, complete authority/source/frame audits and full regression acceptance remain open under Kernel design 20. Canonical acceptance remains 3.48% weighted; 15 of 460 items. Native upstream Rust/LLVM/runtime rebuilding and both full EriX guest build generations remain mandatory and unproven.
Committed terminal-accounting consumers — 21 September 2026:
Procd 66934642c4464fc738152a9e60790914ba27dd1c in WIP PR 2 reserves notification/crash/cleanup storage before effects, obtains exact final CPU evidence, commits local status and cleanup obligations, then acknowledges on every actual event polling path. Lost acknowledgement replies preserve the local result without duplicate counters or notifications. Mediators retain only scalar counters and the original authenticated supervisor identity after disposing all capability columns; supervisor death discharges the pending scalar observation and a replacement cannot inherit it. Four strict 299/305-test configurations, native builds, Clippy and private rustdoc pass. Original CI 298/299 passes from four complete hashed logs, 347,245 bytes, zero warnings. Bug 5 remains open for actual service acceptance.
Rootd 64c97b13c450003d9c2b6bd9ed2a627088684b46 in WIP PR 2 acknowledges bootstrap evidence only after exact native destruction and local endpoint absence; both operations remain unavailable after temporary Process custody transfers to Procd. Four strict 430/429-test configurations, native builds, Clippy and private rustdoc pass. Original 1039/1040 exposed bug 7: a stale source-call inventory and its matching semantic operation declarations. The correction explicitly inventories acknowledgement consumers and preserves the same temporary route and eventual Procd owner. All 64 Python controls, production-boundary, semantic baseline, threat model, phase contract and operation-ownership gates pass. Corrected original CI 1041/1042 passes from four complete verified logs, 222,969 bytes, zero warnings. Bug 7 is corrected; failed original runs remain retained without reruns or weaker gates.
Docs e4525848ad4462901c9a6794ef1794cf85ea9e6b updates the native contract, Procd/Rootd consumer custody and original signed IPC API references. Selector 55 is retired in both the detailed contract and summary; 58/59/60 are cross-checked against the shared registry. All 45 documentation tests and generated-reference checks pass. The complete 2,431-page manual builds without warnings; changed prose, selector and API pages pass visual review. Original documentation CI 995/996 and corrected-table 997/998 passes from eight complete logs, 1,549,628 bytes, with zero warnings in the final LaTeX passes. The 37 earlier convergence candidates per manual log are retained and resolved.
The separate Integration orchestration library checkpoint b06dfad00202765491a64552dde29eaca1c24838 passes four strict 320/321-test host/native configurations. Full service catalogs remain on their prior coherent graph while 35 remaining application/service repositories adopt the original shared revisions. Integration 1697/1698 remains running, and 1699/1700 plus 1701/1702 waits at the latest bounded observations. These are pending full regressions, not successful runtime acceptance.
No new canonical acceptance leaf is closed: 15/460 and 3.48% weighted. Ordinary Launchd metric-consumer restart/disposal semantics, coherent service CPU/profiler VMs, complete realm/provider authority and I/O, source/effect/frame proof, Pagerd, native external Rust/LLVM/runtime rebuilding and both full EriX-in-EriX build generations remain required. The static audit finds 3,162 authored code files below 1,000 lines, 74 manifests, 259 original Git pins, 173 direct missing-docs gates and 92 conventional crate roots; this does not establish semantic authority or complete private-documentation closure.
Actual service CPU scenarios and startup admission — 21 September 2026:
The published original Integration catalog
dff878dd3545c4751b3c05d37b2bdd5e21cce548passes the maintained shell times, two-CPU Extop and out-of-session denial VM scenarios under their unchanged 120-second guest limits. Each retains 106 hashed evidence files, warning-free image builds, empty QEMU stderr and every required, forbidden, ordered and unique marker check. Build-plus-scenario wall times are 119.876978, 34.802620 and 36.173027 seconds respectively; these are not guest-only or startup measurements.Times reports nonzero self and waited-child CPU. Extop observes both CPUs, memory and increasing job CPU nanoseconds with
CTRL no; both per-CPU percentages remain--.--%in the two samples, so numeric utilization is unproven. The denial scenario confirms the existing out-of-session boundary. Complete mediator/lifecycle acceptance in Procd 5 remains open.A separate ordinary development package also builds without warnings, but its required startup-contract preflight exits 1 before any VM or observer starts. Integration 69 records the missing full-runtime-transition contract selections and Kernel effective-feature/original-source evidence. Preserve the refused package and all admission controls; the 120/15/10 capture limits and performance thresholds are unchanged. No startup-profile acceptance is awarded. Complete source/effect/frame and Pagerd proof, native external Rust/LLVM/runtime rebuilding and both full EriX guest build generations remain required by Phase 6 completion.