[FEATURE] Audit authority, code quality and documentation in coreutils #1

Open
opened 2026-09-12 07:57:26 +02:00 by erikinkinen · 0 comments
Owner

Problem and motivation

Audit coreutils throughout Phase 6. Implement native utility applets and their per-operation authority policy over explicit launch and stream grants.

Review each changed boundary at the final published revision. Prioritize security, reliability, then performance.

Proposed behavior and scope

Current inventory: 2 Cargo targets (1 bin, 1 lib); manifests: Cargo.toml. Include explicit and automatically discovered targets, supported features and target-specific configurations.

Inventory every startup/runtime grant and authenticated peer, then success, denial, cancellation, failed transfer, restart and teardown. Trace callers and downstream providers so names, numeric identities and retained aliases never become implicit authority.

Authority, security and reliability

Record each finding's affected revision, invariant, reproduction, owner, regression coverage and resolution. Report suspected vulnerabilities through SECURITY.md and keep sensitive details private. Record a result and rationale for each audit criterion, including caller responsibilities when this component has no process or Rust target.

Acceptance criteria

Checklist
  • Record origin, recipient, object, operations, delegation ceiling, lifetime and aliases for each relevant capability or caller-supplied authority-bearing value.
  • Audit ambient discovery/selectors, cwd/PATH/environment, numeric identities, inherited routes and host fallbacks; require explicit authorized intake.
  • Account for residual authority after success, error, cancellation, timeout, restart, failed transfer and teardown, including fork/exec where implemented.
  • Remove unnecessary endpoints, broad rights, duplicate aliases, provider/admin grants and debug routes.
  • Exercise stale generations, replay, pending replies, forged descriptive identity and object/path resolution races at the owned boundary.
  • Prove independent cleanup attempts and caller-specific error precedence; quarantine or terminate when retained authority cannot be accounted for.
  • Review unsafe/FFI/parser/arithmetic/lifetime/lock/publication invariants and add adversarial coverage where practical.
  • Inventory obsolete APIs, wrappers, fallback/dead paths and duplicate validators with their maintained callers.
  • Migrate callers and delete deprecated contracts/shims in a coherent signed revision graph; reject retired input versions where relevant.
  • Classify each size/count/depth/time bound as ABI, hardware, explicit resource policy or accidental limitation.
  • Test beyond removed boundaries and at allocation/ABI limits while preserving exhaustion errors, denial-of-service controls and bounded waits.
  • Keep every tracked authored code/test/script/workflow file below 1000 physical lines through thematic refactoring.
  • Run the deterministic tracked-source size gate, covering executable fixtures/generators and excluding only genuine non-code data or external payloads.
  • Audit production/test feature and symbol separation, including this repository's effect on rootd test orchestration.
  • Supply changes affecting rootd semantic surface to the maintained same-toolchain baseline; record a justified component-specific applicability result.
  • Check bootstrap ownership and development/release authority parity at this repository's producer/consumer boundary.
  • Use maintained integration#3 profiler evidence before optimizing; preserve live access checks, ownership and success criteria.
  • Audit build/CI inputs, secret handling, private outputs, symlink containment, deletion, subprocess bounds, warnings and host dependencies.
  • meaningful public/private inline documentation, crate/target missing_docs enforcement without blanket allowances or hidden-API escapes, warning-denied private-item rustdoc and an undocumented-public-API negative gate.
  • Validation: Run cargo fmt --all -- --check, strict Clippy, unit/doctests and warning-denied builds/rustdoc for every owned crate/target under default and all valid feature, freestanding/host, SMP and profile combinations; test mutually exclusive combinations separately. Add focused VM regressions for runtime behavior and observe the older catalog and unit tests at the exact published heads.
  • Documentation: update applicable README/ARCHITECTURE/ROADMAP in meta's canonical format and affected technical-manual TeX/API references; keep README evergreen and shared governance byte-identical to meta.
  • Evidence: record exact source/dependency revisions, commands, configurations, real exit status, CI run URLs and results; repeat the audit on final heads and obtain independent review of security closures. No skipped/pending/predecessor result counts as a pass.

Alternatives and tradeoffs

Refactor within the component unless shared semantics justify extracting common code. Preserve regression coverage before deleting superseded paths. Profile before optimizing, and document ABI limits, hardware limits and explicit quotas. Preserve security and reliability requirements.

Matching catalog adoption and real consumer VMs remain requirements; complete realm and full in-guest build acceptance remain open.

Original coherent realm source CI acceptance — 18 September 2026: Signed 9a6f84afa0a450c157bc28293e367f9e00132c3f passes CI 55 and CI 54. Coherent catalog publication, actual consumer VMs and full guest-build acceptance remain separate open requirements.

Tracking and rollout

All 39 maintained local checks pass: development/release feature matrices (145, 146 unit tests per selection), strict host/native Clippy, warning-denied builds, private rustdoc, applicable doctests and component-specific checks, formatting and exact dependency-source validation. Native binaries have executable read-only entry pages and no writable executable pages. Original CI62 and CI63 pass; their complete logs are independently reviewed with zero warnings. No unchanged CI was rerun. Coordinated VM configuration/seal/four-client I/O, repeated managed-row reuse, profiler observation and both full builds inside EriX remain open under the phase checklist. This dependency work earns no separate feature-completion credit.

Counted private startup dependency adoption — 24 September 2026.

These host/build results do not establish actual running mediator readiness or retirement, a complete service-image pass, configuration/seal, client I/O, or either required guest rebuild.

Tracking: owning issue, WIP PR and Phase 6 acceptance.

Detailed scope and contracts

Command and boot dependency alignment — 15 September 2026

The product catalog, product VM acceptance and guest build remain pending.

Full catalog equality and actual consumer-image validation remain required under Integration regression 68. This dependency checkpoint does not close full lifecycle, frame or self-hosting acceptance.

Full service CPU/profiler VM acceptance and guest builds remain open in Phase 6 completion.

Counted endpoint-selected exact mutation

Replace the fixed 252-byte public VfsExactMutationRequestV1 with borrowed
VfsExactMutationRequestV2. Its 24-byte prefix contains request ID and the existing
exact operation at offsets 0/4, explicit version 2 at offset 8, reserved zero at
12 and the complete u64 destination-byte length at 16. Rename alone supplies one
raw basename after offset 24; other admitted effects have an empty tail. Empty,
NUL, slash and dot rename components refuse. Checked full extent and actual
caller/transport storage replace the fixed width and UTF-8 restriction. The
version/reserved fields distinguish valid old requests, including two-byte old
rename names, without fallback or retaining the deprecated public API.

The exact endpoint still supplies the original immutable operation and entry;
no source, parent, namespace, handle selector, new endpoint or capability is
introduced. Vfsd admits complete status capacity before provider effects and
passes the borrowed destination through the original retained owner. Existing
one-use/definite-refusal/uncertain-effect consumption and original cleanup rules
remain authoritative. Neither larger names nor descriptive status grant rights.

Migrate LibIPC, Vfsd dispatch and Coreutils source callers and controls in one
signed source graph. Coreutils uses actual encoded length and a single call;
wrong correlation, unknown status, malformed/lost replies or unexpected authority
never trigger effect replay. Retire all unwanted receipts before refusal and
preserve uncertain cleanup classification. Share the exact status-call framing,
receipt retirement and scratch clearing with existing truncation instead of
adding another transport copy. Preserve the caller's original endpoint ownership.

Author literal V2 byte layout, complete raw names, short/surplus tails, version
and reserved separation, bad shape, output canaries, operation ceilings, short
status preflight, original-provider binding/cleanup and real transport controls.
Update component contracts and maintained manual TeX. Tests/builds/checks,
generated documents, VM/profiler execution and CI observation remain deferred.
Both direct compiled IPC selectors in adopting consumers use the new immutable
revision; runtime catalogs and compiled Rootd helpers remain unchanged pending
coherent runtime adoption. Legacy directory/consumer frames and complete
cross-provider/self-hosting acceptance remain open.

Tracking: codec audit/design,
native ownership,
Coreutils audit,
Phase 6 tracker.
Review: LibIPC PR2,
Vfsd PR3,
Coreutils PR2,
Docs PR4.

The original ENUMERATE
endpoint owns cursor tokens bound to its scope generation; names and tokens
grant no native authority. Old exact generic frames refuse without fallback.

Refusals release the reservation; equal native positions
reuse their token. Different owners and replacement generations cannot inherit
old tokens. Caller names stay unchanged on refusal.

Remaining compatibility/legacy namespace lifetime, complete
cross-provider refusal/restart controls and coherent runtime adoption stay open. Full EriX guest
generations and external Rust/LLVM rebuilding inside EriX remain required and
unperformed; no executed pass or measured speedup is claimed.

Counted endpoint-selected directory enumeration

Replace the exact-endpoint use of fixed public readdir frames with dedicated
VfsExactReadDirectoryRequestV2 and VfsExactReadDirectoryResponseV2. The 32-byte
request has correlation/op at 0/4, version 2/reserved zero at 8/12, the original
endpoint-local cursor at 16 and u64 name capacity at 24. There is no public
handle, provider, candidate or namespace selector. The original ENUMERATE
endpoint remains the sole authority. Old 24-byte requests refuse on this route;
generic namespace/provider V1 contracts remain until their live users migrate.

The response has a 56-byte prefix: correlation/op at 0/4, version 2 at 8,
known result at 12, next token at 16, node kind/end flag at 24/28, observed size
at 32, complete u64 name length at 40 and reserved zero at 48. One complete raw
name follows. End and refusal have no entry, zero metadata and no name bytes;
refusal preserves the request token. A successful entry has a known kind and
a distinct nonzero token. Check the complete shape, original correlation and
offered capacity before publishing any caller-visible name or cursor.

Vfsd retains counted full native cursor state and reserves a position before
observation. Preserve native mount composition, original metadata identity and
known-size checks, original candidate retirement and equal-position reuse.
Admit complete response/name storage before publishing the reserved position;
short capacity, malformed input, provider refusal and stale identity release
the reservation without advancing the original token. Complete raw names replace
the fixed 240-byte copy. Preserve original owner retirement and generation
isolation. No path reopening, hidden fallback or name-derived authority.

Coreutils advertises the minimum of actual caller name storage and actual IPC
reply capacity, submits one call to the original endpoint, retires unexpected
receipts, validates all framing/correlation and only then copies complete names.
Erase message scratch on every return; failed admission leaves caller names
unchanged. Keep lease ownership and existing utility cursor behavior unchanged.
Use shared explicit call/disposal effects and independent reply fixtures.

Author literal codec, actual provider/position/metadata, transport cleanup and
long raw utility-listing controls. Update inline/component/manual contracts and
publish signed source with both IPC selectors in Vfsd/Coreutils coordinated.
Runtime catalogs and compiled Rootd helpers remain unchanged. Tests/builds/checks,
generated references, VM/profiler execution and CI observation remain deferred.
Complete cross-provider lifetime/restart and guest self-hosting remain open.

Tracking: codec design,
native ownership,
Coreutils audit,
Phase 6 tracker.

Related records and contracts
Repository Links
docs issue 1
integration issue 2
kernel issue 20
## Problem and motivation Audit `coreutils` throughout Phase 6. Implement native utility applets and their per-operation authority policy over explicit launch and stream grants. Review each changed boundary at the final published revision. Prioritize security, reliability, then performance. ## Proposed behavior and scope Current inventory: 2 Cargo targets (1 bin, 1 lib); manifests: `Cargo.toml`. Include explicit and automatically discovered targets, supported features and target-specific configurations. Inventory every startup/runtime grant and authenticated peer, then success, denial, cancellation, failed transfer, restart and teardown. Trace callers and downstream providers so names, numeric identities and retained aliases never become implicit authority. ## Authority, security and reliability Record each finding's affected revision, invariant, reproduction, owner, regression coverage and resolution. Report suspected vulnerabilities through SECURITY.md and keep sensitive details private. Record a result and rationale for each audit criterion, including caller responsibilities when this component has no process or Rust target. ## Acceptance criteria <details> <summary>Checklist</summary> - [ ] Record origin, recipient, object, operations, delegation ceiling, lifetime and aliases for each relevant capability or caller-supplied authority-bearing value. - [ ] Audit ambient discovery/selectors, cwd/PATH/environment, numeric identities, inherited routes and host fallbacks; require explicit authorized intake. - [ ] Account for residual authority after success, error, cancellation, timeout, restart, failed transfer and teardown, including fork/exec where implemented. - [ ] Remove unnecessary endpoints, broad rights, duplicate aliases, provider/admin grants and debug routes. - [ ] Exercise stale generations, replay, pending replies, forged descriptive identity and object/path resolution races at the owned boundary. - [ ] Prove independent cleanup attempts and caller-specific error precedence; quarantine or terminate when retained authority cannot be accounted for. - [ ] Review unsafe/FFI/parser/arithmetic/lifetime/lock/publication invariants and add adversarial coverage where practical. - [ ] Inventory obsolete APIs, wrappers, fallback/dead paths and duplicate validators with their maintained callers. - [ ] Migrate callers and delete deprecated contracts/shims in a coherent signed revision graph; reject retired input versions where relevant. - [ ] Classify each size/count/depth/time bound as ABI, hardware, explicit resource policy or accidental limitation. - [ ] Test beyond removed boundaries and at allocation/ABI limits while preserving exhaustion errors, denial-of-service controls and bounded waits. - [ ] Keep every tracked authored code/test/script/workflow file below 1000 physical lines through thematic refactoring. - [ ] Run the deterministic tracked-source size gate, covering executable fixtures/generators and excluding only genuine non-code data or external payloads. - [ ] Audit production/test feature and symbol separation, including this repository's effect on rootd test orchestration. - [ ] Supply changes affecting rootd semantic surface to the maintained same-toolchain baseline; record a justified component-specific applicability result. - [ ] Check bootstrap ownership and development/release authority parity at this repository's producer/consumer boundary. - [ ] Use maintained [integration#3](https://git.erikinkinen.fi/erix/integration/issues/3) profiler evidence before optimizing; preserve live access checks, ownership and success criteria. - [ ] Audit build/CI inputs, secret handling, private outputs, symlink containment, deletion, subprocess bounds, warnings and host dependencies. - [ ] meaningful public/private inline documentation, crate/target `missing_docs` enforcement without blanket allowances or hidden-API escapes, warning-denied private-item rustdoc and an undocumented-public-API negative gate. - [ ] Validation: Run `cargo fmt --all -- --check`, strict Clippy, unit/doctests and warning-denied builds/rustdoc for every owned crate/target under default and all valid feature, freestanding/host, SMP and profile combinations; test mutually exclusive combinations separately. Add focused VM regressions for runtime behavior and observe the older catalog and unit tests at the exact published heads. - [ ] Documentation: update applicable README/ARCHITECTURE/ROADMAP in meta's canonical format and affected technical-manual TeX/API references; keep README evergreen and shared governance byte-identical to meta. - [ ] Evidence: record exact source/dependency revisions, commands, configurations, real exit status, CI run URLs and results; repeat the audit on final heads and obtain independent review of security closures. No skipped/pending/predecessor result counts as a pass. </details> ## Alternatives and tradeoffs Refactor within the component unless shared semantics justify extracting common code. Preserve regression coverage before deleting superseded paths. Profile before optimizing, and document ABI limits, hardware limits and explicit quotas. Preserve security and reliability requirements. Matching catalog adoption and real consumer VMs remain requirements; complete realm and full in-guest build acceptance remain open. Original coherent realm source CI acceptance — 18 September 2026: Signed `9a6f84afa0a450c157bc28293e367f9e00132c3f` passes [CI 55](https://git.erikinkinen.fi/erix/coreutils/actions/runs/55) and [CI 54](https://git.erikinkinen.fi/erix/coreutils/actions/runs/54). Coherent catalog publication, actual consumer VMs and full guest-build acceptance remain separate open requirements. ## Tracking and rollout All 39 maintained local checks pass: development/release feature matrices (145, 146 unit tests per selection), strict host/native Clippy, warning-denied builds, private rustdoc, applicable doctests and component-specific checks, formatting and exact dependency-source validation. Native binaries have executable read-only entry pages and no writable executable pages. Original [CI62](https://git.erikinkinen.fi/erix/coreutils/actions/runs/62) and [CI63](https://git.erikinkinen.fi/erix/coreutils/actions/runs/63) pass; their complete logs are independently reviewed with zero warnings. No unchanged CI was rerun. Coordinated VM configuration/seal/four-client I/O, repeated managed-row reuse, profiler observation and both full builds inside EriX remain open under the [phase checklist](https://git.erikinkinen.fi/erix/integration/issues/65). This dependency work earns no separate feature-completion credit. **Counted private startup dependency adoption — 24 September 2026.** These host/build results do not establish actual running mediator readiness or retirement, a complete service-image pass, configuration/seal, client I/O, or either required guest rebuild. Tracking: [owning issue](https://git.erikinkinen.fi/erix/coreutils/issues/1), [WIP PR](https://git.erikinkinen.fi/erix/coreutils/pulls/2) and [Phase 6 acceptance](https://git.erikinkinen.fi/erix/integration/issues/65). <details> <summary>Detailed scope and contracts</summary> ### Command and boot dependency alignment — 15 September 2026 The product catalog, product VM acceptance and guest build remain pending. Full catalog equality and actual consumer-image validation remain required under [Integration regression 68](https://git.erikinkinen.fi/erix/integration/issues/68). This dependency checkpoint does not close full lifecycle, frame or self-hosting acceptance. Full service CPU/profiler VM acceptance and guest builds remain open in [Phase 6 completion](https://git.erikinkinen.fi/erix/integration/issues/65). ### Counted endpoint-selected exact mutation Replace the fixed 252-byte public VfsExactMutationRequestV1 with borrowed VfsExactMutationRequestV2. Its 24-byte prefix contains request ID and the existing exact operation at offsets 0/4, explicit version 2 at offset 8, reserved zero at 12 and the complete u64 destination-byte length at 16. Rename alone supplies one raw basename after offset 24; other admitted effects have an empty tail. Empty, NUL, slash and dot rename components refuse. Checked full extent and actual caller/transport storage replace the fixed width and UTF-8 restriction. The version/reserved fields distinguish valid old requests, including two-byte old rename names, without fallback or retaining the deprecated public API. The exact endpoint still supplies the original immutable operation and entry; no source, parent, namespace, handle selector, new endpoint or capability is introduced. Vfsd admits complete status capacity before provider effects and passes the borrowed destination through the original retained owner. Existing one-use/definite-refusal/uncertain-effect consumption and original cleanup rules remain authoritative. Neither larger names nor descriptive status grant rights. Migrate LibIPC, Vfsd dispatch and Coreutils source callers and controls in one signed source graph. Coreutils uses actual encoded length and a single call; wrong correlation, unknown status, malformed/lost replies or unexpected authority never trigger effect replay. Retire all unwanted receipts before refusal and preserve uncertain cleanup classification. Share the exact status-call framing, receipt retirement and scratch clearing with existing truncation instead of adding another transport copy. Preserve the caller's original endpoint ownership. Author literal V2 byte layout, complete raw names, short/surplus tails, version and reserved separation, bad shape, output canaries, operation ceilings, short status preflight, original-provider binding/cleanup and real transport controls. Update component contracts and maintained manual TeX. Tests/builds/checks, generated documents, VM/profiler execution and CI observation remain deferred. Both direct compiled IPC selectors in adopting consumers use the new immutable revision; runtime catalogs and compiled Rootd helpers remain unchanged pending coherent runtime adoption. Legacy directory/consumer frames and complete cross-provider/self-hosting acceptance remain open. Tracking: [codec audit/design](https://git.erikinkinen.fi/erix/lib-ipc/issues/1), [native ownership](https://git.erikinkinen.fi/erix/vfsd/issues/16), [Coreutils audit](https://git.erikinkinen.fi/erix/coreutils/issues/1), [Phase 6 tracker](https://git.erikinkinen.fi/erix/integration/issues/65). Review: [LibIPC PR2](https://git.erikinkinen.fi/erix/lib-ipc/pulls/2), [Vfsd PR3](https://git.erikinkinen.fi/erix/vfsd/pulls/3), [Coreutils PR2](https://git.erikinkinen.fi/erix/coreutils/pulls/2), [Docs PR4](https://git.erikinkinen.fi/erix/docs/pulls/4). The original ENUMERATE endpoint owns cursor tokens bound to its scope generation; names and tokens grant no native authority. Old exact generic frames refuse without fallback. Refusals release the reservation; equal native positions reuse their token. Different owners and replacement generations cannot inherit old tokens. Caller names stay unchanged on refusal. Remaining compatibility/legacy namespace lifetime, complete cross-provider refusal/restart controls and coherent runtime adoption stay open. Full EriX guest generations and external Rust/LLVM rebuilding inside EriX remain required and unperformed; no executed pass or measured speedup is claimed. ### Counted endpoint-selected directory enumeration Replace the exact-endpoint use of fixed public readdir frames with dedicated VfsExactReadDirectoryRequestV2 and VfsExactReadDirectoryResponseV2. The 32-byte request has correlation/op at 0/4, version 2/reserved zero at 8/12, the original endpoint-local cursor at 16 and u64 name capacity at 24. There is no public handle, provider, candidate or namespace selector. The original ENUMERATE endpoint remains the sole authority. Old 24-byte requests refuse on this route; generic namespace/provider V1 contracts remain until their live users migrate. The response has a 56-byte prefix: correlation/op at 0/4, version 2 at 8, known result at 12, next token at 16, node kind/end flag at 24/28, observed size at 32, complete u64 name length at 40 and reserved zero at 48. One complete raw name follows. End and refusal have no entry, zero metadata and no name bytes; refusal preserves the request token. A successful entry has a known kind and a distinct nonzero token. Check the complete shape, original correlation and offered capacity before publishing any caller-visible name or cursor. Vfsd retains counted full native cursor state and reserves a position before observation. Preserve native mount composition, original metadata identity and known-size checks, original candidate retirement and equal-position reuse. Admit complete response/name storage before publishing the reserved position; short capacity, malformed input, provider refusal and stale identity release the reservation without advancing the original token. Complete raw names replace the fixed 240-byte copy. Preserve original owner retirement and generation isolation. No path reopening, hidden fallback or name-derived authority. Coreutils advertises the minimum of actual caller name storage and actual IPC reply capacity, submits one call to the original endpoint, retires unexpected receipts, validates all framing/correlation and only then copies complete names. Erase message scratch on every return; failed admission leaves caller names unchanged. Keep lease ownership and existing utility cursor behavior unchanged. Use shared explicit call/disposal effects and independent reply fixtures. Author literal codec, actual provider/position/metadata, transport cleanup and long raw utility-listing controls. Update inline/component/manual contracts and publish signed source with both IPC selectors in Vfsd/Coreutils coordinated. Runtime catalogs and compiled Rootd helpers remain unchanged. Tests/builds/checks, generated references, VM/profiler execution and CI observation remain deferred. Complete cross-provider lifetime/restart and guest self-hosting remain open. Tracking: [codec design](https://git.erikinkinen.fi/erix/lib-ipc/issues/1), [native ownership](https://git.erikinkinen.fi/erix/vfsd/issues/16), [Coreutils audit](https://git.erikinkinen.fi/erix/coreutils/issues/1), [Phase 6 tracker](https://git.erikinkinen.fi/erix/integration/issues/65). </details> <details> <summary>Related records and contracts</summary> | Repository | Links | | --- | --- | | docs | [issue 1](https://git.erikinkinen.fi/erix/docs/issues/1) | | integration | [issue 2](https://git.erikinkinen.fi/erix/integration/issues/2) | | kernel | [issue 20](https://git.erikinkinen.fi/erix/kernel/issues/20) | </details>
erikinkinen changed title from [FEATURE] [P02.R04] Audit authority, code quality and documentation in coreutils to [FEATURE] Audit authority, code quality and documentation in coreutils 2026-09-12 08:02:11 +02:00
erikinkinen added this to the Alpha 1 milestone 2026-10-30 09:33:35 +01:00
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
erix/coreutils#1
No description provided.