[FEATURE] Audit authority, code quality and documentation in fatd #1
Labels
No labels
alpha-1
bug
ci
docs
duplicate
enhancement
help wanted
invalid
performance
phase-6
question
refactor
security
wontfix
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
erix/fatd#1
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Problem and motivation
Audit
fatdthroughout Phase 6. Serve FAT/exFAT filesystem operations using explicit backing-store grants and independently retired exact file/directory handles.Review each changed boundary at the final published revision. Prioritize security, reliability, then performance.
Proposed behavior and scope
Current inventory: 1 Cargo target (1 bin); manifests:
Cargo.toml. Include explicit and automatically discovered targets, supported features and target-specific configurations.Inventory every startup/runtime grant and authenticated peer, then success, denial, cancellation, failed transfer, restart and teardown. Trace callers and downstream providers so names, numeric identities and retained aliases never become implicit authority.
Authority, security and reliability
Record each finding's affected revision, invariant, reproduction, owner, regression coverage and resolution. Report suspected vulnerabilities through SECURITY.md and keep sensitive details private. Record a result and rationale for each audit criterion, including caller responsibilities when this component has no process or Rust target.
Acceptance criteria
Checklist
missing_docsenforcement without blanket allowances or hidden-API escapes, warning-denied private-item rustdoc and an undocumented-public-API negative gate.cargo fmt --all -- --check, strict Clippy, unit/doctests and warning-denied builds/rustdoc for every owned crate/target under default and all valid feature, freestanding/host, SMP and profile combinations; test mutually exclusive combinations separately. Add focused VM regressions for runtime behavior and observe the older catalog and unit tests at the exact published heads.Alternatives and tradeoffs
Refactor within the component unless shared semantics justify extracting common code. Preserve regression coverage before deleting superseded paths. Profile before optimizing, and document ABI limits, hardware limits and explicit quotas. Preserve security and reliability requirements.
Matching catalog adoption and real consumer VMs remain requirements; complete realm and full in-guest build acceptance remain open.
Tracking and rollout
All 34 maintained local checks pass: development/release feature matrices (175 unit tests per selection), strict host/native Clippy, warning-denied builds, private rustdoc, applicable doctests and component-specific checks, formatting and exact dependency-source validation. Native binaries have executable read-only entry pages and no writable executable pages. Original CI100 and CI101 pass; their complete logs are independently reviewed with zero warnings. No unchanged CI was rerun. Coordinated VM configuration/seal/four-client I/O, repeated managed-row reuse, profiler observation and both full builds inside EriX remain open under the phase checklist. This dependency work earns no separate feature-completion credit.
All 4 selected development/release configurations pass (175 unit tests per configuration). Strict host/native Clippy, warning-denied host/native builds, doctests where provided, private rustdoc, formatting, Markdown and dependency checks pass with zero warnings.
Matching-image native Posixd startup, two-realm independence and retirement acceptance remain pending. These local checks do not establish sealing, client I/O, a complete EriX guest build or the external toolchain rebuild. Publication CI is tracked separately from local validation.
VFS requires frozen Write for regular-file resize and live admitted file/directory custody for sync. Known refusals preserve original admission; uncertainty keeps cleanup-only custody, with no retry or pathname reopen.
No request or adapter adds a device, mount, credentials, new native rights or capability transfer.
They cover literal framing, unchanged output on refusal, full correlation, access direction, actual aliases and zero growth, storage/format refusal, clean recovery ownership, lost barriers/replies, retained cleanup and short reply capacity.
Existing ext logical-range conversions and sparse-gap traversal still need the size-boundary audit; compatibility truncate/ftruncate/fsync/fdatasync consumers, complete cancellation controls and new live workloads remain unfinished. Current phase completion remains 19.20% weighted, 7.61% whole acceptance: 27/460 parents and 486/949 children.
Both full guest EriX generations and external Rust/LLVM rebuilding remain required and unperformed.
Detailed scope and contracts
Cross-provider filesystem lifetime — signed source acceptance, 2 October 2026
Two live binding scenarios cover RAM, ext and separately selected FAT32/exFAT: conditional replacement/unlink, retained aliases and original identities, moved/detached directories, name reuse, supported links and read-only, cross-filesystem and unsupported-link refusals. Final media observers require reclaimed allocations, consistent reverse indices and clear undo ownership.
Both full EriX guest generations and external Rust/LLVM guest rebuilding remain required and unperformed.
Phase progress: 19.12% weighted, 7.61% whole acceptance; 27/460 parents and 485/949 sub-items.
Runtime consumer dependency alignment — 15 September 2026
The product catalog, product VM acceptance and guest build remain pending.
Full catalog equality and actual consumer-image validation remain required under Integration regression 68. This dependency checkpoint does not close full lifecycle, frame or self-hosting acceptance.
Full service CPU/profiler VM acceptance and guest builds remain open in Phase 6 completion.
Persistent FAT metadata recovery design — 30 September 2026
Complete original FAT metadata needs one durable update boundary joining original media changes and its retained identity/owner/mode/time records. The completed device barrier is a prerequisite, not a transaction implementation. Introduce a reusable, allocation-free original-device undo journal in LibBlock, followed by explicit FAT metadata-format and mount integration in the same full status/directory work.
The caller supplies the original device, an allowed target prefix and a disjoint journal extent. The library neither discovers an extent nor reads policy from an ambient path. On-media selectors cannot choose another device or escape those regions. The original device metadata must agree with logical block size and bounds; writable transaction/recovery requires explicit flush support and refuses read-only media. Caller-owned two-block scratch supplies all working storage; journal capacity derives from the selected actual extent rather than a compiled record count.
Use a checksummed versioned header and two-block undo records: a descriptor binding transaction sequence, record index, original target LBA and before-image CRC, followed by the original complete block. Header and descriptor checksums cover the full block with canonical zero padding. CRC detects corruption, not authentication. No atomic-sector-write assumption or fallback to a plausible older header is allowed. Corrupt or torn headers/records refuse recovery instead of silently initializing storage or publishing metadata.
Before each target write, persist the complete before-images, then the header's counted prefix, with a durability barrier before the first target effect. Repeated writes to one target have separate records and recover in reverse order, avoiding a quadratic duplicate search. Commit persists target changes before clearing and persisting the active header. Semantic refusal can restore a fully known journal. Any storage/transport uncertainty poisons the current adapter; it cannot resume ordinary I/O or retry through another source. Dropping an active owner does no hidden I/O: the original active header remains for explicit recovery by a later owner.
Recovery validates the entire original counted journal before its first target write, checks every original geometry/sequence/index/checksum and restores in reverse order. A crash during restoration leaves the active journal available for another idempotent recovery. Clear it only after restored targets are durable. Malformed late records must cause zero restore writes. Distinguish successful recovery, clean semantic refusal, capacity exhaustion and uncertainty; a caller cannot swallow a failed block operation and then obtain a successful commit.
The subsequent FAT metadata format will explicitly reserve this extent within its original mounted block authority, outside normal file-data access. Its records retain allocated object serials and authenticated administrative ownership/mode, independent event times and exact original-media binding. Mount/status clocks must not replace missing history. Complete permission/status integration, rename/reuse consistency, native VM fixtures and recovery remain part of the same feature; the journal core alone does not complete FAT metadata or compatibility fsync.
Author independent literal format, hostile extent, wrong-device, late-corruption, repeated-write, capacity, semantic-abort, swallowed-error and interrupted-storage controls. Tests/builds/lints/generated docs/VM/profiler/CI execution remain deferred. Keep original authority and error precedence, document all new APIs, retain strict missing_docs and files below 1000 lines, and publish coherent signed revisions. No power-loss or runtime pass is claimed from authored controls.
Explicit original-device undo journal — 30 September 2026
Corrupt or torn evidence refuses without automatic formatting or older-header fallback.
Swallowed block failures cannot commit. Uncertain storage poisons the owner and prevents further I/O; a later owner must reopen the original journal for explicit recovery. Device selectors and log records cannot discover another route or escape the original extents. Filesystem callers must separately restore or retire their volatile state and serialize other writers.
16 new tests are authored and unrun, with independently encoded media and checksums, separate volatile/stable device images, failure before/after every transaction and recovery call, hostile extents, wrong record bindings, late corruption, repeated writes, capacity/sequence refusal, swallowed errors and callback/backend unwinding.
The full status/directory feature remains open for mounted FAT metadata format, source identity/ownership/mode/event records, permissions/status, mutation/rename/reuse consistency and native integration. Both full EriX guest-build generations and external Rust/LLVM rebuilding inside EriX remain required and unperformed.
Original FAT metadata integration design — 30 September 2026
The owning feature proposal specifies explicit original-partition format selection, recovery before mount repair, persistent object records and one transaction across filesystem and metadata changes. Provider rollback must preserve consumed keys and permission evidence. The implementation and native integration remain open; no new phase acceptance is claimed. Tests/builds/CI execution remains deferred.
Original FAT mount recovery and mutation rollback — 1 October 2026
Ext and memory families cannot reinterpret this flag.
Swallowed wrong-mount or reserved-extent refusals cannot commit.
They include independent FAT12 media, original before-images/checksums, hostile extents, clean and recovery-required read-only mounts, actual mutation rollback, key/handle retirement, nested helpers, wrong-mount refusal and unwinding. Four new native scenarios cover clean admission, interrupted-boot recovery, corrupt counted records and read-only refusal; positive workloads retain existing mounted filesystem/lifecycle gates.
The complete FAT metadata feature and full status/directory feature remain open. Both complete EriX guest generations and external Rust/LLVM guest rebuilding remain required and unperformed.
Original FAT catalog and shared metadata — 1 October 2026
A metadata change followed by restoration cannot revive old permission evidence.
They cover literal codecs, full-width attributes, serial preservation/nonreuse, links, aliases, permission freshness, remount admission, rollback, read events and refusal. Two native catalog scenarios extend the complete existing storage workload and require full-width ownership, independent timestamps and cleanup. Hostile verifier controls reject false facts and mismatched replies.
The complete FAT metadata feature remains open for format-wide audit, including existing exFAT entry-set and directory-growth restrictions and broader original fixture coverage. Persistent records and the native catalog workloads are implemented; full feature acceptance and the complete status/directory feature remain open. Both complete EriX guest generations and external Rust/LLVM guest rebuilding remain required and unperformed.
Complete original metadata and status/directory source acceptance — 1 October 2026
Source acceptance covers explicit format admission/recovery, persistent original identities and full metadata, shared ordinary/acquired observations, fresh permissions, atomic filesystem/catalog updates, alias and cleanup lifetime, compatibility stat families and shared seekable directory descriptions. Imported unavailable facts remain explicit refusals.
Runtime and independent fixture producers emit the mandatory stream allocation flag, which admission and image verification require.
This checkpoint authors 14 Rust and four Python controls plus two selected exFAT VM scenarios; two existing FAT scenarios now also require six-child directory growth, post-growth observations and complete cleanup.
Both complete EriX guest generations and external Rust/LLVM guest rebuilding remain required and unperformed.
Conditional retained-entry namespace custody — 1 October 2026
Exact response correlation distinguishes completed effects, definite no-effect refusals and uncertainty.
Missing, malformed or uncorrelated replies preserve cleanup and never permit replay. The installed namespace is currently immutable and append-only; future unmount/reconfiguration requires a persistent nonwrapping epoch.
Ext and FAT return explicit correlated unsupported no-effect dispositions for the new family. Their conditional transactions remain unfinished. Complete compatibility path/dirfd, ACL/sticky, final-link and trailing-separator mediation and legacy cwd migration remain open in the owning design. Required legacy cwd and administrative spelling guards remain.
Thirty-nine new Rust controls are authored and unrun: ten literal wire controls, seventeen RAM provider controls, eight actual Vfsd dispatch controls, and two explicit-refusal controls each for ext and FAT.
Both complete EriX guest generations and external Rust/LLVM rebuilding remain required and unperformed.
Disk namespace transaction and lifetime design — 1 October 2026
The next implementation completes conditional ext/FAT entry effects before migrating legacy cwd consumers. Each selection retains its original root, parent, fixed role, full basename, binding freshness and permission fingerprint. Target observations remain metadata-only. Both roles are consumed once before the conditional effect; stale, cross-filesystem and invalid evidence must refuse without publishing a name.
Ext will use its existing metadata transaction owner and standard persistent orphan representation to retain unlinked inodes until original handles, entry owners and detached directory ancestry retire. Final reclamation and recovery must preserve inode generations, block ownership, checksums, ACL value ownership and quota accounting. FAT will use the selected semantic catalog and undo transaction owner for shared object bindings and deferred storage reclamation. Neither implementation may claim rollback after uncertain media publication.
Creation must preserve explicit mode, owner/group, mask, inherited default/access ACLs, raw symbolic targets and original timestamps. Removal, hardlink and rename must share retained-parent mutation cores with existing administrative operations, including atomic replacement and actual ancestry checks. Keep legacy cwd guards until exact-object/launch and redirection callers use original entry custody on every required provider.
Author independent media/provider and lifecycle controls; maintain inline contracts and component/manual documentation. This is a planned implementation boundary, not accepted completion. Tests, builds, lint checks, VM/profiler execution and CI observation remain deferred by instruction. Owning namespace design; complete phase requirements.
Design: conditional original FAT entry operations
The existing selected original catalog and undo owner will serve metadata-only retained-entry selection, attributed regular-file/directory creation, unlink/rmdir and two-parent rename. Original root/acquisition cleanup keys, retained actual parents, complete catalog identity/revision evidence, raw binding observations and one-use roles precede a single filesystem/catalog transaction. No path reconstruction or new provider/device route is permitted. Short replies precede all custody/effects; lost replies retain original cleanup without replay.
The established namespace contract allows explicit format-specific Unsupported. FAT/exFAT have no native hardlink or symbolic-link representation; those effects will return the correlated no-effect disposition. This does not waive any operation required by the selected guest build filesystem. Creation ownership/mode/mask, actual directory links, qualified events and atomic rename/replacement remain required on selected FAT media.
Retained-object detachment and final/restart storage reclamation remain part of this same namespace/lifetime pass. Existing spelling guards stay until their consumers have original-object lifetime. They cannot be silently removed or treated as completed POSIX lifetime. Appropriate provider and VM controls, authority/rollback review, component/manual documentation and coherent signed publication accompany implementation. Tests, builds and CI observation remain deferred by instruction.
Conditional original FAT entries — 1 October 2026
Entry rows cannot supply ordinary I/O, navigation, candidate access or legacy exact mutation. Complete replies are admitted before custody/effects; lost replies retain original-key cleanup, and consumed roles cannot replay.
Legacy 240-byte observations refuse without truncation or cursor publication. Every operation required by the selected build workload still needs implementation on its selected filesystem.
The complete FAT namespace/lifetime items remain open: persistent detached objects, alias updates, original final-close/restart reclamation, legacy cwd migration and additional VM coverage are unfinished. Existing open-object refusal guards remain until that implementation is complete.
Original FAT detached-object design — 1 October 2026
Continue the original namespace implementation through complete FAT/exFAT object lifetime. A zero-link detached record retains its never-reissued original catalog serial, data allocation, independent metadata and actual parent serial. Separate canonical detached binding domains encode only the original serial; deleted native directory slots can be reused without aliasing old objects. Detached records have no namespace entry digest and cannot be opened by pathname. Unknown domains continue to refuse complete catalog admission on older providers.
The same original undo transaction removes or replaces the native name and publishes detached catalog state whenever original handles, entry custody or detached descendants still require the object. Existing live aliases switch to the original detached serial. Subsequent content, truncate, status and permission operations use that held original object and never touch its deleted directory slot. Renaming a live object refreshes original aliases without changing identity. Detached directories retain actual original parents for counted navigation and cleanup; no directory reconstruction or hidden-name namespace is introduced.
Original close, acquisition retirement and anchor retirement must retain cleanup-only custody until required reclamation is positively complete. Reclaim unowned detached leaves before parents under the original undo owner. Mount recovery must admit complete detached state and allocation ownership before reclaiming interrupted lifetimes. In particular, no detached allocation may overlap live objects, other detached objects, the original root, exFAT bitmaps or the name-comparison table. Read-only mounts refuse required cleanup; uncertain cleanup permanently disables ordinary endpoint use. No catalog number or orphan record confers another block route.
Complete the original allocation-error distinctions and remaining format-derived name allocation bounds, then add provider and VM controls for held file/directory content, alias updates, replacement, slot reuse, cleanup failures, restart reclamation and original allocation substitution. Testing and CI observation remain deferred. Full FAT namespace/lifetime acceptance stays open until the complete implementation exists.
Original FAT detached lifetime — 1 October 2026
Known cleanup refusal preserves original custody; uncertain publication permanently disables ordinary endpoint use. Read-only reclamation refuses before mount publication. Earlier providers reject unknown detached binding domains.
Eight new provider controls are authored and unrun for both formats: aliases and slot reuse, content and truncate, rename replacement, retained parents, restart/read-only refusal, cleanup retry, uncertain reclamation, overlapping allocation and original acquisition/anchor keys.
Dedicated VM controls, storage-versus-custody exhaustion, short-alias allocation limits, shared provider-status prose and legacy cwd/exact consumer migration remain open.
Both complete EriX guest build generations and external Rust/LLVM rebuilding remain required and unperformed.
FAT namespace capacity and recovery controls — 1 October 2026
Complete long names allocate all required directory slots. Native hardlink/symlink formats explicitly refuse, so the selected build filesystem must still implement every workload-required operation.
Six host controls and eight dedicated FAT/exFAT VM scenarios cover original detached parent/child restart, undo-before-recovery ordering, read-only policy and overlapping-allocation refusal. Positive media observations require native allocation integrity, catalog and reverse-index retirement, serial nonreuse and clear original undo.
Whole compatibility namespace/lifetime parents remain open, including legacy cwd/exact consumer migration and complete cross-provider controls.
Both full EriX guest build generations and external Rust/LLVM rebuilding remain required and unperformed.
Current FAT parent observations after moves — 1 October 2026
Ordinary selected acquisitions no longer keep obsolete entering handles; pending entry roles and imported media preserve their required custody.
Four additional provider controls cover both FAT and exFAT: moved directories and old-parent reclamation, aliases/repeated moves, detached parents after entering handles retire, root clamping and cyclic ancestry refusal.
Vfsd cwd/launch/exact/redirection consumers still reconstruct path spellings; their migration must precede removal of spelling pins. Complete cross-provider lifetime controls also remain open. Checklist acceptance is unchanged: 18.82% weighted, 6.81% whole-item; 25/460 parents and 473/943 sub-items across 165 decomposed parents.
Both full EriX guest build generations and external Rust/LLVM rebuilding remain required and unperformed.
Current independent FAT directory anchors — 1 October 2026
Imported clones copy only required retained ancestors below their ceiling, carrying no source cleanup keys or broader original roots.
Full Vfsd cwd/launch/exact/redirection migration and complete cross-provider lifetime remain open; legacy spelling guards stay until their consumers migrate.
Both full EriX guest build generations and external Rust/LLVM rebuilding remain required and unperformed.
Retire legacy provider pathname and spelling-pin adapters
Current contextual consumers use independently retained native anchors,
conditional component/fixed-entry acquisition and source-free retained effects.
The older provider cwd-open, whole-target readlink and path-selected exact-entry
families now remain in provider dispatch, compatibility adapters and tests.
Inventory every opcode, codec and direct caller before deleting its producer.
Retire cwd component open (0x639), its whole-target link observation (0x63A), and
the obsolete path-selected fixed-entry acquisition/effect adapters where all
ordinary consumers have migrated. Old operation numbers remain unassigned and
must refuse without fallback. Keep actual native metadata, raw link pages,
conditional namespace effects, retained entry effects, original anchors and
acquisition-key retirement. Do not remove ordinary namespace/file interfaces
that still have real users, including generic provider readdir.
Remove cwd spelling-pin fields and their move/unlink guards only after the
corresponding creation paths and dependent tests migrate. Remove legacy exact
binding/parent spelling guards with their final acquisition producers. Preserve
original object, parent and navigation-ceiling ownership; empty directory moves
and detachment must retain actual held objects without pathname reopening or
reuse. Conditional entry freshness, stale-binding refusal, original one-use
effect semantics, permission evidence and positive cleanup remain mandatory.
Imported filesystem-format limits remain explicit and are not bypassed.
Migrate controls to actual native root/anchor/component/fixed-entry acquisition
with original keys known before submission. Do not emulate removed APIs with
test-only path reopeners, fabricated handles, copied identity or renamed legacy
production code. Preserve independent identity/mount separation, maximal ID and
capacity refusal, raw link lifetime, moved/detached ancestors, wrong handle kind,
stale selection/replacement, timestamps, media recovery and cleanup ordering.
Retain literal rejection controls for retired operation numbers. Share cohesive
fixture helpers where useful; all authored code stays below 1,000 lines.
Work through RAM, ext and FAT producers and dependent controls, then remove the
unused shared codecs/IDs and remaining Vfsd adapter references. Publish signed
checkpoints regularly and coordinate adopting immutable dependency revisions.
Update component contracts and authored manual/API addenda. Tests/builds/checks,
generated references, VM/profiler execution and CI observation remain deferred.
Runtime catalogs and both compiled Rootd helper selections remain unchanged
until coherent runtime adoption; full guest EriX and Rust/LLVM rebuilding remains
required. Partial provider retirement does not close the complete lifetime item.
Tracking: codec audit,
RAM audit,
ext audit,
FAT audit,
native ownership,
Phase 6 tracker.
Ordinary V2 response correlation no longer admits retired effect operations.
They retain
root/parent/key fields, full-width handles and timestamps, optional trusted time,
explicit selected fields, invalid destination refusal and untouched short outputs. New literal stat controls cover framing, canaries and retired-family refusal. Retired numeric replies cannot satisfy ordinary V2 or native effect correlation. Vfsd's last old exported-name references become literal timestamp-adapter refusals
for all thirteen retired operations.
The existing superseded-bridge/provider-spelling-pin acceptance criterion in
the native ownership feature
is source-complete. Required imported FAT/exFAT binding and geometry guards remain. The remaining cwd/contextual-serving and complete lifecycle/control/graph criteria
need a current-source acceptance audit and any missing implementation, followed by
coherent runtime adoption. No synthetic weighted sub-item or full lifetime acceptance is added. Recomputed
phase progress remains 18.95% weighted and 6.81% whole acceptance: 25/460 parents
and 480/949 sub-items. Full EriX guest generations and external Rust/LLVM rebuilding
inside EriX remain required and unperformed.
Related records and contracts
[FEATURE] [P02.R19] Audit authority, code quality and documentation in fatdto [FEATURE] Audit authority, code quality and documentation in fatderikinkinen referenced this issue2026-09-12 12:53:43 +02:00
erikinkinen referenced this issue2026-09-30 22:29:03 +02:00