WIP: Compile retained native graph units #522

Draft
erikinkinen wants to merge 29 commits from feature/native-boot-compiler-stages into feature/native-graph-product-integrity
Owner

Run source-bound Cargo compilation and retain the actual compiler outputs and unit registrations for later graph consumers.

  • Implement the retained producer and its ownership checks.
  • Complete matching-source native and current-head CI acceptance.

Depends on #521.

Run source-bound Cargo compilation and retain the actual compiler outputs and unit registrations for later graph consumers. - [x] Implement the retained producer and its ownership checks. - [ ] Complete matching-source native and current-head CI acceptance. Depends on [#521](https://git.erikinkinen.fi/erix/integration/pulls/521).
erikinkinen added this to the Alpha 1 milestone 2026-11-07 18:56:42 +01:00
fix: verify host compiler inputs and executable headers
Some checks failed
CI / rust (pull_request) Has been skipped
CI / host_zero (pull_request) Has been skipped
CI / host_one (pull_request) Has been skipped
CI / host_two (pull_request) Has been skipped
CI / host_three (pull_request) Has been skipped
CI / markdown (pull_request) Has been skipped
CI / integration (pull_request) Has been skipped
Coreutils provider graph / coreutils (pull_request) Has been skipped
CI / markdown (push) Successful in 5s
Coreutils provider graph / coreutils (push) Successful in 1m51s
CI / rust (push) Successful in 2m37s
CI / host_zero (push) Successful in 7m50s
CI / host_three (push) Successful in 9m41s
CI / host_one (push) Failing after 10m41s
CI / host_two (push) Successful in 11m34s
CI / integration (push) Has been skipped
411b7f93db
Recheck the selected compiler driver alongside the held linker route before and after compilation. Keep detected input changes sticky after restoration, and require valid ELF versions, header size and program-header placement before accepting host companions.
Retain bounded private argv/cwd/allowlisted-environment observations under the same original Cargo producer and native operation. Keep observations separate from source, artifact and compiler-origin authority; preserve actual Cargo diagnostics, unit matching, finite work clocks and first-failure retirement.
Register pending file duties before effects, preserve the original query and compilation clock, and retain same-file capture and artifact reads until source retirement. Bind the complete 53-body helper closure and outer intake together, with real Git/Cargo controls for finite capture, partial admission, tampering and cleanup; observations remain descriptive rather than compiler-origin authority.
fix: Refuse incomplete compiler target census
Some checks failed
CI / host_zero (pull_request) Has been skipped
CI / host_one (pull_request) Has been skipped
CI / host_two (pull_request) Has been skipped
CI / host_three (pull_request) Has been skipped
CI / markdown (pull_request) Has been skipped
CI / integration (pull_request) Has been skipped
Coreutils provider graph / coreutils (pull_request) Has been skipped
CI / markdown (push) Successful in 4s
Coreutils provider graph / coreutils (push) Successful in 1m51s
CI / rust (push) Successful in 2m38s
CI / host_zero (push) Successful in 8m48s
CI / host_two (push) Failing after 9m26s
CI / host_one (push) Successful in 10m8s
CI / host_three (push) Failing after 11m54s
CI / integration (push) Has been skipped
CI / rust (pull_request) Has been skipped
371427a8cf
Propagate every original traversal error and check the same finite work budget at each directory and entry. Cover a controlled scandir failure after real Cargo compilation, and update the defining helper body and outer intake together so no skipped subtree can become accepted product accounting.
Preserve the independently reviewed product identity and same-build byte integrity checks while implementing genuine compiler stages.
Preserve original graph-parented work and complete product integrity while normally joining the checked 53-helper compiler observation cohort. Resolve the outer intake by retaining the complete defining adapter and only replacing its actual helper digest; preserve every syntax vector.
fix: retain uncertain compiler descriptor retirement
Some checks failed
CI / host_zero (push) Successful in 8m46s
CI / host_two (push) Failing after 9m37s
CI / host_one (push) Successful in 10m10s
CI / host_three (push) Failing after 12m3s
CI / integration (push) Has been skipped
CI / rust (pull_request) Has been skipped
CI / host_zero (pull_request) Has been skipped
CI / host_one (pull_request) Has been skipped
CI / host_two (pull_request) Has been skipped
CI / host_three (pull_request) Has been skipped
CI / markdown (pull_request) Has been skipped
CI / integration (pull_request) Has been skipped
Coreutils provider graph / coreutils (pull_request) Has been skipped
CI / markdown (push) Successful in 4s
Coreutils provider graph / coreutils (push) Successful in 1m53s
CI / rust (push) Successful in 2m37s
4592ee8a77
Keep each descriptor acquisition and close attempt with its original duty until positively retired. Preserve close failures across repeated or reentrant cleanup, and end the executable writer before acquiring a distinct read-only occupation so a reused descriptor number never becomes a cleanup target.
Normally join public497 pending-open and close-uncertainty ownership while retaining the original parent budget, complete graph adapter and actual helper table. No old capture or native allowance is reopened.
test: align Cargo regression fixtures with their defining stages
Some checks failed
CI / rust (pull_request) Has been skipped
CI / host_zero (pull_request) Has been skipped
CI / host_one (pull_request) Has been skipped
CI / host_two (pull_request) Has been skipped
CI / host_three (pull_request) Has been skipped
CI / markdown (pull_request) Has been skipped
CI / integration (pull_request) Has been skipped
Coreutils provider graph / coreutils (pull_request) Has been skipped
CI / markdown (push) Successful in 6s
Coreutils provider graph / coreutils (push) Successful in 1m53s
CI / rust (push) Successful in 2m39s
CI / host_zero (push) Successful in 8m52s
CI / host_two (push) Successful in 9m39s
CI / host_one (push) Successful in 10m2s
CI / host_three (push) Successful in 12m9s
CI / integration (push) Failing after 11m19s
fad4ab2ee3
Inject the cross-root lock mutation only after real metadata execution, preserving the workspace-location preflight and original lock-isolation assertions. Select the existing explicit nightly fixture tools for compiler unit-graph queries while ordinary Cargo fixtures retain their selected tools.
Borrow the same actual unsealed original source and completed Cargo unit query into a new bounded compiler birth rather than reopening retired captures or clocks. Bind closed native PIC ABI choices before query and real no-link compilation, retain complete raw CGU file duties and actual producing-unit resources, and preserve graph-end passive source/terminal lifetime. Whole runtime link and image execution remain separate gates.
Share bounded ordinary archive framing and preserve long byte names as views. Separate every native x86-64 relocatable Rust CGU from compiler metadata, decode metadata wrappers without linking them as code, and refuse unsupported or ambiguous members. Parsed bytes remain data and cannot construct source or compiler authority.
merge: include defining Cargo regression fixtures
Some checks failed
CI / rust (pull_request) Has been skipped
CI / host_zero (pull_request) Has been skipped
CI / host_one (pull_request) Has been skipped
CI / host_two (pull_request) Has been skipped
CI / host_three (pull_request) Has been skipped
CI / markdown (pull_request) Has been skipped
CI / integration (pull_request) Has been skipped
Coreutils provider graph / coreutils (push) Successful in 1m51s
Coreutils provider graph / coreutils (pull_request) Has been skipped
CI / markdown (push) Successful in 7s
CI / rust (push) Successful in 2m41s
CI / host_zero (push) Successful in 12m8s
CI / host_two (push) Successful in 14m2s
CI / host_three (push) Successful in 15m47s
CI / host_one (push) Failing after 17m11s
CI / integration (push) Has been skipped
4fd7bdf774
Compose the fixture-stage and explicit nightly selection corrections before enrolling the archive reader in the maintained host suite.
Retain genuine source and produced-file resources across effect-free lexical unit loops, with full independent checks before and after each interval. Keep partially registered units inaccessible until positive native retirement and freeze the original graph on admission or postcheck failure. This removes repeated whole-source verification without changing the original material or clock allowances.
Join the published data-only complete CGU parser and its fixture prerequisites while retaining the actual graph compiler and source owners. Archive framing does not create compiler origin or replace the producing resource.

# Conflicts:
#	scripts/check-python-host-syntax.sh
#	tests/fixtures/python-host-syntax-commands.json
Read root objects and complete Rust archive geometry only while the actual producing stage retains its original source and file duties. Check the independently emitted metadata against the archive payload and keep all returned views descriptive; no archive parser constructs source or compiler authority.
Prove the original compiler and source directory relationship before excluding the duplicate child census from aggregate admission. Keep local bounds, actual external growth, terminal reservations and the next effect charged, and retain admission failures after paths are restored.
Retain the complete actual parent and child namespace proof for aggregate retained-space admission while preserving the existing graph compiler and source owners. Replay source work is preserved separately and will extend the same defining proof before admission.
Acquire the selected target archives through original held directory and file occupations. Execute the pinned compiler query in a separate registered graph phase with the maintained caller and native forest, and reject reports that differ from the explicit selection. Keep source and file duties through native retirement for the final linker.
Refuse native runtime-input admission before registering a new phase when the graph holds an actual lexical compiler-unit borrow. A readonly compiler query still creates native effects and must wait for that original borrow to end.
Construct actual native compiler phases over the same original source and compilation owners. Recompile affected libraries and roots against retained selected archive and metadata outputs, preserving actual Cargo codegen and closed ABI choices. Keep full caller/native retirement, source and manual-file pre/post checks, complete transitive CGU access and the original finite allowances. Final runtime and Kernel links remain separate acceptance obligations.
Expose direct aliases to actual retained producing owners for every original, cached and recompiled unit under the same source/file borrow as complete CGU geometry. Verify unrecompiled dependency metadata and recompiled owner relationships against the original registered unit graph; no source key or record adapter constructs another owner.
Join coherent compiler replay with explicitly selected Rust runtime inputs. Keep every defining native phase in one closed registration, lifetime, counter and containment table, preserving lexical no-effect borrows and original graph limits.
Initialize the existing profile reservation field explicitly in the real-filesystem census model and cover the closed compiler DAG target and both capture layers. Production limits and source/native ownership rules are unchanged.
Join actual explicitly granted sysroot archive owners and the closed typed native phase registry. Preserve replay source/file borrows and all source, compiler, query, retirement and containment relationships without adding an alternate native engine.
ci: Use shared development inputs for retained graph features
Some checks failed
CI / rust (pull_request) Has been skipped
CI / host_zero (pull_request) Has been skipped
CI / host_one (pull_request) Has been skipped
CI / host_two (pull_request) Has been skipped
CI / host_three (pull_request) Has been skipped
CI / integration (pull_request) Has been skipped
Coreutils provider graph / coreutils (pull_request) Has been skipped
Markdown / markdown (pull_request) Has been skipped
CI / rust (push) Has been cancelled
CI / host_zero (push) Has been cancelled
CI / host_one (push) Has been cancelled
CI / host_two (push) Has been cancelled
CI / host_three (push) Has been cancelled
CI / integration (push) Has been cancelled
Coreutils provider graph / coreutils (push) Has been cancelled
Markdown / markdown (push) Has been cancelled
5b7dca63a2
Keep Cargo declarations on main and select the shared coordinated provider profile. Record the actual fetched helper and immutable source choices, preserving the existing Cargo gates and frozen native acceptance catalogs. Separate Markdown workflow concurrency to keep each workflow below the source size limit.
fix: Check actual unresolved Kernel symbols against dynamic exports
Some checks failed
CI / rust (pull_request) Has been skipped
CI / host_zero (pull_request) Has been skipped
CI / host_one (pull_request) Has been skipped
CI / host_two (pull_request) Has been skipped
CI / host_three (pull_request) Has been skipped
CI / integration (pull_request) Has been skipped
Coreutils provider graph / coreutils (pull_request) Has been skipped
Markdown / markdown (pull_request) Has been skipped
CI / rust (push) Has been cancelled
CI / host_zero (push) Has been cancelled
CI / host_one (push) Has been cancelled
CI / host_two (push) Has been cancelled
CI / host_three (push) Has been cancelled
CI / integration (push) Has been cancelled
Coreutils provider graph / coreutils (push) Has been cancelled
Markdown / markdown (push) Has been cancelled
0f7ad56b5f
Inspect decoded nm output from the original Kernel link operation against
that operation's selected DSO exports. Reject binary observations and role
labels as symbol sets, and retain ELF visibility so hidden, internal and
local definitions cannot satisfy another object's unresolved references.
test: Initialize the non-profiled heap reservation fixture
Some checks failed
CI / rust (push) Has been cancelled
CI / host_zero (push) Has been cancelled
CI / host_one (push) Has been cancelled
CI / host_two (push) Has been cancelled
CI / host_three (push) Has been cancelled
CI / integration (push) Has been cancelled
Coreutils provider graph / coreutils (push) Has been cancelled
Markdown / markdown (push) Has been cancelled
CI / rust (pull_request) Has been skipped
CI / host_zero (pull_request) Has been skipped
CI / host_one (pull_request) Has been skipped
CI / host_two (pull_request) Has been skipped
CI / host_three (pull_request) Has been skipped
CI / integration (pull_request) Has been skipped
Coreutils provider graph / coreutils (pull_request) Has been skipped
Markdown / markdown (pull_request) Has been skipped
e17b0b07c9
Match Scope initialization before checking destination filesystem space.
The fixture selects no telemetry sink, so its initial reservation is zero;
the actual checkpoint still accounts for pending writes and destination space.
fix: Preserve the Kernel's actual allocation failure policy
Some checks failed
CI / rust (push) Has been cancelled
CI / host_zero (push) Has been cancelled
CI / host_one (push) Has been cancelled
CI / host_two (push) Has been cancelled
CI / host_three (push) Has been cancelled
CI / integration (push) Has been cancelled
Coreutils provider graph / coreutils (push) Has been cancelled
Markdown / markdown (push) Has been cancelled
CI / rust (pull_request) Has been skipped
CI / host_zero (pull_request) Has been skipped
CI / host_one (pull_request) Has been skipped
CI / host_two (pull_request) Has been skipped
CI / host_three (pull_request) Has been skipped
CI / integration (pull_request) Has been skipped
Coreutils provider graph / coreutils (pull_request) Has been skipped
Markdown / markdown (pull_request) Has been skipped
99610da1a8
Bind Abort to the retained successful root compiler invocations and forward
observed allocation wrappers to the original custom handler or the uniquely
observed default in the selected held liballoc archive. Refuse unsupported
or ambiguous contracts instead of synthesizing a privileged halt handler.
fix: Refuse legacy custom Kernel allocation handlers
Some checks failed
CI / rust (pull_request) Has been skipped
CI / host_zero (pull_request) Has been skipped
CI / host_one (pull_request) Has been skipped
CI / host_two (pull_request) Has been skipped
CI / host_three (pull_request) Has been skipped
CI / integration (pull_request) Has been skipped
Coreutils provider graph / coreutils (pull_request) Has been skipped
Markdown / markdown (pull_request) Has been skipped
Markdown / markdown (push) Successful in 17s
Coreutils provider graph / coreutils (push) Successful in 4m40s
CI / rust (push) Successful in 7m15s
CI / host_two (push) Failing after 18m34s
CI / host_one (push) Failing after 20m45s
CI / host_zero (push) Failing after 23m59s
CI / host_three (push) Failing after 23m36s
CI / integration (push) Has been skipped
e2dfea66a8
Keep observed legacy and unsupported custom handler names in the refusal set.
An unrecognized custom allocation policy must not disappear during symbol
selection and silently select the liballoc default instead.
Some checks failed
CI / rust (pull_request) Has been skipped
CI / host_zero (pull_request) Has been skipped
CI / host_one (pull_request) Has been skipped
CI / host_two (pull_request) Has been skipped
CI / host_three (pull_request) Has been skipped
CI / integration (pull_request) Has been skipped
Coreutils provider graph / coreutils (pull_request) Has been skipped
Markdown / markdown (pull_request) Has been skipped
Markdown / markdown (push) Successful in 17s
Coreutils provider graph / coreutils (push) Successful in 4m40s
CI / rust (push) Successful in 7m15s
CI / host_two (push) Failing after 18m34s
CI / host_one (push) Failing after 20m45s
CI / host_zero (push) Failing after 23m59s
CI / host_three (push) Failing after 23m36s
CI / integration (push) Has been skipped
This pull request has changes conflicting with the target branch.
  • scripts/check-python-host-syntax.sh
  • tests/fixtures/python-host-syntax-commands.json
View command line instructions

Manual merge helper

Use this merge commit message when completing the merge manually.

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u origin 'feature/native-boot-compiler-stages:feature/native-boot-compiler-stages'
git switch 'feature/native-boot-compiler-stages'

Merge

Merge the changes and update on Forgejo.

Warning: The "Autodetect manual merge" setting is not enabled for this repository, you will have to mark this pull request as manually merged afterwards.

git switch 'feature/native-graph-product-integrity'
git merge --no-ff 'feature/native-boot-compiler-stages'
git switch 'feature/native-boot-compiler-stages'
git rebase 'feature/native-graph-product-integrity'
git switch 'feature/native-graph-product-integrity'
git merge --ff-only 'feature/native-boot-compiler-stages'
git switch 'feature/native-boot-compiler-stages'
git rebase 'feature/native-graph-product-integrity'
git switch 'feature/native-graph-product-integrity'
git merge --no-ff 'feature/native-boot-compiler-stages'
git switch 'feature/native-graph-product-integrity'
git merge --squash 'feature/native-boot-compiler-stages'
git switch 'feature/native-graph-product-integrity'
git merge --ff-only 'feature/native-boot-compiler-stages'
git switch 'feature/native-graph-product-integrity'
git merge 'feature/native-boot-compiler-stages'
git push origin 'feature/native-graph-product-integrity'
Sign in to join this conversation.
No description provided.