[FEATURE] Build and assemble the entire EriX product graph within a confined guest realm #8

New issue

Open

opened 2026-09-12 07:56:30 +02:00 by erikinkinen · 0 comments

erikinkinen commented

2026-09-12 07:56:30 +02:00

Owner

Problem and motivation

A compiler working inside EriX still leaves orchestration, native helpers, signatures, disk filesystems and documentation to implement. Full self-hosting requires every declared build and packaging step to run in the guest.

Proposed behavior and scope

Provide verified source-only bundles, offline guest orchestration and bounded build/output/tmp roots. Build all products and native boot/hash tools before use, generate signed metadata, assemble GPT/FAT/ext4 image files, and produce manuals, hashes and compressed artifacts with reproducible failure recovery.

This issue records planned work; its unchecked criteria are not implementation proof. The normative basis is Phase 6 and AC1–AC24.

Authority, security and reliability

Keep originals immutable and generated worktrees explicit. Build image files without raw disk or mount-administration rights; signing requests name exact objects and finite operations without private keys entering the realm. Native boot-tool/sha2-tool use native capability APIs, while Linux backends preserve existing behavior.

Apply the priority order: security, reliability, then performance. Keep suspected vulnerabilities in the repository’s restricted SECURITY.md reporting channel.

Acceptance criteria

Provide a signed/hash-verified source bundle of all exact component revisions,
locks, recipes, configs and local dependency metadata with no EriX precompiled output. Reject
missing or unexpected sources before building.
Implement the build realm: immutable source/toolchain inputs, separate
write-only-as-needed build/output/tmp trees, explicit execution and child lifecycle authority,
fixed locale/time policy and bounded resources.
Resolve source-write assumptions by declaring writable generated worktrees
separately from immutable originals; never silently grant compiler writes to the source
authority root.
Adapt component orchestration and Cargo override/lock resolution for fully offline
EriX execution; record every invoked guest tool, target, input/output digest and job
dependency. Remove host /proc//sys assumptions.
Build EriX-hosted boot-tool, sha2-tool and every other maintained build helper
inside EriX before using them to produce final artifacts; the first two use native EriX APIs
through integration#8–integration#8, not a POSIX-only port.
Build the complete target/profile graph from integration#1, including all services/libraries,
kernel, EFI bootloader and Integration test artifacts; fail if a catalog entry is silently
skipped due to missing build configuration.
Generate complete dynamic/launch/boot manifests and signed boot.img inside EriX,
with deterministic source epoch, ordering, padding, feature identity and no preassembled host
artifact substitution.
Resolve current dd, debugfs, mkfs.fat, mkfs.ext4, mcopy, mmd, sgdisk,
truncate/hash/compression and Python packaging dependencies. Prefer a reviewed image-file-only
path using owned code where appropriate; any imported alternative needs integration#4 admission and
actual guest functional tests.
Assemble GPT, FAT ESP and ext4 partitions into ordinary output image files inside
EriX, without granting the build realm raw block, mount administration or access to the
currently running disk device.
Build clean dev/release and explicitly selected extended profiles with equivalent
validated boot/runtime closures; verify clean images exclude imported compiler payloads even
though an imported compiler produced them.
Define least-authority artifact signing: a build requests signing of an exact
reviewed artifact identity under a dedicated test/development key policy; no private key
bytes, broad key service or release key enter the realm.
Produce checksums, compression, manifests, generated API references and complete
technical manual within EriX using the declared tool closure. Run warning-free guest
fmt/Clippy/unit/documentation checks on built sources.
Implement restartable build steps with verified input/output cache identities,
bounded parallelism, meaningful progress and deterministic cleanup; final clean-build evidence
must disable reuse of prior EriX outputs.
Test failed compiler, OOM/ENOSPC, interrupted build, malformed cache, signer denial
and partial image creation; retain diagnostics and remove only owned temporary artifacts
without corrupting source or running media.
Verify image-file parsing, signatures, partition/filesystem metadata, payload
closures and reproducibility inside EriX before exporting results for independent host
inspection/boot orchestration.
Separate boot-tool and sha2-tool command/format/hash logic from platform I/O;
specify native EriX and Linux backends sharing exact CLI, errors, output and artifact formats
without routing the EriX backend through libc.
Port sha2-tool to native startup/CLI/stdio/file capability APIs; stream large
inputs, preserve stdin/file modes, bound temporary storage and retire all input/output grants
on success, denial and interruption.
Port boot-tool image operations to native EriX file/stream APIs; preserve
build/inspect/verify/signing-payload semantics and the detached signer boundary. Neither
platform backend may gain implicit private-key access.
Keep Linux targets usable with the existing CLI and build recipes; test shared
logic plus both backends and byte-identical image/hash output from the same fixture inputs,
including empty, large and malformed files.
Package both native executables/metadata/man pages on ext4, add scoped launch
contracts and new serial/PS2 in-OS CLI tests for both tools; test wrong/missing grants and
partial-output cleanup without program-specific launch authorization.
Use the newly guest-built native tools in integration#9 for image operations and checksums;
show Linux regression CI remains green and no host fallback or POSIX adapter secretly performs
their native guest operations.

For each implementation slice, retain actual formatting, strict Clippy, unit/doctest and warning-denied build results for all altered Rust repositories and valid configurations. Add relevant runtime VM coverage, monitor older unit/VM regressions in exact-head CI, and update canonical component documents and affected technical-manual/API material. Every authored code file must remain below 1,000 physical lines, with meaningful inline documentation and missing_docs enforcement in Rust crates.

Alternatives and tradeoffs

Host image assembly or prebuilt helper substitution would leave the build incomplete. Shared pure command/format logic with separate native and Linux I/O backends preserves parity without routing native tools through a POSIX-only adapter.

Tracking and rollout

Dependencies: integration#1, posixd#1, posixd#2, lib-cstd#1, exsh#1, integration#4, integration#7, bootloader#1

Dependencies identify required contracts and closure gates; preparatory inventory/design can proceed in parallel under one owner per edited file. Link bounded implementation issues and their PRs here before claiming acceptance. Use feature/posix-compat, regular signed commits in the canonical contribution format, and WIP PRs linked to the exact coherent component graph. All cross-repository Cargo/catalog selections and CI helpers use full 40-character lowercase commit hashes, including transitive dependencies; do not substitute branch, tag or implicit HEAD selection.

Close criteria only with their own reviewed deliverables and validation evidence. Pending, skipped, cancelled, failed or predecessor-only results remain distinct. Keep main images unchanged until explicit promotion direction; technical completion does not authorize merges, release tags or publication.

## Problem and motivation A compiler working inside EriX still leaves orchestration, native helpers, signatures, disk filesystems and documentation to implement. Full self-hosting requires every declared build and packaging step to run in the guest. ## Proposed behavior and scope Provide verified source-only bundles, offline guest orchestration and bounded build/output/tmp roots. Build all products and native boot/hash tools before use, generate signed metadata, assemble GPT/FAT/ext4 image files, and produce manuals, hashes and compressed artifacts with reproducible failure recovery. This issue records planned work; its unchecked criteria are not implementation proof. The normative basis is [Phase 6 and AC1–AC24](https://git.erikinkinen.fi/erix/docs/src/branch/main/phases/6.md). ## Authority, security and reliability Keep originals immutable and generated worktrees explicit. Build image files without raw disk or mount-administration rights; signing requests name exact objects and finite operations without private keys entering the realm. Native boot-tool/sha2-tool use native capability APIs, while Linux backends preserve existing behavior. Apply the priority order: security, reliability, then performance. Keep suspected vulnerabilities in the repository’s restricted SECURITY.md reporting channel. ## Acceptance criteria - [ ] Provide a signed/hash-verified source bundle of all exact component revisions, locks, recipes, configs and local dependency metadata with no EriX precompiled output. Reject missing or unexpected sources before building. - [ ] Implement the build realm: immutable source/toolchain inputs, separate write-only-as-needed build/output/tmp trees, explicit execution and child lifecycle authority, fixed locale/time policy and bounded resources. - [ ] Resolve source-write assumptions by declaring writable generated worktrees separately from immutable originals; never silently grant compiler writes to the source authority root. - [ ] Adapt component orchestration and Cargo override/lock resolution for fully offline EriX execution; record every invoked guest tool, target, input/output digest and job dependency. Remove host `/proc`/`/sys` assumptions. - [ ] Build EriX-hosted `boot-tool`, `sha2-tool` and every other maintained build helper inside EriX before using them to produce final artifacts; the first two use native EriX APIs through [integration#8](https://git.erikinkinen.fi/erix/integration/issues/8)–[integration#8](https://git.erikinkinen.fi/erix/integration/issues/8), not a POSIX-only port. - [ ] Build the complete target/profile graph from [integration#1](https://git.erikinkinen.fi/erix/integration/issues/1), including all services/libraries, kernel, EFI bootloader and Integration test artifacts; fail if a catalog entry is silently skipped due to missing build configuration. - [ ] Generate complete dynamic/launch/boot manifests and signed `boot.img` inside EriX, with deterministic source epoch, ordering, padding, feature identity and no preassembled host artifact substitution. - [ ] Resolve current `dd`, `debugfs`, `mkfs.fat`, `mkfs.ext4`, `mcopy`, `mmd`, `sgdisk`, truncate/hash/compression and Python packaging dependencies. Prefer a reviewed image-file-only path using owned code where appropriate; any imported alternative needs [integration#4](https://git.erikinkinen.fi/erix/integration/issues/4) admission and actual guest functional tests. - [ ] Assemble GPT, FAT ESP and ext4 partitions into ordinary output image files inside EriX, without granting the build realm raw block, mount administration or access to the currently running disk device. - [ ] Build clean dev/release and explicitly selected extended profiles with equivalent validated boot/runtime closures; verify clean images exclude imported compiler payloads even though an imported compiler produced them. - [ ] Define least-authority artifact signing: a build requests signing of an exact reviewed artifact identity under a dedicated test/development key policy; no private key bytes, broad key service or release key enter the realm. - [ ] Produce checksums, compression, manifests, generated API references and complete technical manual within EriX using the declared tool closure. Run warning-free guest fmt/Clippy/unit/documentation checks on built sources. - [ ] Implement restartable build steps with verified input/output cache identities, bounded parallelism, meaningful progress and deterministic cleanup; final clean-build evidence must disable reuse of prior EriX outputs. - [ ] Test failed compiler, OOM/ENOSPC, interrupted build, malformed cache, signer denial and partial image creation; retain diagnostics and remove only owned temporary artifacts without corrupting source or running media. - [ ] Verify image-file parsing, signatures, partition/filesystem metadata, payload closures and reproducibility inside EriX before exporting results for independent host inspection/boot orchestration. - [ ] Separate `boot-tool` and `sha2-tool` command/format/hash logic from platform I/O; specify native EriX and Linux backends sharing exact CLI, errors, output and artifact formats without routing the EriX backend through libc. - [ ] Port `sha2-tool` to native startup/CLI/stdio/file capability APIs; stream large inputs, preserve stdin/file modes, bound temporary storage and retire all input/output grants on success, denial and interruption. - [ ] Port `boot-tool` image operations to native EriX file/stream APIs; preserve build/inspect/verify/signing-payload semantics and the detached signer boundary. Neither platform backend may gain implicit private-key access. - [ ] Keep Linux targets usable with the existing CLI and build recipes; test shared logic plus both backends and byte-identical image/hash output from the same fixture inputs, including empty, large and malformed files. - [ ] Package both native executables/metadata/man pages on ext4, add scoped launch contracts and new serial/PS2 in-OS CLI tests for both tools; test wrong/missing grants and partial-output cleanup without program-specific launch authorization. - [ ] Use the newly guest-built native tools in [integration#9](https://git.erikinkinen.fi/erix/integration/issues/9) for image operations and checksums; show Linux regression CI remains green and no host fallback or POSIX adapter secretly performs their native guest operations. For each implementation slice, retain actual formatting, strict Clippy, unit/doctest and warning-denied build results for all altered Rust repositories and valid configurations. Add relevant runtime VM coverage, monitor older unit/VM regressions in exact-head CI, and update canonical component documents and affected technical-manual/API material. Every authored code file must remain below 1,000 physical lines, with meaningful inline documentation and missing_docs enforcement in Rust crates. ## Alternatives and tradeoffs Host image assembly or prebuilt helper substitution would leave the build incomplete. Shared pure command/format logic with separate native and Linux I/O backends preserves parity without routing native tools through a POSIX-only adapter. ## Tracking and rollout Dependencies: [integration#1](https://git.erikinkinen.fi/erix/integration/issues/1), [posixd#1](https://git.erikinkinen.fi/erix/posixd/issues/1), [posixd#2](https://git.erikinkinen.fi/erix/posixd/issues/2), [lib-cstd#1](https://git.erikinkinen.fi/erix/lib-cstd/issues/1), [exsh#1](https://git.erikinkinen.fi/erix/exsh/issues/1), [integration#4](https://git.erikinkinen.fi/erix/integration/issues/4), [integration#7](https://git.erikinkinen.fi/erix/integration/issues/7), [bootloader#1](https://git.erikinkinen.fi/erix/bootloader/issues/1) Dependencies identify required contracts and closure gates; preparatory inventory/design can proceed in parallel under one owner per edited file. Link bounded implementation issues and their PRs here before claiming acceptance. Use `feature/posix-compat`, regular signed commits in the canonical contribution format, and WIP PRs linked to the exact coherent component graph. All cross-repository Cargo/catalog selections and CI helpers use full 40-character lowercase commit hashes, including transitive dependencies; do not substitute branch, tag or implicit HEAD selection. Close criteria only with their own reviewed deliverables and validation evidence. Pending, skipped, cancelled, failed or predecessor-only results remain distinct. Keep main images unchanged until explicit promotion direction; technical completion does not authorize merges, release tags or publication.

erikinkinen added the

labels

2026-09-12 07:56:30 +02:00

erikinkinen referenced this issue

2026-09-12 07:56:30 +02:00

[FEATURE] Demonstrate two complete generations of EriX built inside EriX #9

erikinkinen referenced this issue

2026-09-12 07:56:30 +02:00

[FEATURE] Close final security, documentation, CI and self-hosting acceptance #10

erikinkinen referenced this issue from erix/ttyd

2026-09-12 08:02:04 +02:00

[FEATURE] Complete independent consoles, vttys, TUI, keyboard layouts, mouse and boot logo #1

erikinkinen changed title from ~~[FEATURE] [P15] Build and assemble the entire EriX product graph within a confined guest realm~~ to [FEATURE] Build and assemble the entire EriX product graph within a confined guest realm

2026-09-12 08:02:07 +02:00

erikinkinen referenced this issue

2026-09-19 11:36:52 +02:00

[FEATURE] Track Phase 6 POSIX compatibility and full EriX self-hosting #65