[FEATURE] Load compatibility ELF, TLS, constructors and declared runtime modules safely #1

Open
opened 2026-09-12 07:56:28 +02:00 by erikinkinen · 0 comments
Owner

Problem and motivation

Imported runtimes emit TLS, constructors, exceptions and library features beyond the native abort-only artifact policy. Supporting them requires a distinct verified contract rather than removing native loader guards.

Proposed behavior and scope

Inventory the exact toolchain’s ELF features and implement bounded relocations, symbol rules, TLS/module lifecycle, constructor/finalizer order, declared runtime loading and compatibility-only unwind registration. Run actual C++/Rust runtime VMs alongside native-loader regressions.

This issue records planned work; its unchecked criteria are not implementation proof. The normative basis is Phase 6 and AC1–AC24.

Authority, security and reliability

Validate complete signed module identity and dependency closure before initialization. Confine library lookup to granted roots, preserve W^X/RELRO and exact mapped object identity, and retire mappings only after their users are gone. Unsupported dangerous binding modes require separate design and evidence.

Apply the priority order: security, reliability, then performance. Keep suspected vulnerabilities in the repository’s restricted SECURITY.md reporting channel.

Acceptance criteria

Checklist
  • Inventory emitted ELF/interpreter/dynamic tags, relocations, symbol versions, TLS
    models, constructor/finalizer and exception metadata from exact selected compiler/runtime
    packages.
  • Define/version a compatibility artifact contract distinct from native
    abort-only/no-TLS/no-init-fini policy; preserve rejection of unsupported native artifacts
    instead of simply deleting existing guards.
  • Implement required relocations and dynamic symbol rules with exact bounds,
    dependency closure, cycle/duplicate behavior and no ambient library search.
  • Implement static/dynamic TLS layout and per-thread/module lifecycle with checked
    template/BSS alignment, unload rules and kernel#1 synchronization.
  • Implement ordered constructors/finalizers and runtime teardown with failure
    handling; no initializer executes before validation and final mappings.
  • Implement declared runtime library loading needed by Python modules and compiler
    plugins/proc macros; restrict dlopen-like search to granted library roots and retire
    mappings/aliases only after their users are gone.
  • Implement required unwind registration/lookup in compatibility userspace only;
    validate metadata and keep native/kernel abort policy intact.
  • Enforce W^X, relocation/RELRO permissions, manifest identity and dependency closure
    for imported objects; reject text relocation/lazy binding/ IFUNC modes unless separately
    required, designed, tested and approved.
  • Test malformed ELF, dependency substitution, TLS races, constructor failure, symbol
    ambiguity, plugin teardown and out-of-realm library lookup.
  • Run actual compatibility ELF/TLS/C++ exception/Rust runtime VMs and native-loader
    regression scenarios with strict warning-free component CI.

For each implementation slice, retain actual formatting, strict Clippy, unit/doctest and warning-denied build results for all altered Rust repositories and valid configurations. Add relevant runtime VM coverage, monitor older unit/VM regressions in exact-head CI, and update canonical component documents and affected technical-manual/API material. Every authored code file must remain below 1,000 physical lines, with meaningful inline documentation and missing_docs enforcement in Rust crates.

Alternatives and tradeoffs

A permissive generic ELF loader would silently weaken the native contract. Version compatibility artifacts separately and admit only emitted features justified by the selected packages.

Matching catalog adoption and real consumer VMs remain requirements; complete realm and full in-guest build acceptance remain open.

Original coherent realm source CI acceptance — 18 September 2026: Signed ff2cb5f2b8059c26d8e3bff990d0dba60100113e passes CI 40 and CI 39. Coherent catalog publication, actual consumer VMs and full guest-build acceptance remain separate open requirements.

Tracking and rollout

All 34 maintained local checks pass: development/release feature matrices (15 unit tests per selection), strict host/native Clippy, warning-denied builds, private rustdoc, applicable doctests and component-specific checks, formatting and exact dependency-source validation. Native binaries have executable read-only entry pages and no writable executable pages. Original CI49 and CI50 pass; their complete logs are independently reviewed with zero warnings. No unchanged CI was rerun. Coordinated VM configuration/seal/four-client I/O, repeated managed-row reuse, profiler observation and both full builds inside EriX remain open under the phase checklist. This dependency work earns no separate feature-completion credit.

All 4 selected development/release configurations pass (13 unit tests per configuration). Strict host/native Clippy, warning-denied host/native builds, doctests where provided, private rustdoc, formatting, Markdown and dependency checks pass with zero warnings.

Matching-image native Posixd startup, two-realm independence and retirement acceptance remain pending. These local checks do not establish sealing, client I/O, a complete EriX guest build or the external toolchain rebuild. Publication CI is tracked separately from local validation.

Detailed scope and contracts

Runtime consumer dependency alignment — 15 September 2026

The product catalog, product VM acceptance and guest build remain pending.

Dependencies identify required contracts and closure gates; preparatory inventory/design can proceed in parallel under one owner per edited file. Link bounded implementation issues and their PRs here before claiming acceptance.

Close criteria only with their own reviewed deliverables and validation evidence.

Full catalog equality and actual consumer-image validation remain required under Integration regression 68. This dependency checkpoint does not close full lifecycle, frame or self-hosting acceptance.

Full service CPU/profiler VM acceptance and guest builds remain open in Phase 6 completion.

Native process/thread ownership specification — 4 October 2026

The complete ownership specification defines one original process with shared CSpace/VSpace, capability/fd/action/cwd/creation-mask policy and distinct authenticated native execution occupations. An explicitly sealed original-process factory and retained thread handles authorize Prepare/Start and cleanup; clients gain no global Process route, root or MAP grant. The actual original creator and counted enrollment precede Start. Native membership and caller origin never come from payload labels or copied ClientSlot policy.

The contract separates thread-only exit from process exit. Initial-thread departure preserves peers and shared resources. Last-thread preparation serializes creation/Start, settles potentially startable original members and retains the final context for process finalization. Whole-group retirement drains every original member/provider before shared authority reclamation and emits one process event. Join/detach arbitrate one original target, preserve joinability on cancelled uncommitted joins and reconcile committed disposition through reply loss without affecting a replacement generation.

Stack/guard, caller-owned storage, TLS/errno/module/destructor and complete architecture state have explicit original lifetime and reclamation boundaries. Current OIR1's 64-byte header uses bytes 56–63 for terminal status; the proposed versioned OIR2 extent is 72 bytes, adding thread identity/generation at 64–71. No runtime ABI/codec is changed by this specification. Mediator process policy remains shared while real enrolled threads own distinct masks/handlers. Fork/exec, native provisioning, target geometry, memory/loader/linker/libc and real workload adoption are coordinated requirements.

Integration's native lifecycle contract records independent actual membership, blocked-peer progress, initial/last exit, TLS/context, guards, join/detach, failure/reuse, group loss, profiling and tool-consumer evidence. The imported-tool source/version/options and transitive interface matrix remains unresolved. Current one-TCB asynchronous calls do not implement native shared-process threads.

Specification acceptance completes the ownership specification only. Native process/member/factory/origin mechanisms, thread-only exit, join/detach, per-thread TLS, libc/imported consumers and real profiling remain open. Both complete EriX guest generations and external Rust/LLVM rebuilding inside EriX remain required and unperformed.

Three signed commits across Kernel, Docs and Integration publish 17 paths: 14 maintained documentation sources and three catalog files containing four selector updates. No runtime or Cargo source changes; no Rustfmt inputs. Read-only source/Git inventory records all 269 compiled edges and 162 catalog selections coherent, 4,346 code files below 1,000 lines, retained missing-docs gates, and 153 guards across 76 workflows. All 76 feature trees are clean except preserved Meta cache. Rootd and helper catalog entries retain independent compiled helper cc878220040dfa741491249434d6b8f8fe2252f7.

Signed documentation/adoption: docs af376f65, integration 5daf30c3, kernel 9660dacf.

Tracking: ownership specification, native implementation, socket consumers, libc, compatibility ELF/TLS, phase completion.

The complete ownership specification is source-accepted. Every native runtime, thread/TLS, imported consumer, profiling and guest-build implementation/evidence requirement retains its existing open scope.

Original ELF TLS image and linking source — 11 October 2026

Static native TLS refusal remains intact.

Complete caller-image preflight precedes every byte effect; refusal and spare bytes remain unchanged. Scalar addresses, IDs and generations grant no native authority.

Ordinary graph planning refuses TLS, and the distinct patch type cannot enter existing eight-byte-only handoff consumers. Descriptor resolver addresses remain descriptive; actual register-preserving code and its native mapping must be authenticated separately. Missing weak TLS providers, static-linker annotations and complete actual imported-artifact coverage remain unfinished.

No allocation, ambient lookup, numeric address dereference, capability minting, FS setter or runtime policy widening is introduced. Twenty-one independent template/image/expression/graph controls cover malformed/overflow/alignment/refusal, independent images, provider identity, full descriptor targets and late-refusal/retry output custody.

Actual Loaderd/Dynlinkd/Procd/Kernel initial and additional TLS image production, LibCstd errno/TLS/descriptor accessors, ordinary creation/deployment, inherited FP state, member exit/join/detach, module mutation/unload/destructors and all required native/imported consumers remain unfinished. Existing executable/artifact TLS refusal stays until every thread is initialized. Both complete EriX guest build generations and external Rust/LLVM rebuilding inside EriX remain mandatory and unperformed.

Canonical completion is 21.88% weighted and 10.11% whole acceptance: 33/460 accepted parents, 505/978 completed sub-items under 168 decomposed parents. Two explicit template/image source sub-items are credited; complete TLS and native thread lifecycle requirements remain open.

Tracking: complete TLS, native thread acceptance, linker runtime, libc, thread consumers, phase completion.

Related records and contracts
Repository Links
bootloader PR 3
docs PR 4
dynlinkd PR 3
integration issue 1, PR 12
kernel issue 20, PR 3
lib-dynlink PR 3
lib-elf PR 2
lib-posixabi issue 1
loaderd PR 2
rootd PR 2
## Problem and motivation Imported runtimes emit TLS, constructors, exceptions and library features beyond the native abort-only artifact policy. Supporting them requires a distinct verified contract rather than removing native loader guards. ## Proposed behavior and scope Inventory the exact toolchain’s ELF features and implement bounded relocations, symbol rules, TLS/module lifecycle, constructor/finalizer order, declared runtime loading and compatibility-only unwind registration. Run actual C++/Rust runtime VMs alongside native-loader regressions. This issue records planned work; its unchecked criteria are not implementation proof. The normative basis is [Phase 6 and AC1–AC24](https://git.erikinkinen.fi/erix/docs/src/branch/main/phases/6.md). ## Authority, security and reliability Validate complete signed module identity and dependency closure before initialization. Confine library lookup to granted roots, preserve W^X/RELRO and exact mapped object identity, and retire mappings only after their users are gone. Unsupported dangerous binding modes require separate design and evidence. Apply the priority order: security, reliability, then performance. Keep suspected vulnerabilities in the repository’s restricted SECURITY.md reporting channel. ## Acceptance criteria <details> <summary>Checklist</summary> - [ ] Inventory emitted ELF/interpreter/dynamic tags, relocations, symbol versions, TLS models, constructor/finalizer and exception metadata from exact selected compiler/runtime packages. - [ ] Define/version a compatibility artifact contract distinct from native abort-only/no-TLS/no-init-fini policy; preserve rejection of unsupported native artifacts instead of simply deleting existing guards. - [ ] Implement required relocations and dynamic symbol rules with exact bounds, dependency closure, cycle/duplicate behavior and no ambient library search. - [ ] Implement static/dynamic TLS layout and per-thread/module lifecycle with checked template/BSS alignment, unload rules and [kernel#1](https://git.erikinkinen.fi/erix/kernel/issues/1) synchronization. - [ ] Implement ordered constructors/finalizers and runtime teardown with failure handling; no initializer executes before validation and final mappings. - [ ] Implement declared runtime library loading needed by Python modules and compiler plugins/proc macros; restrict dlopen-like search to granted library roots and retire mappings/aliases only after their users are gone. - [ ] Implement required unwind registration/lookup in compatibility userspace only; validate metadata and keep native/kernel abort policy intact. - [ ] Enforce W^X, relocation/RELRO permissions, manifest identity and dependency closure for imported objects; reject text relocation/lazy binding/ IFUNC modes unless separately required, designed, tested and approved. - [ ] Test malformed ELF, dependency substitution, TLS races, constructor failure, symbol ambiguity, plugin teardown and out-of-realm library lookup. - [ ] Run actual compatibility ELF/TLS/C++ exception/Rust runtime VMs and native-loader regression scenarios with strict warning-free component CI. For each implementation slice, retain actual formatting, strict Clippy, unit/doctest and warning-denied build results for all altered Rust repositories and valid configurations. Add relevant runtime VM coverage, monitor older unit/VM regressions in exact-head CI, and update canonical component documents and affected technical-manual/API material. Every authored code file must remain below 1,000 physical lines, with meaningful inline documentation and missing_docs enforcement in Rust crates. </details> ## Alternatives and tradeoffs A permissive generic ELF loader would silently weaken the native contract. Version compatibility artifacts separately and admit only emitted features justified by the selected packages. Matching catalog adoption and real consumer VMs remain requirements; complete realm and full in-guest build acceptance remain open. Original coherent realm source CI acceptance — 18 September 2026: Signed `ff2cb5f2b8059c26d8e3bff990d0dba60100113e` passes [CI 40](https://git.erikinkinen.fi/erix/dynlinkd/actions/runs/40) and [CI 39](https://git.erikinkinen.fi/erix/dynlinkd/actions/runs/39). Coherent catalog publication, actual consumer VMs and full guest-build acceptance remain separate open requirements. ## Tracking and rollout All 34 maintained local checks pass: development/release feature matrices (15 unit tests per selection), strict host/native Clippy, warning-denied builds, private rustdoc, applicable doctests and component-specific checks, formatting and exact dependency-source validation. Native binaries have executable read-only entry pages and no writable executable pages. Original [CI49](https://git.erikinkinen.fi/erix/dynlinkd/actions/runs/49) and [CI50](https://git.erikinkinen.fi/erix/dynlinkd/actions/runs/50) pass; their complete logs are independently reviewed with zero warnings. No unchanged CI was rerun. Coordinated VM configuration/seal/four-client I/O, repeated managed-row reuse, profiler observation and both full builds inside EriX remain open under the [phase checklist](https://git.erikinkinen.fi/erix/integration/issues/65). This dependency work earns no separate feature-completion credit. All 4 selected development/release configurations pass (13 unit tests per configuration). Strict host/native Clippy, warning-denied host/native builds, doctests where provided, private rustdoc, formatting, Markdown and dependency checks pass with zero warnings. Matching-image native Posixd startup, two-realm independence and retirement acceptance remain pending. These local checks do not establish sealing, client I/O, a complete EriX guest build or the external toolchain rebuild. Publication CI is tracked separately from local validation. <details> <summary>Detailed scope and contracts</summary> ### Runtime consumer dependency alignment — 15 September 2026 The product catalog, product VM acceptance and guest build remain pending. Dependencies identify required contracts and closure gates; preparatory inventory/design can proceed in parallel under one owner per edited file. Link bounded implementation issues and their PRs here before claiming acceptance. Close criteria only with their own reviewed deliverables and validation evidence. Full catalog equality and actual consumer-image validation remain required under [Integration regression 68](https://git.erikinkinen.fi/erix/integration/issues/68). This dependency checkpoint does not close full lifecycle, frame or self-hosting acceptance. Full service CPU/profiler VM acceptance and guest builds remain open in [Phase 6 completion](https://git.erikinkinen.fi/erix/integration/issues/65). ### Native process/thread ownership specification — 4 October 2026 [The complete ownership specification](https://git.erikinkinen.fi/erix/kernel/src/branch/feature/posix-compat/docs/process-thread-ownership.md) defines one original process with shared CSpace/VSpace, capability/fd/action/cwd/creation-mask policy and distinct authenticated native execution occupations. An explicitly sealed original-process factory and retained thread handles authorize Prepare/Start and cleanup; clients gain no global Process route, root or MAP grant. The actual original creator and counted enrollment precede Start. Native membership and caller origin never come from payload labels or copied ClientSlot policy. The contract separates thread-only exit from process exit. Initial-thread departure preserves peers and shared resources. Last-thread preparation serializes creation/Start, settles potentially startable original members and retains the final context for process finalization. Whole-group retirement drains every original member/provider before shared authority reclamation and emits one process event. Join/detach arbitrate one original target, preserve joinability on cancelled uncommitted joins and reconcile committed disposition through reply loss without affecting a replacement generation. Stack/guard, caller-owned storage, TLS/errno/module/destructor and complete architecture state have explicit original lifetime and reclamation boundaries. Current OIR1's 64-byte header uses bytes 56–63 for terminal status; the proposed versioned OIR2 extent is 72 bytes, adding thread identity/generation at 64–71. No runtime ABI/codec is changed by this specification. Mediator process policy remains shared while real enrolled threads own distinct masks/handlers. Fork/exec, native provisioning, target geometry, memory/loader/linker/libc and real workload adoption are coordinated requirements. [Integration's native lifecycle contract](https://git.erikinkinen.fi/erix/integration/src/branch/feature/posix-compat/docs/native-thread-lifecycle.md) records independent actual membership, blocked-peer progress, initial/last exit, TLS/context, guards, join/detach, failure/reuse, group loss, profiling and tool-consumer evidence. The imported-tool source/version/options and transitive interface matrix remains unresolved. Current one-TCB asynchronous calls do not implement native shared-process threads. Specification acceptance completes the ownership specification only. Native process/member/factory/origin mechanisms, thread-only exit, join/detach, per-thread TLS, libc/imported consumers and real profiling remain open. Both complete EriX guest generations and external Rust/LLVM rebuilding inside EriX remain required and unperformed. Three signed commits across Kernel, Docs and Integration publish 17 paths: 14 maintained documentation sources and three catalog files containing four selector updates. No runtime or Cargo source changes; no Rustfmt inputs. Read-only source/Git inventory records all 269 compiled edges and 162 catalog selections coherent, 4,346 code files below 1,000 lines, retained missing-docs gates, and 153 guards across 76 workflows. All 76 feature trees are clean except preserved Meta cache. Rootd and helper catalog entries retain independent compiled helper `cc878220040dfa741491249434d6b8f8fe2252f7`. Signed documentation/adoption: [docs af376f65](https://git.erikinkinen.fi/erix/docs/commit/af376f658022125af232de67c22427cd7554c202), [integration 5daf30c3](https://git.erikinkinen.fi/erix/integration/commit/5daf30c307a59d2df10a8c694f89d283b10deaf0), [kernel 9660dacf](https://git.erikinkinen.fi/erix/kernel/commit/9660dacf70b80cfce387046e5df6e54232f8619e). Tracking: [ownership specification](https://git.erikinkinen.fi/erix/kernel/issues/44), [native implementation](https://git.erikinkinen.fi/erix/kernel/issues/1), [socket consumers](https://git.erikinkinen.fi/erix/posixd/issues/34), [libc](https://git.erikinkinen.fi/erix/lib-cstd/issues/1), [compatibility ELF/TLS](https://git.erikinkinen.fi/erix/dynlinkd/issues/1), [phase completion](https://git.erikinkinen.fi/erix/integration/issues/65). The complete ownership specification is source-accepted. Every native runtime, thread/TLS, imported consumer, profiling and guest-build implementation/evidence requirement retains its existing open scope. ### Original ELF TLS image and linking source — 11 October 2026 Static native TLS refusal remains intact. Complete caller-image preflight precedes every byte effect; refusal and spare bytes remain unchanged. Scalar addresses, IDs and generations grant no native authority. Ordinary graph planning refuses TLS, and the distinct patch type cannot enter existing eight-byte-only handoff consumers. Descriptor resolver addresses remain descriptive; actual register-preserving code and its native mapping must be authenticated separately. Missing weak TLS providers, static-linker annotations and complete actual imported-artifact coverage remain unfinished. No allocation, ambient lookup, numeric address dereference, capability minting, FS setter or runtime policy widening is introduced. Twenty-one independent template/image/expression/graph controls cover malformed/overflow/alignment/refusal, independent images, provider identity, full descriptor targets and late-refusal/retry output custody. Actual Loaderd/Dynlinkd/Procd/Kernel initial and additional TLS image production, LibCstd errno/TLS/descriptor accessors, ordinary creation/deployment, inherited FP state, member exit/join/detach, module mutation/unload/destructors and all required native/imported consumers remain unfinished. Existing executable/artifact TLS refusal stays until every thread is initialized. Both complete EriX guest build generations and external Rust/LLVM rebuilding inside EriX remain mandatory and unperformed. Canonical completion is 21.88% weighted and 10.11% whole acceptance: 33/460 accepted parents, 505/978 completed sub-items under 168 decomposed parents. Two explicit template/image source sub-items are credited; complete TLS and native thread lifecycle requirements remain open. Tracking: [complete TLS](https://git.erikinkinen.fi/erix/lib-dynlink/issues/4), [native thread acceptance](https://git.erikinkinen.fi/erix/kernel/issues/47), [linker runtime](https://git.erikinkinen.fi/erix/dynlinkd/issues/1), [libc](https://git.erikinkinen.fi/erix/lib-cstd/issues/1), [thread consumers](https://git.erikinkinen.fi/erix/posixd/issues/34), [phase completion](https://git.erikinkinen.fi/erix/integration/issues/65). </details> <details> <summary>Related records and contracts</summary> | Repository | Links | | --- | --- | | bootloader | [PR 3](https://git.erikinkinen.fi/erix/bootloader/pulls/3) | | docs | [PR 4](https://git.erikinkinen.fi/erix/docs/pulls/4) | | dynlinkd | [PR 3](https://git.erikinkinen.fi/erix/dynlinkd/pulls/3) | | integration | [issue 1](https://git.erikinkinen.fi/erix/integration/issues/1), [PR 12](https://git.erikinkinen.fi/erix/integration/pulls/12) | | kernel | [issue 20](https://git.erikinkinen.fi/erix/kernel/issues/20), [PR 3](https://git.erikinkinen.fi/erix/kernel/pulls/3) | | lib-dynlink | [PR 3](https://git.erikinkinen.fi/erix/lib-dynlink/pulls/3) | | lib-elf | [PR 2](https://git.erikinkinen.fi/erix/lib-elf/pulls/2) | | lib-posixabi | [issue 1](https://git.erikinkinen.fi/erix/lib-posixabi/issues/1) | | loaderd | [PR 2](https://git.erikinkinen.fi/erix/loaderd/pulls/2) | | rootd | [PR 2](https://git.erikinkinen.fi/erix/rootd/pulls/2) | </details>
erikinkinen changed title from [FEATURE] [P11] Load compatibility ELF, TLS, constructors and declared runtime modules safely to [FEATURE] Load compatibility ELF, TLS, constructors and declared runtime modules safely 2026-09-12 08:02:06 +02:00
erikinkinen added this to the Alpha 1 milestone 2026-10-30 09:33:44 +01:00
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
erix/dynlinkd#1
No description provided.