Compare current source bytes with bounded read buffers #570

Open
erikinkinen wants to merge 8 commits from feature/retained-source-range-comparison into feature/retained-image-failed-queue
Owner

Reread every current held source and Git-context byte through bounded reusable buffers. Preserve original source registration, blob framing, metadata and named edges, deadlines and sticky first failures while removing intermediate copies that exhausted material work. Buffer allocations and read requests remain charged before effects, including conservative short-read charges.

Depends on #566, the complete helper inventory in #558, and completed syntax progress in #567.

  • Preserve current byte checks, independent descriptor ownership and sticky failures.
  • Charge bounded buffers and every admitted read without deadline renewal or short-read rebates.
  • Pass actual descriptor, original Cargo, syntax and default-CLI controls.
  • Complete a fresh Rootd producer with 77 native operations and clean retirement within the original 32 GiB allowance.

Broader image and directory-stream/SHA2 guest acceptance remains separate in #551 and #559. The Rootd artifact is retained as data; it does not replace those fresh image producers or establish guest acceptance.

Reread every current held source and Git-context byte through bounded reusable buffers. Preserve original source registration, blob framing, metadata and named edges, deadlines and sticky first failures while removing intermediate copies that exhausted material work. Buffer allocations and read requests remain charged before effects, including conservative short-read charges. Depends on [#566](https://git.erikinkinen.fi/erix/integration/pulls/566), the complete helper inventory in [#558](https://git.erikinkinen.fi/erix/integration/pulls/558), and completed syntax progress in [#567](https://git.erikinkinen.fi/erix/integration/pulls/567). - [x] Preserve current byte checks, independent descriptor ownership and sticky failures. - [x] Charge bounded buffers and every admitted read without deadline renewal or short-read rebates. - [x] Pass actual descriptor, original Cargo, syntax and default-CLI controls. - [x] Complete a fresh Rootd producer with 77 native operations and clean retirement within the original 32 GiB allowance. Broader image and directory-stream/SHA2 guest acceptance remains separate in [#551](https://git.erikinkinen.fi/erix/integration/pulls/551) and [#559](https://git.erikinkinen.fi/erix/integration/pulls/559). The Rootd artifact is retained as data; it does not replace those fresh image producers or establish guest acceptance.
erikinkinen added this to the Alpha 1 milestone 2026-11-08 02:57:00 +01:00
Resolve the explicit source-only Cstd companion once and share the verified frozen snapshot with each independent source consumer. Keep transport and compiler duties separate, preserve original finite caller limits, and retain the existing ordered helper checks in a thematic script.
fix: Refresh the complete native source verifier helper graph
Some checks failed
CI / rust (pull_request) Has been skipped
CI / native_sources (pull_request) Has been skipped
CI / host_zero (pull_request) Has been skipped
CI / host_one (pull_request) Has been skipped
CI / host_two (pull_request) Has been skipped
CI / host_three (pull_request) Has been skipped
CI / integration (pull_request) Has been skipped
Coreutils provider graph / coreutils (pull_request) Has been skipped
Markdown / markdown (pull_request) Has been skipped
Native descriptor query diagnostic / diagnostic (pull_request) Has been skipped
Markdown / markdown (push) Failing after 8s
CI / rust (push) Failing after 42s
Coreutils provider graph / coreutils (push) Successful in 4m7s
CI / native_sources (push) Has been skipped
CI / host_zero (push) Has been skipped
CI / host_one (push) Has been skipped
CI / host_two (push) Has been skipped
CI / host_three (push) Has been skipped
CI / integration (push) Has been skipped
e31e50995c
Keep the original native source verifier aligned with every literal transitive Cargo helper, including the explicit optional FD helpers. Refresh the whole reviewed declaration and consumer digest together instead of accepting partial hash variants.
Read every current original descriptor range with explicit offsets and unchanged edge, framing, clock and registration checks. Charge both buffers and each admitted read request before effects, retaining the exact first refusal without short-read rebates.
Bind the range comparison implementation to the reviewed complete helper closure before native acceptance.

# Conflicts:
#	scripts/check-python-host-syntax.sh
#	scripts/validate-integration-scripts.sh
#	tests/fixtures/python-host-syntax-commands.json
#	tests/fixtures/script-syntax-commands.json
fix: Report completed Python syntax work to the original capture owner
Some checks failed
CI / rust (pull_request) Has been skipped
CI / native_sources (pull_request) Has been skipped
CI / host_zero (pull_request) Has been skipped
CI / host_one (pull_request) Has been skipped
CI / host_two (pull_request) Has been skipped
CI / host_three (pull_request) Has been skipped
CI / integration (pull_request) Has been skipped
Coreutils provider graph / coreutils (pull_request) Has been skipped
Markdown / markdown (pull_request) Has been skipped
Native descriptor query diagnostic / diagnostic (pull_request) Has been skipped
Markdown / markdown (push) Failing after 14s
CI / rust (push) Failing after 2m3s
Coreutils provider graph / coreutils (push) Successful in 4m27s
CI / native_sources (push) Has been skipped
CI / host_one (push) Has been skipped
CI / host_zero (push) Has been skipped
CI / host_two (push) Has been skipped
CI / host_three (push) Has been skipped
CI / integration (push) Has been skipped
37ed64a3a3
Emit an opt-in literal record only after every selected source has compiled and retired successfully. Keep direct calls silent by default, preserve the complete ordered host input inventory, and leave the original hard and quiet limits unchanged.
Read every current original object byte through the same bounded buffers before checking its captured hash. Keep actual context registration, literal metadata, closure, descriptor edges and original clocks unchanged, charging every buffer and read before effects.
build: Compose completed syntax progress reporting
Some checks failed
CI / rust (pull_request) Has been skipped
CI / native_sources (pull_request) Has been skipped
CI / host_zero (pull_request) Has been skipped
CI / host_one (pull_request) Has been skipped
CI / host_two (pull_request) Has been skipped
CI / host_three (pull_request) Has been skipped
CI / integration (pull_request) Has been skipped
Coreutils provider graph / coreutils (pull_request) Has been skipped
Markdown / markdown (pull_request) Has been skipped
Markdown / markdown (push) Failing after 12s
CI / rust (push) Failing after 2m4s
Coreutils provider graph / coreutils (push) Successful in 3m58s
CI / native_sources (push) Has been skipped
CI / host_two (push) Has been skipped
CI / host_one (push) Has been skipped
CI / host_zero (push) Has been skipped
CI / host_three (push) Has been skipped
CI / integration (push) Has been skipped
712db2b8c4
Keep every explicit syntax input and independent ordered oracle while reporting actual completed checks to the original capture owner.

# Conflicts:
#	scripts/check-python-host-syntax.sh
#	tests/fixtures/python-host-syntax-commands.json
erikinkinen changed title from WIP: Compare current source bytes with bounded read buffers to Compare current source bytes with bounded read buffers 2026-11-08 03:22:48 +01:00
docs: Wrap feature documentation consistently
Some checks failed
CI / rust (pull_request) Has been skipped
CI / native_sources (pull_request) Has been skipped
CI / host_zero (pull_request) Has been skipped
CI / host_one (pull_request) Has been skipped
CI / host_two (pull_request) Has been skipped
CI / host_three (pull_request) Has been skipped
CI / integration (pull_request) Has been skipped
Coreutils provider graph / coreutils (pull_request) Has been skipped
Markdown / markdown (pull_request) Has been skipped
Markdown / markdown (push) Successful in 49s
CI / rust (push) Failing after 2m21s
Coreutils provider graph / coreutils (push) Successful in 6m48s
CI / native_sources (push) Has been skipped
CI / host_zero (push) Has been skipped
CI / host_two (push) Has been skipped
CI / host_one (push) Has been skipped
CI / host_three (push) Has been skipped
CI / integration (push) Has been skipped
24bc1c79de
Keep the existing feature documentation within the shared Markdown layout rules without changing its content.
Author
Owner

Prepared review branch: feature/retained-source-range-comparison-merge-ready-9b231c76. The original branch is unchanged.

The comparison uses bounded reusable buffers and the existing pread API. It rereads current held bytes, preserves descriptor cursors and sticky failures, and charges both fixed buffers and transient read results before effects. In the paired 4 × 8 MiB measurement, traced peak allocation falls from 33.6 MB to 206 KB; elapsed time is 0.080 s versus 0.075 s. Total charged material is slightly higher because it includes the fixed buffers.

This history depends on both #566 and #567. Neither prerequisite is approved or merged by this preparation. Reconcile the destination before merging; fresh Rootd and image acceptance must not be inferred from the scoped Cargo/source controls.

Prepared review branch: [feature/retained-source-range-comparison-merge-ready-9b231c76](https://git.erikinkinen.fi/erix/integration/compare/5bdc58306dfe28d79b6ac0f1a10c124db56a5938...9b231c76df72454070ad58fe7e9c72fc0d868744). The original branch is unchanged. The comparison uses bounded reusable buffers and the existing pread API. It rereads current held bytes, preserves descriptor cursors and sticky failures, and charges both fixed buffers and transient read results before effects. In the paired 4 × 8 MiB measurement, traced peak allocation falls from 33.6 MB to 206 KB; elapsed time is 0.080 s versus 0.075 s. Total charged material is slightly higher because it includes the fixed buffers. This history depends on both #566 and #567. Neither prerequisite is approved or merged by this preparation. Reconcile the destination before merging; fresh Rootd and image acceptance must not be inferred from the scoped Cargo/source controls.
Author
Owner

The corrected prepared branch is feature/retained-source-range-comparison-merge-ready-bbaf8a9e. Earlier review refs and the original branch are preserved.

Fresh Rootd production now completes all 77 native operations with confirmed retirement, using 32,809,215,548 bytes within the unchanged 32 GiB material allowance. The earlier portable-only attempt exceeded that allowance and is retained as a failed result.

The reader selects the optional positioned-vector API when available and otherwise uses bounded pread. Errors from the selected API remain failures; they never trigger a second read through another route. Both routes have exact-range, cursor, short-read, accounting and cancellation controls.

This demonstrates the host producer. Native EriX Python/libc API support and material acceptance, plus image and guest acceptance, remain separate. Both #566 and #567 must still be integrated into the reconciled destination before merging this history. This rewritten commit requires fresh review.

The corrected prepared branch is [feature/retained-source-range-comparison-merge-ready-bbaf8a9e](https://git.erikinkinen.fi/erix/integration/compare/5bdc58306dfe28d79b6ac0f1a10c124db56a5938...bbaf8a9eac2f47e918d1641f0854b9486f39ecd7). Earlier review refs and the original branch are preserved. Fresh Rootd production now completes all 77 native operations with confirmed retirement, using 32,809,215,548 bytes within the unchanged 32 GiB material allowance. The earlier portable-only attempt exceeded that allowance and is retained as a failed result. The reader selects the optional positioned-vector API when available and otherwise uses bounded pread. Errors from the selected API remain failures; they never trigger a second read through another route. Both routes have exact-range, cursor, short-read, accounting and cancellation controls. This demonstrates the host producer. Native EriX Python/libc API support and material acceptance, plus image and guest acceptance, remain separate. Both #566 and #567 must still be integrated into the reconciled destination before merging this history. This rewritten commit requires fresh review.
Author
Owner

Prepared dfb8c6e7768c1215ac19d7017a0252f88d5f97e5 on feature/retained-source-range-comparison-merge-ready-dfb8c6e7 now includes the canonical helper, fixture and Rust archive fixes from #564. The feature remains separate, and the original source and previous prepared ref are preserved. Review should use this new commit.

Prepared `dfb8c6e7768c1215ac19d7017a0252f88d5f97e5` on `feature/retained-source-range-comparison-merge-ready-dfb8c6e7` now includes the canonical helper, fixture and Rust archive fixes from #564. The feature remains separate, and the original source and previous prepared ref are preserved. Review should use this new commit.
Some checks failed
CI / rust (pull_request) Has been skipped
CI / native_sources (pull_request) Has been skipped
CI / host_zero (pull_request) Has been skipped
CI / host_one (pull_request) Has been skipped
CI / host_two (pull_request) Has been skipped
CI / host_three (pull_request) Has been skipped
CI / integration (pull_request) Has been skipped
Coreutils provider graph / coreutils (pull_request) Has been skipped
Markdown / markdown (pull_request) Has been skipped
Markdown / markdown (push) Successful in 49s
CI / rust (push) Failing after 2m21s
Coreutils provider graph / coreutils (push) Successful in 6m48s
CI / native_sources (push) Has been skipped
CI / host_zero (push) Has been skipped
CI / host_two (push) Has been skipped
CI / host_one (push) Has been skipped
CI / host_three (push) Has been skipped
CI / integration (push) Has been skipped
This pull request can be merged automatically.
You are not authorized to merge this pull request.
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u origin 'feature/retained-source-range-comparison:feature/retained-source-range-comparison'
git switch 'feature/retained-source-range-comparison'

Merge

Merge the changes and update on Forgejo.

Warning: The "Autodetect manual merge" setting is not enabled for this repository, you will have to mark this pull request as manually merged afterwards.

git switch 'feature/retained-image-failed-queue'
git merge --no-ff 'feature/retained-source-range-comparison'
git switch 'feature/retained-source-range-comparison'
git rebase 'feature/retained-image-failed-queue'
git switch 'feature/retained-image-failed-queue'
git merge --ff-only 'feature/retained-source-range-comparison'
git switch 'feature/retained-source-range-comparison'
git rebase 'feature/retained-image-failed-queue'
git switch 'feature/retained-image-failed-queue'
git merge --no-ff 'feature/retained-source-range-comparison'
git switch 'feature/retained-image-failed-queue'
git merge --squash 'feature/retained-source-range-comparison'
git switch 'feature/retained-image-failed-queue'
git merge --ff-only 'feature/retained-source-range-comparison'
git switch 'feature/retained-image-failed-queue'
git merge 'feature/retained-source-range-comparison'
git push origin 'feature/retained-image-failed-queue'
Sign in to join this conversation.
No description provided.