[BUG] Catalog helper acquisition exhausts file descriptors on larger source graphs #728

Open
opened 2026-11-11 09:49:53 +01:00 by erikinkinen · 0 comments
Owner

Summary

Catalog helper verification opens every selected helper before reading the batch. A sufficiently large source graph can exhaust the process descriptor allowance before any helper body is verified.

Reproduction

Run the maintained native build-script producer controls, including test_actual_two_same_stage_libraries_keep_owned_captures, with the workflow's selected Cargo and Rust compiler. The cargo job in run 9968 reproduces the failure at commit 0c01f1c020853a098a5b5e9de89674e4d2662268.

Expected and actual behavior

Helper verification should use bounded live descriptor headroom as the helper roster grows, while retaining the original file proofs, complete namespace PRE/POST and provisional results.

Instead, the batch captures all helper files at once and raises OSError: [Errno 24] Too many open files: 'source_directory_records.py' before completing verification. Increasing descriptor limits would leave the growth problem in place.

Environment and identity

The failure occurred in the Fedora x86-64 cargo CI job, global job 95544. The log does not report the actual descriptor limit or a live descriptor census.

Evidence and prior checks

The traceback enters CatalogHelperVerification._capture, then TreeAccess.open_file and the original native opener in source_descriptor_capture. This establishes descriptor exhaustion during helper acquisition; it does not establish a descriptor leak.

Tracking

Related: #725, which reduces helper body borrow overhead. The descriptor lifetime fix must preserve those original body and edge checks.

## Summary Catalog helper verification opens every selected helper before reading the batch. A sufficiently large source graph can exhaust the process descriptor allowance before any helper body is verified. ## Reproduction Run the maintained native build-script producer controls, including `test_actual_two_same_stage_libraries_keep_owned_captures`, with the workflow's selected Cargo and Rust compiler. The cargo job in [run 9968](https://git.erikinkinen.fi/erix/integration/actions/runs/9968/jobs/0) reproduces the failure at commit `0c01f1c020853a098a5b5e9de89674e4d2662268`. ## Expected and actual behavior Helper verification should use bounded live descriptor headroom as the helper roster grows, while retaining the original file proofs, complete namespace PRE/POST and provisional results. Instead, the batch captures all helper files at once and raises `OSError: [Errno 24] Too many open files: 'source_directory_records.py'` before completing verification. Increasing descriptor limits would leave the growth problem in place. ## Environment and identity The failure occurred in the Fedora x86-64 cargo CI job, global job 95544. The log does not report the actual descriptor limit or a live descriptor census. ## Evidence and prior checks The traceback enters `CatalogHelperVerification._capture`, then `TreeAccess.open_file` and the original native opener in `source_descriptor_capture`. This establishes descriptor exhaustion during helper acquisition; it does not establish a descriptor leak. ## Tracking Related: #725, which reduces helper body borrow overhead. The descriptor lifetime fix must preserve those original body and edge checks.
erikinkinen added this to the Alpha 1 milestone 2026-11-11 09:49:53 +01:00
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
erix/integration#728
No description provided.