[BUG] Host CI jobs remain in setup before checkout #180

Open
opened 2026-10-23 20:59:26 +02:00 by erikinkinen · 2 comments
Owner

Summary

Both host jobs in Integration CI 2960 have remained in setup for over 48 minutes.
Neither has reached checkout or executed a repository test, and later runs are
waiting. This prevents checking the published cancellation repair.

Reproduction

Open jobs host_zero and host_one in
CI 2960,
revision 5eed43c1c0931740f1fe7719eaacdffa3a261445.
At 18:55 UTC on 23 October, both show only “Set up job” running; every later step
is waiting. The web interface's live setup-log query returns no lines.

Expected and actual behavior

The runner should complete setup or report a concrete failure, then start
checkout and the host tests. Both tasks instead remain running without an
uploaded setup diagnostic. This is a setup stall; it is not a host-test timeout
or a reported test failure. The underlying cause is still unknown.

Environment and identity

Forgejo runner label docker; both jobs select rust:latest.
The tasks started at 18:07:18 and 18:07:19 UTC. The same run's Rust and Markdown
jobs passed. Previous CI 2957
reached both host suites; its separate assertion failure is tracked in #179.

Evidence and prior checks

The job API reports both tasks running. The actual web state confirms that
checkout has not started, and both live setup-log responses at 18:55:57 UTC are
complete HTTP 200 responses with an empty line list and cursor zero.
No Docker, image, network or storage cause has been established. No cancellation,
rerun or runner restart has been attempted for this incident.

Tracking

Related: #175, #179 and PR 12.

  • Identify the exact stalled stage and retain its actual live state.
  • Establish the runner/task delivery or setup failure and repair its cause.
  • Observe setup completion and actual host execution on a corrected run.
## Summary Both host jobs in Integration CI 2960 have remained in setup for over 48 minutes. Neither has reached checkout or executed a repository test, and later runs are waiting. This prevents checking the published cancellation repair. ## Reproduction Open jobs `host_zero` and `host_one` in [CI 2960](https://git.erikinkinen.fi/erix/integration/actions/runs/2960), revision `5eed43c1c0931740f1fe7719eaacdffa3a261445`. At 18:55 UTC on 23 October, both show only “Set up job” running; every later step is waiting. The web interface's live setup-log query returns no lines. ## Expected and actual behavior The runner should complete setup or report a concrete failure, then start checkout and the host tests. Both tasks instead remain running without an uploaded setup diagnostic. This is a setup stall; it is not a host-test timeout or a reported test failure. The underlying cause is still unknown. ## Environment and identity Forgejo runner label `docker`; both jobs select `rust:latest`. The tasks started at 18:07:18 and 18:07:19 UTC. The same run's Rust and Markdown jobs passed. Previous [CI 2957](https://git.erikinkinen.fi/erix/integration/actions/runs/2957) reached both host suites; its separate assertion failure is tracked in #179. ## Evidence and prior checks The job API reports both tasks running. The actual web state confirms that checkout has not started, and both live setup-log responses at 18:55:57 UTC are complete HTTP 200 responses with an empty line list and cursor zero. No Docker, image, network or storage cause has been established. No cancellation, rerun or runner restart has been attempted for this incident. ## Tracking Related: #175, #179 and [PR 12](https://git.erikinkinen.fi/erix/integration/pulls/12). - [x] Identify the exact stalled stage and retain its actual live state. - [ ] Establish the runner/task delivery or setup failure and repair its cause. - [ ] Observe setup completion and actual host execution on a corrected run.
Author
Owner

The runner never recorded receiving either affected task, and neither task created a container or uploaded a log. Forgejo logged a task-fetch timeout and a commit-status database request exceeding its context deadline around assignment. The same runner continues to execute other work, including Exsh CI626.

The runner’s fetch timeout is five seconds. A bounded increase and safe activation are being prepared, with recovery of the two empty original tasks kept separate. No original test result is being treated as a pass.

The runner never recorded receiving either affected task, and neither task created a container or uploaded a log. Forgejo logged a task-fetch timeout and a commit-status database request exceeding its context deadline around assignment. The same runner continues to execute other work, including Exsh CI626. The runner’s fetch timeout is five seconds. A bounded increase and safe activation are being prepared, with recovery of the two empty original tasks kept separate. No original test result is being treated as a pass.
Author
Owner

The affected jobs in run 2960 never reached checkout and uploaded no setup output. Server and runner logs show task-fetch deadlines during assignment. Their original failures are preserved.

Raised the runner's fetch timeout from 5 seconds to 30 seconds. The two running host jobs finished normally before the runner restarted; their clocks and results were preserved.

The next Integration job in run 2963 has completed setup, checkout and component cloning, and is running the helper checks. Run 2966 is still queued. Keeping this open until delivery and setup are confirmed for that head.

The affected jobs in [run 2960](https://git.erikinkinen.fi/erix/integration/actions/runs/2960) never reached checkout and uploaded no setup output. Server and runner logs show task-fetch deadlines during assignment. Their original failures are preserved. Raised the runner's fetch timeout from 5 seconds to 30 seconds. The two running host jobs finished normally before the runner restarted; their clocks and results were preserved. The next Integration job in [run 2963](https://git.erikinkinen.fi/erix/integration/actions/runs/2963) has completed setup, checkout and component cloning, and is running the helper checks. [Run 2966](https://git.erikinkinen.fi/erix/integration/actions/runs/2966) is still queued. Keeping this open until delivery and setup are confirmed for that head.
erikinkinen added this to the Alpha 1 milestone 2026-10-30 09:34:21 +01:00
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
erix/integration#180
No description provided.