[BUG] Missing job logs prevent CI failure diagnosis #116

Open
opened 2026-10-14 03:54:35 +02:00 by erikinkinen · 5 comments
Owner

Summary

The job output for SHA2 workflow run 40 is unavailable through the supported log routes. Both jobs are marked failed, but their causes and warning output cannot be inspected.

Reproduction

Open the linked run and download the test and Markdown job logs. Download the complete run-log archive and inspect its entries. Query the run's artifact list as a separate evidence route. These read-only requests reproduce the missing output without rerunning the workflow.

Expected and actual behavior

The log routes should return the original output or an explicit retention state. Both job routes return HTTP 500 because their backing log files do not exist. The archive route returns HTTP 200 with two .MISSING entries reporting the same absent files. The artifact list is empty.

The observations do not establish whether the files were never stored or removed later, or why either job failed.

Environment and identity

  • Repository: erix/lib-sha2, commit 721e72b0bc02590da8f1e4921be4ce251aa07861, branch feature/posix-compat.
  • Original push workflow: run 40, test and Markdown jobs, attempt 1.
  • The selected workflow declares the docker runner with rust:latest and davidanson/markdownlint-cli2. Actual command execution is unavailable in the missing logs.
  • Read-only observations: 13 and 14 October 2026. Server storage and retention settings were not independently inspected.

Evidence and prior checks

The deployed API schema documents the job-log, run-log archive and artifact-list routes. The responses and the two archive entries have been preserved. The original workflow remains failed; compiler, test and warning outcomes are unavailable.

Tracking

CI workflow and monitoring, phase completion. Integration #113 tracks descriptor-query compatibility separately; the missing SHA2 output leaves its failure cause unknown.

## Summary The job output for [SHA2 workflow run 40](https://git.erikinkinen.fi/erix/lib-sha2/actions/runs/40) is unavailable through the supported log routes. Both jobs are marked failed, but their causes and warning output cannot be inspected. ## Reproduction Open the linked run and download the test and Markdown job logs. Download the complete run-log archive and inspect its entries. Query the run's artifact list as a separate evidence route. These read-only requests reproduce the missing output without rerunning the workflow. ## Expected and actual behavior The log routes should return the original output or an explicit retention state. Both job routes return HTTP 500 because their backing log files do not exist. The archive route returns HTTP 200 with two `.MISSING` entries reporting the same absent files. The artifact list is empty. The observations do not establish whether the files were never stored or removed later, or why either job failed. ## Environment and identity - Repository: `erix/lib-sha2`, commit `721e72b0bc02590da8f1e4921be4ce251aa07861`, branch `feature/posix-compat`. - Original push workflow: [run 40](https://git.erikinkinen.fi/erix/lib-sha2/actions/runs/40), test and Markdown jobs, attempt 1. - The selected workflow declares the `docker` runner with `rust:latest` and `davidanson/markdownlint-cli2`. Actual command execution is unavailable in the missing logs. - Read-only observations: 13 and 14 October 2026. Server storage and retention settings were not independently inspected. ## Evidence and prior checks The [deployed API schema](https://git.erikinkinen.fi/api/swagger) documents the job-log, run-log archive and artifact-list routes. The responses and the two archive entries have been preserved. The original workflow remains failed; compiler, test and warning outcomes are unavailable. ## Tracking [CI workflow and monitoring](https://git.erikinkinen.fi/erix/integration/issues/3), [phase completion](https://git.erikinkinen.fi/erix/integration/issues/65). [Integration #113](https://git.erikinkinen.fi/erix/integration/issues/113) tracks descriptor-query compatibility separately; the missing SHA2 output leaves its failure cause unknown.
Author
Owner

The original LibInput 205, dynlinkd 228, loaderd 284, posixd 318, timed 233 jobs also have missing logs. Their pages show roughly an hour in setup, then zero-second failures for all later steps and an empty step log. This does not identify a compiler or test failure.

A separate LibInput run 207 has reached checkout and pinned dependency preparation. The setup cause remains unknown, and the original failed runs are preserved.

The original [LibInput 205](https://git.erikinkinen.fi/erix/lib-input/actions/runs/205/jobs/0/attempt/1), [dynlinkd 228](https://git.erikinkinen.fi/erix/dynlinkd/actions/runs/228/jobs/0/attempt/1), [loaderd 284](https://git.erikinkinen.fi/erix/loaderd/actions/runs/284/jobs/1/attempt/1), [posixd 318](https://git.erikinkinen.fi/erix/posixd/actions/runs/318/jobs/1/attempt/1), [timed 233](https://git.erikinkinen.fi/erix/timed/actions/runs/233/jobs/1/attempt/1) jobs also have missing logs. Their pages show roughly an hour in setup, then zero-second failures for all later steps and an empty step log. This does not identify a compiler or test failure. A separate [LibInput run 207](https://git.erikinkinen.fi/erix/lib-input/actions/runs/207) has reached checkout and pinned dependency preparation. The setup cause remains unknown, and the original failed runs are preserved.
Author
Owner

Docs run 1624 has the same setup-only failure pattern: setup takes 1h0m4s, then checkout and Markdown report failure at zero seconds. The log API returns the missing-storage error; no source diagnostic is available. The manual job succeeds. I retained that attempt and started one distinct workflow on the unchanged 8df2fc04 source.

Docs [run 1624](https://git.erikinkinen.fi/erix/docs/actions/runs/1624) has the same setup-only failure pattern: setup takes 1h0m4s, then checkout and Markdown report failure at zero seconds. The log API returns the missing-storage error; no source diagnostic is available. The manual job succeeds. I retained that attempt and started one distinct workflow on the unchanged `8df2fc04` source.
Author
Owner

The setup stall is still present on Posixd 94e8cb8f (run 321). At 04:44 UTC, both jobs had been in “Set up job” since 03:49; checkout and every later step remained unstarted. The Rust log request returned HTTP 500. Integration 63bb744d (run 2669) is waiting. The runner API lists runner 2 as active but exposes no heartbeat or container-health information, so that flag does not establish recovery.

The setup stall is still present on Posixd `94e8cb8f` ([run 321](https://git.erikinkinen.fi/erix/posixd/actions/runs/321)). At 04:44 UTC, both jobs had been in “Set up job” since 03:49; checkout and every later step remained unstarted. The Rust log request returned HTTP 500. Integration `63bb744d` ([run 2669](https://git.erikinkinen.fi/erix/integration/actions/runs/2669)) is waiting. The runner API lists runner 2 as active but exposes no heartbeat or container-health information, so that flag does not establish recovery.
Author
Owner

Posixd’s original run 321 failed before checkout after about an hour in setup. Its task-log requests returned HTTP 500.

I dispatched one fresh run on the same 94e8cb8f revision after confirming no attempt was active. Forgejo accepted it as run 323. At 05:16 UTC, both jobs had spent about seven minutes in setup, with checkout and later steps still waiting; the Rust task log again returned HTTP 500. No source or test failure is established.

Bootloader’s run 176 completed using the same configured container images. The API’s active runner flag does not explain why these Posixd tasks cannot start. Runner-side logs are needed to diagnose that boundary; no further retry has been sent.

Posixd’s original [run 321](https://git.erikinkinen.fi/erix/posixd/actions/runs/321) failed before checkout after about an hour in setup. Its task-log requests returned HTTP 500. I dispatched one fresh run on the same `94e8cb8f` revision after confirming no attempt was active. Forgejo accepted it as [run 323](https://git.erikinkinen.fi/erix/posixd/actions/runs/323). At 05:16 UTC, both jobs had spent about seven minutes in setup, with checkout and later steps still waiting; the Rust task log again returned HTTP 500. No source or test failure is established. Bootloader’s [run 176](https://git.erikinkinen.fi/erix/bootloader/actions/runs/176) completed using the same configured container images. The API’s active runner flag does not explain why these Posixd tasks cannot start. Runner-side logs are needed to diagnose that boundary; no further retry has been sent.
Author
Owner

The single fresh run on Posixd 94e8cb8f also failed before source execution. Both jobs spent just over an hour in startup; checkout and later steps show zero duration, with no command logs. Both original job-log routes return HTTP 500.

The source failure remains unproven. Runner or server logs are needed before another retry. Integration’s current run is still queued.

The single fresh run on Posixd `94e8cb8f` also failed before source execution. Both jobs spent just over an hour in startup; checkout and later steps show zero duration, with no command logs. Both original job-log routes return HTTP 500. The source failure remains unproven. Runner or server logs are needed before another retry. Integration’s current run is still queued.
erikinkinen added this to the Alpha 1 milestone 2026-10-30 09:39:15 +01:00
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
erix/integration#116
No description provided.