[BUG] Forgejo HTTPS requests stall under frontend saturation #181

Open
opened 2026-10-23 23:33:07 +02:00 by erikinkinen · 0 comments
Owner

Summary

Public Forgejo HTTPS requests intermittently stall before an HTTP response,
while its local backend remains responsive. This delays issue updates and CI
checkout or artifact publication.

Reproduction

On 23 October 2026, ordinary HTTPS requests to the public API timed out during
TLS negotiation. A request from the server using the same public hostname also
stalled; direct backend version and action-metadata requests returned 200.
The failure is intermittent; a deterministic reproducer is not established.

Expected and actual behavior

New HTTPS requests should reach the backend. Failed observations contain zero
response bytes and a TLS timeout. Apache had 144–148 of 150 workers busy and
400–434 connections queued against a 511 backlog. All twelve CPUs were busy.
These observations establish saturation, without proving its complete cause.

Environment and identity

Forgejo 16, Apache 2.4.68 with event MPM, Git 2.39.5 and Linux 6.14.0-1-pve.
Integration task 36092 continued useful VM work. Most sampled active frontend
requests were blame views of a public Git mirror; no traffic intent is inferred.

Evidence and prior checks

Docs CI 1713 first built
its PDF successfully, then timed out publishing it. Its original Markdown task
never reached checkout. Original outcomes are retained separately from attempt 2.

A low-priority, single-CPU commit-graph update added changed-path Bloom data to
the busy mirror. Git exited 0; all original commit records, refs and configuration
were preserved. HTTPS subsequently responded and worker occupancy fell, but no
controlled comparison establishes causation or a complete repair. No frontend
capacity or traffic policy changed, and the active CI job was preserved.

Tracking

Related: runner delivery.

  • Retain the original HTTPS failures and direct-backend responses.
  • Complete and verify the narrowly scoped Git index maintenance.
  • Establish the remaining request-lifetime and resource bottlenecks.
  • Confirm stable fresh checkout and artifact publication through HTTPS.
## Summary Public Forgejo HTTPS requests intermittently stall before an HTTP response, while its local backend remains responsive. This delays issue updates and CI checkout or artifact publication. ## Reproduction On 23 October 2026, ordinary HTTPS requests to the public API timed out during TLS negotiation. A request from the server using the same public hostname also stalled; direct backend version and action-metadata requests returned 200. The failure is intermittent; a deterministic reproducer is not established. ## Expected and actual behavior New HTTPS requests should reach the backend. Failed observations contain zero response bytes and a TLS timeout. Apache had 144–148 of 150 workers busy and 400–434 connections queued against a 511 backlog. All twelve CPUs were busy. These observations establish saturation, without proving its complete cause. ## Environment and identity Forgejo 16, Apache 2.4.68 with event MPM, Git 2.39.5 and Linux 6.14.0-1-pve. Integration task 36092 continued useful VM work. Most sampled active frontend requests were blame views of a public Git mirror; no traffic intent is inferred. ## Evidence and prior checks [Docs CI 1713](https://git.erikinkinen.fi/erix/docs/actions/runs/1713) first built its PDF successfully, then timed out publishing it. Its original Markdown task never reached checkout. Original outcomes are retained separately from attempt 2. A low-priority, single-CPU commit-graph update added changed-path Bloom data to the busy mirror. Git exited 0; all original commit records, refs and configuration were preserved. HTTPS subsequently responded and worker occupancy fell, but no controlled comparison establishes causation or a complete repair. No frontend capacity or traffic policy changed, and the active CI job was preserved. ## Tracking Related: [runner delivery](https://git.erikinkinen.fi/erix/integration/issues/180). - [x] Retain the original HTTPS failures and direct-backend responses. - [x] Complete and verify the narrowly scoped Git index maintenance. - [ ] Establish the remaining request-lifetime and resource bottlenecks. - [ ] Confirm stable fresh checkout and artifact publication through HTTPS.
erikinkinen added this to the Alpha 1 milestone 2026-10-30 09:34:21 +01:00
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
erix/integration#181
No description provided.