Diagnosing Failed Webhook Deliveries in Forgejo

Recognizable condition

You configured a webhook in Forgejo to notify an external service—such as a CI runner, chat bot, or deployment tool—but the delivery never arrives. In the repository's webhook settings, recent deliveries show repeated failures, timeouts, or non-2xx HTTP responses. The external system logs no inbound request, or logs an error when the request does arrive.

The useful takeaway: failed webhook deliveries in Forgejo almost always reduce to five causes—network reachability, authentication, payload size, rate limiting, or an internal worker problem. You can isolate the cause by reading Forgejo's delivery history, replicating the request from the Forgejo host, and checking the receiver's logs.

Cause and diagnostic table

Symptom	Likely cause	First diagnostic
DNS failure, connection refused, or TLS handshake error	Network or certificate issue between Forgejo and the target	Run curl from the Forgejo host
401 Unauthorized or 403 Forbidden	Missing or incorrect secret, token, or signature header	Compare headers against receiver documentation
413 Payload Too Large	JSON body exceeds the receiver's maximum request size	Check Content-Length and receiver limits
429 Too Many Requests or connection reset	Receiver is rate-limiting Forgejo	Review receiver access logs for throttle entries
Timeout, worker crash, or silent failure	Forgejo webhook executor error or resource exhaustion	Inspect Forgejo service logs and system resources

Ordered checks

1. Capture the exact failure from Forgejo

Open the repository or organization webhook settings and review the recent delivery list. Note the HTTP status code, response snippet, and timestamp. This tells you whether Forgejo reached the target at all.

Where to run: Forgejo web interface.
Permissions: Repository owner or administrator.
Risk: None; this is read-only.

2. Inspect Forgejo's logs

On the Forgejo server, check the service logs for webhook-related errors. If your deployment writes webhook events to a dedicated file, it is commonly found under a path such as /var/log/forgejo/webhook.log, but the exact location depends on your installation method and configuration.

sudo journalctl -u forgejo --since "2 hours ago" | grep -i webhook

Where to run: Forgejo host shell.
Permissions: sudo or membership in the forgejo group.
Expected result: Log lines containing the target URL, error codes, or timeout messages. Redact any tokens or secrets before sharing these logs.

3. Replicate the request from the Forgejo host

Run a manual curl from the same host and network namespace that Forgejo uses. This separates network problems from Forgejo-internal problems.

curl -v -X POST \
  -H "Content-Type: application/json" \
  -H "X-Forgejo-Event: push" \
  --data @payload.json \
  https://receiver.example.com/hook

Replace receiver.example.com/hook with your target URL and payload.json with a representative JSON body. If your webhook uses a signature or token, add the corresponding header; header names vary by Forgejo version and configuration, so verify them in the webhook settings.

Where to run: Forgejo host shell.
Permissions: Standard user with network access.
Expected result: Verbose output showing DNS resolution, TLS negotiation, and the HTTP status line. A 2xx status confirms basic reachability. A TLS or DNS error confirms a network-layer problem.
Risk: Shell history may capture secrets. Prefer reading tokens from a file or environment variable.

4. Check the receiver's logs

On the target service, look for the inbound request. Confirm whether it arrived, its size, and the response code. This distinguishes between "Forgejo never sent it" and "the receiver rejected it."

Where to run: Target service host or log aggregator.
Expected result: An access log entry showing the request method, path, status, and body size.

Fixes tied to findings

Network or TLS failure

Correct the target URL scheme and hostname. If the receiver uses an internal certificate authority, ensure Forgejo trusts that CA; the exact TLS trust configuration depends on your Forgejo version and deployment. Open any firewall egress rules blocking Forgejo, and confirm the receiver is listening on the expected port.

Authentication mismatch

Regenerate the webhook secret in Forgejo and update the receiver with the new value, or correct the token in the webhook configuration. Verify that header names and signature algorithms match what the receiver expects. A 401 or 403 in the delivery history almost always means the receiver rejected the credentials before processing the payload.

Payload too large

Forgejo sends the complete JSON payload for the selected events. If the receiver returns 413, either raise the receiver's maximum body size or reduce the event scope. Selecting fewer events—such as restricting to push only instead of all events—reduces payload size and frequency.

Rate limiting

If the receiver returns 429, increase its rate limit or allowlist Forgejo's source IP. In Forgejo's webhook settings, configure a reasonable retry window and backoff so failed deliveries do not hammer the receiver. Avoid setting retries to infinite.

Internal executor errors

If Forgejo's logs show worker timeouts, panics, or memory exhaustion, check the service's resource limits. Restarting the Forgejo service may clear a stuck worker, but repeated crashes indicate a bug or resource constraint that requires upstream investigation. Ensure the host has adequate memory and file descriptors for your webhook volume.

Escalation criteria

• Infrastructure team: DNS records, firewall rules, TLS certificates, or load balancer health checks are outside Forgejo's control.
• Receiver maintainers: The receiver logs show the request arrived but was rejected by application logic, undocumented size limits, or authentication rules you cannot change.
• Upstream Forgejo issue: You observe worker panics, deliveries that never appear in logs, or resource exhaustion that persists after restart. Include the Forgejo version, redacted log excerpts, and steps to reproduce.

Limitations and verification

Log paths, configuration keys, and header names vary across Forgejo versions and installation methods. Verify them against your deployed version's documentation. A successful curl test proves network reachability but does not guarantee Forgejo's internal worker can reach the target if it runs inside a container with a different network namespace.

To confirm a fix, trigger a real event or use the webhook test function if your Forgejo version provides one, then verify a 2xx response in both Forgejo's delivery history and the receiver's access log.