Choosing Between ShotGrid Event Daemon and Webhooks for Pipeline Automation

The Problem: Triggering External Pipelines on ShotGrid Changes

Studios that push asset updates, version publishes, or status changes into downstream tools (render farms, review systems, custom databases) need a reliable way to react to ShotGrid entity mutations. Two mechanisms exist today: the self‑hosted Event Daemon (legacy, plugin‑based) and the cloud‑native Webhooks system. Both deliver “something changed” notifications, but they differ in deployment model, delivery guarantees, and operational overhead. This post compares them side‑by‑so you can decide which fits your pipeline — or how to run both during a migration.

Event Daemon: Self‑Hosted Plugin Model

• Runtime: A long‑lived Python process you manage on studio infrastructure (Linux/macOS/Windows).
• Plugin API: Each plugin receives a raw event dictionary (the same structure returned by the ShotGrid REST API) and acknowledges via a local queue.
• Event coverage: All entity types, custom entities, and even custom‑event scripts are supported.
• Authentication: Plugins use a script‑user API key stored on the host (typically shotgun_api3.Shotgun with script_name/api_key).
• Operational burden: You own uptime, log rotation, Python version upgrades, and compatibility with ShotGrid site updates.

Because the daemon runs locally, plugins can introspect the full site schema at runtime and issue additional shotgun_api3 calls without extra network hops. However, a slow plugin blocks the daemon’s internal queue, delaying every subsequent event.

Webhooks: Cloud‑Native HTTP Callbacks

• Configuration: Created in Site Preferences → Webhooks or via the REST API (POST /api/v1/webhooks).
• Delivery: HTTPS POST with a JSON payload (max 256 KB) to a URL you control. Payload includes only changed fields plus minimal context.
• Security: Each webhook gets a one‑time HMAC‑SHA256 secret (shown once at creation). Your endpoint verifies the X‑Shotgrid‑Signature header using that secret — no long‑lived credentials stored.
• Retry policy: At‑least‑once delivery with exponential backoff up to 24 hours. Endpoint must respond <10 s or the attempt is retried.
• Event coverage: Subset of entity create/update/delete, status changes, note/version events. Custom non‑project entities and some legacy triggers (e.g., TaskTemplate application) are not yet supported.

Webhooks shift operational responsibility to ShotGrid (TLS 1.2+, managed retries) but require your handler to be stateless and fast — heavy work should be offloaded to a message bus (RabbitMQ, SQS, Kafka) and the HTTP handler should return 202 Accepted quickly.

Worked Example: Registering a Webhook and Verifying Signatures

The following curl command creates a webhook that fires on Version create and update events. Run it from a workstation that holds a script‑user with Admin permission (or a user with webhook_admin scope).

curl -X POST "https://YOUR_SITE.shotgrid.autodesk.com/api/v1/webhooks" \
  -H "Authorization: Bearer YOUR_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "version-pipeline-trigger",
    "url": "https://pipeline.example.com/shotgrid/webhook",
    "event_types": ["Version_New", "Version_Change"],
    "secret": "REPLACE_WITH_GENERATED_SECRET"
  }'

• Placeholders: YOUR_SITE, YOUR_API_TOKEN (short‑lived OAuth token), REPLACE_WITH_GENERATED_SECRET (you can omit secret to let ShotGrid generate one).
• Expected check: Response 201 Created with a JSON body containing id, secret (only shown once), and status: "active".
• Risk: The secret is displayed only once — store it immediately in a secrets manager. Losing it forces you to delete and recreate the webhook.

To test locally without exposing a public endpoint, start ngrok http 8000 and use the generated https://xxxx.ngrok.io/shotgrid/webhook as the url. In your handler (example in Python/Flask):

import hmac, hashlib, json
from flask import Flask, request, abort

app = Flask(__name__)
WEBHOOK_SECRET = b"YOUR_STORED_SECRET"

@app.route("/shotgrid/webhook", methods=["POST"])
def shotgrid_webhook():
    signature = request.headers.get("X-Shotgrid-Signature")
    if not signature:
        abort(400, "Missing signature")
    expected = hmac.new(WEBHOOK_SECRET, request.get_data(), hashlib.sha256).hexdigest()
    if not hmac.compare_digest(expected, signature):
        abort(401, "Invalid signature")
    payload = request.get_json()
    # Push to message bus, return fast
    queue.push(payload)
    return "", 202

if __name__ == "__main__":
    app.run(port=8000)

Run the Flask app, then hit the ShotGrid “Test Webhook” button (Site Preferences → Webhooks → your webhook → Test). You should see a synthetic payload arrive and a 202 response in ngrok’s inspector.

Trade‑offs and Migration Checklist

Factor	Event Daemon	Webhooks
Event‑type coverage	All entities, custom events	Subset; verify developer portal matrix before relying on a trigger
Payload richness	Full event dict (can query relationships)	Changed fields only (≤256 KB); often need a follow‑up REST call for full data
Authentication	Script‑user API key on host	HMAC secret (stateless verification)
Operational ownership	Self‑hosted process, upgrades, logs	Managed by ShotGrid; you only run the HTTP endpoint
Secret rotation	Update script‑user key in config	Create new webhook with new secret; no in‑place rotation
Rate‑limit handling	Plugin controls throughput; can block queue	Must respond <10 s; retries up to 24 h

Limitation: If your pipeline depends on CustomNonProjectEntity01 create events or TaskTemplate application, Webhooks cannot capture them today. You would need to keep the daemon for those triggers or poll via the REST API.

Practical verification: After configuring a webhook, capture a real delivery with webhook.site or ngrok and confirm the X-Shotgrid-Signature header matches your secret. For the daemon, tail its log (/var/log/shotgun_event_daemon.log) and verify plugin process_event runs without exceptions.

Next Steps

1. Audit event types — List every ShotGrid trigger your pipelines consume. Cross‑reference with the current Webhook event‑type matrix.
2. Run both in parallel — Keep the daemon active while you add webhooks for the supported events. This gives a safety net and lets you compare latency.
3. Offload heavy work — Whether daemon or webhook, push long‑running tasks (transcoding, render submission) to a durable queue and return quickly.
4. Plan secret rotation — Script a zero‑downtime swap: create new webhook, update DNS/load‑balancer to new endpoint, then delete old webhook after confirming deliveries.
5. Document fallback — For any unsupported event, keep a minimal daemon plugin or a scheduled REST poll as a fallback.

By matching each pipeline trigger to the mechanism that covers it with the least operational friction, you reduce single points of failure and keep the automation layer maintainable as ShotGrid evolves.