ShotGrid change notifications: pick Event Daemon over polling for low-latency pipelines

The decision

You need a Python pipeline service that reacts to new Versions, Task state changes and custom entity updates within seconds, on mixed on-prem and cloud ShotGrid sites, with limited ops headcount and script-token authentication. The decision is which supported notification mechanism to use: Event Daemon, REST polling, or webhook-style push.

Useful takeaway: for near real-time delivery with bounded API usage, Event Daemon is the supported server-side option on sites where it is enabled. Polling remains the universal fallback. General webhook push is not a public, site-wide feature for most deployments.

Constraints that shape the choice

• Latency target: seconds, not minutes.
• API cost: avoid find calls that grow with entity count per poll interval.
• Runtime: Python services, script key/secret auth, long-lived process acceptable.
• Ops: small team, needs stable reconnects and clear logs.
• Compatibility: must work on older on-prem sites and newer cloud sites.

Supported options compared

Trade-offs

Event Daemon

Minimizes latency and API quota usage and provides built-in retry and de-duplication. The daemon holds a persistent connection to the site and dispatches callbacks locally. It needs a continuously running process with network access to the ShotGrid site, secure storage of script credentials, and monitoring for reconnects. Script tokens used by the daemon have site-wide read access; rotate credentials and restrict egress.

Version sensitivity matters. Event types and subscription APIs change across ShotGrid versions. Older on-prem sites may lack certain event types or require different authentication scopes. Check the site admin console for enabled event types and Event Daemon compatibility before committing.

REST polling

Simplest to deploy and works behind restrictive firewalls. You run periodic sg.find calls filtered by last_updated and entity type. Latency is bounded by the poll interval and total find calls grow with entity volume. Polling with last_updated can miss rapid successive updates within the same window; design for idempotency and duplicate processing.

Webhook push

Where available it offloads delivery to ShotGrid. Availability varies by deployment and is typically limited to specific integrations. It introduces external endpoint reliability and security requirements.

Concrete implementation sketch

Deploy a small Python service that uses shotgun_api3 and the Event Daemon library. Run it as a non-privileged user with access to the script key and secret stored in a secrets manager.

# run on the pipeline host with network access to ShotGrid
import shotgun_api3
from event_daemon import ShotgunEventDaemon

SG_URL = 'https://<site>.shotgunstudio.com'
SG_SCRIPT_KEY = '<script_key>'
SG_SCRIPT_SECRET = '<script_secret>'

sg = shotgun_api3.Shotgun(SG_URL, SG_SCRIPT_KEY, SG_SCRIPT_SECRET)

def handle_event(event):
    # validate payload shape
    entity_type = event.get('entity_type')
    event_type = event.get('event_type')
    entity_id = event.get('entity_id')
    # enqueue work only for relevant changes
    if entity_type in ['Version', 'Task'] and event_type in ['create', 'update']:
        # enqueue to internal queue
        pass

daemon = ShotgunEventDaemon(
    shotgun=sg,
    subscriptions=[
        {'entity_type': ['Version', 'Task'], 'event_type': ['create', 'update']}
    ],
    handlers=[handle_event]
)
daemon.start()

Required permissions: a script user with read access to Version, Task and any custom entities you subscribe to. The daemon will connect using the script token; ensure the token is not exposed in logs or process args.

Risks: long-lived connection can be dropped by firewalls or proxies. Monitor daemon logs for reconnect events. Do not store entity data in the daemon process; enqueue and process elsewhere.

Validation without assuming behavior

• Create a sandbox ShotGrid project and subscribe to a narrow filter, e.g., entity_type Version in that project. Create and update a test Version and observe that the handler receives an event with matching entity_type, entity_id and event_type within seconds.
• Compare API call counts over a representative workday between a polling interval and daemon operation using site admin API usage reports.
• Review site admin console for enabled event types and Event Daemon compatibility for your ShotGrid version before committing.

Limitations: Event Daemon does not guarantee ordering across entities. Payload contains change metadata, not a full entity snapshot; you still need a read to fetch current fields if required. If the daemon is not supported on a site, fall back to polling with a reasonable interval and idempotent processing.