Using Azure Cosmos DB Change Feed for Event‑Driven Processing: An Architecture Note

Problem Statement

Many services need to react to data modifications in Cosmos DB—e.g., updating caches, sending notifications, or triggering downstream workflows. Azure Cosmos DB’s Change Feed exposes a continuously updated list of inserts and updates, but only for partitioned containers. The challenge is to build a reliable, secure, and scalable pipeline that consumes this feed without missing events or violating data boundaries.

Requirements

• Container must be partitioned (a partition key is mandatory).
• Change Feed must be enabled on the container.
• Consumer (Azure Function, Service Bus, or custom app) must have read access to the container.
• Network isolation: VNet integration or Private Link for Cosmos DB.
• Retry logic for 429 (throttling) and transient failures.
• Fine‑grained RBAC: manage identity with Cosmos DB Account Reader or custom role.
• Monitoring of Cosmos DB metrics (Request Units, Change Feed lag) and consumer logs.

Minimal Suitable Design

For a lightweight, event‑driven use case, the smallest architecture is:

• Cosmos DB Account – single region, provisioned throughput.
• Database – one database.
• Container – partition key /id, Change Feed enabled.
• Azure Function – consumption plan, triggered by the Change Feed Processor library.
• Managed identity for the function with read‑only access.

Enabling Change Feed

# Run in Azure Cloud Shell (bash) with the subscription set
az cosmosdb update \
  --name <cosmos-account> \
  --resource-group <rg> \
  --enable-change-feed true

# Enable continuous feed on the container
az cosmosdb collection update \
  --account-name <cosmos-account> \
  --resource-group <rg> \
  --database-name <db> \
  --name <container> \
  --enable-change-feed true

Check the portal: Change Feed toggle should be On in the container settings.

Azure Function Trigger

Function.json example (JSON format):

{
  "bindings": [
    {
      "name": "input",
      "type": "cosmosDBTrigger",
      "direction": "in",
      "leaseCollectionName": "leases",
      "leaseCollectionConnectionStringSetting": "CosmosDBConnectionString",
      "connectionStringSetting": "CosmosDBConnectionString",
      "databaseName": "<db>",
      "collectionName": "<container>",
      "createLeaseCollectionIfNotExists": true
    }
  ]
}

In run.csx (C#), iterate over input and process each document. Implement exponential back‑off for 429 responses.

Trust & Data Boundaries

• Network Isolation: Create a private endpoint for Cosmos DB and place the Function App in the same VNet.
• Identity: Assign a managed identity to the Function App. Grant it the Cosmos DB Account Reader role scoped to the container.
• Least Privilege: Do not use Cosmos DB Account Contributor; limit to read‑only on the container.
• Secrets: Store connection strings in Azure Key Vault and reference them via ConnectionStringSetting.

Operational Checks

• Cosmos DB Metrics – Total Request Units, Change Feed Lag (in seconds). Use Azure Monitor or Log Analytics.
• Function Logs – verify Change Feed triggered events. Check for 429 or Timeout errors.
• Dead‑Letter Queue – configure a Service Bus queue or storage blob for events that repeatedly fail after retries.
• Periodic Verification – insert a test document via SDK and confirm it appears in Function logs within 30 s.

Alert Example (Azure Monitor)

Metric: Cosmos DB – Change Feed Lag
Condition: > 60 seconds
Action: Send email to ops team

Failure Modes

• Throttling (429) – can silently drop changes if the consumer does not retry. Use RetryPolicy with exponential back‑off.
• Function Crash – unprocessed events accumulate. Implement a dead‑letter queue and monitor function uptime.
• Partition Key Change – after enabling Change Feed, changing the partition key invalidates the feed. Plan migration carefully.
• High Throughput – if RU/s > ~10,000, the single consumer may lag. Consider scaling out or using a dedicated Change Feed Processor host cluster.

When to Re‑Design

Modify the architecture if any of the following conditions arise:

• Throughput > 10,000 RU/s or sustained peak > 20,000 RU/s.
• Need ordered processing across all partitions.
• Cross‑region writes are required for latency or compliance.
• Event volume exceeds the consumption plan’s burst capacity; switch to Premium plan.
• The consumer must maintain state across restarts; add a durable lease collection in a separate account.

In such cases, add multiple partitions (increase partition key cardinality), deploy a dedicated Change Feed Processor host cluster (e.g., a Kubernetes pod set), or enable multi‑region writes and region‑specific consumers.

Concrete Example: End‑to‑End Setup

1. Create Cosmos DB account (single region):

   az cosmosdb create \
     --name <cosmos-account> \
     --resource-group <rg> \
     --kind GlobalDocumentDB \
     --default-consistency-level Session \
     --enable-change-feed true
   

2. Create database and container:

   az cosmosdb sql database create \
     --account-name <cosmos-account> \
     --resource-group <rg> \
     --name <db>
   
   az cosmosdb sql container create \
     --account-name <cosmos-account> \
     --resource-group <rg> \
     --database-name <db> \
     --name <container> \
     --partition-key-path "/id" \
     --throughput 400
   

3. Enable Change Feed on container (already enabled during creation, confirm):

   az cosmosdb sql container update \
     --account-name <cosmos-account> \
     --resource-group <rg> \
     --database-name <db> \
     --name <container> \
     --enable-change-feed true
   

4. Set up Azure Function:
   • Create a Function App in the same VNet.
   • Assign a system‑assigned managed identity.
   • Grant the identity Cosmos DB Account Reader scoped to the container.
   • Store the Cosmos DB connection string in Key Vault and reference it.
   • Deploy the trigger code as shown above.
5. Test:
   • Insert a document via SDK:

     var client = new CosmosClient(connectionString);
     var container = client.GetContainer("<db>", "<container>");
     await container.CreateItemAsync(new { id = "test1", value = 42 });
     

   • Verify the Function logs a trigger within 30 s.

Monitoring & Alerts

• Cosmos DB – monitor Change Feed Lag and Request Units via Azure Monitor.
• Function App – use Application Insights for execution counts, failures, and latency.
• Set alerts for 429 responses, high lag, or function failures.

Limitations & Practical Checks

• The Change Feed works only on partitioned containers; non‑partitioned containers are unsupported.
• Throttling can silently drop events if the consumer does not handle 429. Always implement retry policies.
• Changing the container’s partition key after enabling Change Feed invalidates the feed; migration is required.
• Consumption plan may incur unpredictable costs under high event volumes; consider Premium plan for steady performance.
• Verify the feed is catching up by comparing Change Feed Lag to Request Units and ensuring no missed events in the dead‑letter queue.

Conclusion

By enabling Change Feed on a partitioned container, securing access with RBAC and VNet isolation, and consuming events via an Azure Function with robust retry logic, you can build a minimal yet reliable event‑driven pipeline. Monitor key metrics to detect throttling or lag, and be prepared to scale or redesign if throughput or ordering requirements grow. This architecture keeps operational complexity low while ensuring data integrity and compliance with trust boundaries.