Leveraging Azure Cosmos DB Change Feed for Real‑Time Event‑Driven Apps

Problem: Latency in Data‑Driven Apps

Modern applications—think live dashboards, real‑time analytics, or microservice event streams—need to react to database changes as soon as they happen. Traditional approaches rely on polling or custom triggers that add latency, increase cost, and complicate scaling. Developers want a low‑cost, reliable way to capture inserts, updates, and deletes across partitions and regions without reinventing the wheel.

Solution: Cosmos DB’s Built‑In Change Feed

The Change Feed is an ordered, append‑only stream of documents that have been modified in a container. It is read‑only, but it guarantees that each change is delivered once per partition. The .NET SDK offers a ChangeFeedProcessor that handles checkpointing, lease distribution, and retries, so you can focus on business logic.

Key benefits:

• Zero‑polling: the processor listens for new changes.
• Automatic partition ownership: multiple instances can run in parallel.
• Idempotent processing: checkpoints prevent re‑processing.
• Cost‑effective: no extra compute for the feed itself.

Worked Example: A Minimal C# Consumer

Below is a stripped‑down console app that registers a ChangeFeedProcessor, writes a new document, and logs the change. Replace placeholders with your own values.

using System;
using System.Threading.Tasks;
using Microsoft.Azure.Cosmos;

class Program
{
    private const string EndpointUri = "https://<YOUR_ACCOUNT>.documents.azure.com:443/";
    private const string PrimaryKey = "<YOUR_PRIMARY_KEY>";
    private const string DatabaseId = "SampleDb";
    private const string ContainerId = "SampleContainer";
    private const string LeaseContainerId = "LeaseContainer";

    static async Task Main(string[] args)
    {
        var cosmosClient = new CosmosClient(EndpointUri, PrimaryKey,
            new CosmosClientOptions { ApplicationName = "ChangeFeedDemo" });
        var database = cosmosClient.GetDatabase(DatabaseId);
        var container = database.GetContainer(ContainerId);
        var leaseContainer = database.GetContainer(LeaseContainerId);

        var processor = container.GetChangeFeedProcessorBuilder<dynamic>(
                "processorName",
                async (IReadOnlyCollection<dynamic> changes, CancellationToken cancellationToken) =>
                {
                    foreach (var doc in changes)
                    {
                        Console.WriteLine($"Change detected: ID={doc.id}");
                    }
                    await Task.CompletedTask;
                })
            .WithInstanceName("demo-instance")
            .WithLeaseContainer(leaseContainer)
            .Build();

        // Start the processor – it will run until stopped.
        await processor.StartAsync();
        Console.WriteLine("Processor started. Press any key to exit.");

        // Demo: insert a new document.
        var newDoc = new { id = Guid.NewGuid().ToString(), name = "Sample", timestamp = DateTime.UtcNow };
        await container.CreateItemAsync(newDoc, new PartitionKey(newDoc.id));

        Console.ReadKey();
        await processor.StopAsync();
    }
}

Execution flow:

1. The processor starts and acquires leases for partitions.
2. A new document is inserted.
3. The feed emits the change; the callback prints the document ID.
4. Checkpoints are written to LeaseContainer automatically.

To verify the processor is working, look for console output and confirm a lease document appears in the lease container. Azure Monitor can also surface processor metrics.

Trade‑Offs and Limitations

• Read‑Only Snapshot: The feed only contains the latest committed state. If a document is deleted, you’ll receive a tombstone entry but not the original content. For audit or rollback scenarios, maintain a separate audit log.
• Rapid Successive Updates: If a document is updated multiple times before the processor reads it, only the last version appears in the batch. Design consumers to be idempotent or store intermediate snapshots if you need every state change.
• Throughput Considerations: The change feed consumes Request Units (RUs) like any read operation. Ensure your container’s RU‑s provisioned for the expected change volume.
• Cross‑Region Replication: The feed is local to the region where the container resides. If you need global event streams, you may need to replicate changes manually or use Azure Event Grid integration.

Practical Checklist Before Production

• Enable Change Feed on the target container (Azure Portal → Container → Settings → Enable Change Feed).
• Create a dedicated lease container with adequate throughput.
• Configure the processor’s InstanceName uniquely per deployment to avoid lease conflicts.
• Implement graceful error handling and retry logic in the change handler.
• Monitor processor metrics (e.g., CosmosDB.ChangeFeedProcessor.*) via Azure Monitor.
• Plan for idempotency or state snapshots if you need to capture every intermediate change.

Conclusion

The Cosmos DB Change Feed is a powerful, low‑latency foundation for event‑driven architectures. By leveraging the SDK’s processor, you avoid the overhead of polling or custom triggers and gain built‑in scaling and reliability. Just remember its read‑only nature and the need for careful consumer design when rapid updates or deletions are involved. Once you’ve set up the feed and verified the processor, you’re ready to build real‑time dashboards, downstream analytics, or microservice event pipelines with minimal friction.