Azure Queue Storage: use visibility timeouts without losing track of work

A retrieved message is temporarily hidden, not completed

Queue Storage lets producers place work in a queue and consumers retrieve messages asynchronously. Retrieving a message makes it temporarily invisible to other consumers for the configured visibility interval. It does not remove the message. Successful processing is followed by the supported delete operation using the message identifier and current pop receipt.

If a worker stops before deleting the message, it can become visible again and be processed by another worker. That behavior supports recovery, but it also means the application must tolerate repeated processing. Design around at-least-once delivery rather than treating a retrieved message as exclusive ownership of a business operation forever.

Size the visibility interval for real processing

A short visibility interval can allow another worker to retrieve the message while the first one is still working. An excessively long interval delays recovery after a crash. Measure the expected processing duration, choose a suitable interval within the service limits, and use the supported message update operation when a long-running task needs its invisibility period extended.

Updating or retrieving a message changes the pop receipt used for later operations. Keep the current receipt with the active processing state. A stale receipt can cause a delete or update to fail, which should be handled as a message-state conflict rather than silently ignored. Log safe message identifiers and failure categories without copying sensitive payloads into diagnostics.

Make the business operation idempotent

1. Put a stable business-operation key in the message.
2. Validate the message before performing external effects.
3. Use a durable uniqueness constraint or transaction to apply the operation once.
4. Extend visibility when supported processing genuinely needs more time.
5. Delete the message only after the required durable result is established.

Consider a worker that commits a database change and crashes before deleting its message. The next delivery must recognize that committed result. An in-memory flag or a key based on the latest worker invocation does not solve that case. Persist the deduplication key with the result so a replacement worker can make the same decision.

Provide an explicit path for poison work

A low-level Queue Storage consumer should define how it handles repeatedly invalid or unprocessable messages. It can use the dequeue count and an application-specific policy to move them to an inspection queue or record them in durable failure state. Do not assume the raw queue automatically supplies the same poison-message handling as a particular higher-level trigger integration.

Transient downstream failures deserve bounded retries and backoff. Invalid input or a permanently missing dependency needs an actionable failure record. Separate those categories so a single bad message does not consume all worker capacity. Record the operation key, attempt history and the reason the work is waiting or requires correction.

Rehearse the crash windows

Test a worker crash before the durable operation, after the operation commits and during a visibility extension. Also run two consumers against the same redelivered business key. Check the queue depth and the age of unfinished work, not only the number of successful deletions. The worker is dependable when repeated delivery preserves the intended result and unfinished work remains discoverable.

References

• Introduction to Azure Queue Storage - Azure Storage — Microsoft Learn
• Get Messages (REST API) - Azure Storage — Microsoft Learn