How to route dead letter messages back to the original topic?
May 7
I have a Google Cloud Pub/Sub subscription that is using a dead-letter topic. I recently had an outage that was preventing a number of messages from being processed & they ended up in the dead-letter topic.
The outage was resolved and I'd like to easily send the contents of dead letter subscription back to the original subscription. They're all present in the queue still (I have nothing consuming the dead letter sub) so I just need to route them somewhere.
This is an admin task so I'd like it to be manually initiated, if that makes any difference. Ideally via the UI but I can't see anything there.
1 answer
Accepted answer · original discussion
May 7
You have a few options:
- Use a Dataflow pipeline to move messages from dead letter topic to your topic.
- Update existing pipeline to read from both original topic and dead letter topic based on configuration
- Create a new system that, when enabled, moves messages from one topic to another.
The right answer might depend on your system design and requirements.
In case your use case for dead letter topic always includes moving the messages back to the primary topic after a delay, you might want to use configurable exponential backoff in Cloud Pub/Sub. This feature will have general availability towards the end of Q2 2020.
0 question comments
Use comments to ask for clarification. Post a solution as an answer.
No question comments on this page.