ParserError: Expected X fields, line Y when using read_csv with on_bad_lines='error'

Goal: ensure that when a CSV contains occasional malformed rows, the reading step can be retried automatically without causing duplicate writes of already processed data to downstream storage.

Constraints: pandas.read_csv does not retry internally; raising a ParserError when on_bad_lines='error' forces the caller to catch the exception and decide whether to skip, fix, or re‑attempt the read. Any subsequent write (e.g., to_csv with mode='a' and header=False) must be idempotent, otherwise a retry could append the same rows again, violating exactly‑once semantics. The library offers no transactional write API, so external coordination is required.

What patterns can be used to make the write operation idempotent when retrying after a ParserError? How can one safely combine a retry loop for read_csv with append‑mode writes to avoid duplicate data? Is there a recommended way to signal success/failure between the read and write steps without introducing external state?