Bulk API 2.0: BOM in CSV Files Causes Partial Failure Mapping Issues

The Solution

To resolve partial failure mapping issues and restore accurate line numbering in Bulk API 2.0 error logs, you must strip the Byte Order Mark (BOM) from the CSV file before uploading. Bulk API 2.0 does not provide a configuration setting to ignore the BOM; it treats the leading bytes as part of the first column header, which shifts the parser's alignment and obscures record correlation in the failed results log.

Likely Explanation

When a UTF-8 BOM (EF BB BF) is present, the API parser interprets these bytes as characters belonging to the first header field. This creates a mismatch between the expected header name and the actual bytes received. Consequently, the parser may fail to map the first column correctly or offset the line count. Because the error is rooted in the file's metadata rather than a specific data row, the failedResults log often fails to provide a reliable line offset, as the internal pointer for the first record is corrupted.

Implementation Steps

Since this behavior cannot be overridden via API settings, apply one of the following pre-processing commands based on your environment:

• Linux/macOS: Use sed to remove the BOM from the first line:

  sed -i '1s/^\xEF\xBB\xBF//' data_upload.csv

• Windows (PowerShell): Read the file and trim the BOM character before saving:

  [IO.File]::ReadAllText('data_upload.csv').TrimStart("\uFEFF") | Set-Content -Encoding UTF8 data_upload.csv

Verification Process

To measure the impact and verify the fix without affecting production data, use a sandbox environment with API version 58.0 or later:

1. Create a test CSV with a header and five data rows. Intentionally introduce a type mismatch (e.g., text in a numeric field) on row 3.
2. Upload the file with the BOM and retrieve the results via GET /services/data/vXX.0/jobs/ingest/{jobId}/failedResults/. Note if the line number is missing or offset.
3. Strip the BOM using the commands above and upload the file again.
4. Verify that the failedResults log now correctly identifies row 3 as the failure point.

Diagnostic Requirement

To refine this recommendation, please specify the line ending format (CRLF vs LF) used in your source files, as multi-byte character handling can occasionally interact with line-break parsing to produce varying offsets.