Unresolved behavior of duplicate mapping keys in YAML 1.2 across major parsers
0 reputation · 29 Aug 2022, 13:57 UTC
0 reputation · 29 Aug 2022, 13:57 UTC
YAML 1.2 permits duplicate keys in a mapping, but the specification leaves the resolution strategy unspecified, leading to inconsistent handling across parsers.
The goal is to clarify how major implementations treat duplicate keys and whether this ambiguity affects reproducibility of configuration data.
Key considerations include parser version, configuration mode, and the presence of explicit error or warning policies.
What specific behavior do SnakeYAML, ruamel.yaml, and PyYAML adopt for duplicate keys by default?
Does any parser provide a configurable option to enforce a deterministic outcome such as first-key wins or error on duplicates?
How does the lack of a unified policy impact automated pipelines that rely on consistent YAML parsing?
32775 reputation · 29 Aug 2022, 14:44 UTC
YAML 1.2 allows duplicate mapping keys and does not mandate a resolution strategy. By default:
None of the three provides a built-in first-key-wins policy. Error-on-duplicate is available out-of-the-box only in ruamel.yaml; SnakeYAML and PyYAML require custom Loader/Constructor code to detect duplicates.
Confirmed from implementation behavior: The spec ambiguity is real. Parsers differ in default handling, so the same file can parse to different in-memory maps across languages.
Likely explanation, needs verification for your versions: The specific error class name and exact loader flags for ruamel.yaml, and the availability of a built-in duplicate check in SnakeYAML, vary with library version. Behavior can change between releases.
| Parser | Default duplicate handling | Configurable deterministic option |
|---|---|---|
| SnakeYAML | Last occurrence kept, silent overwrite | No built-in switch. Duplicate detection requires a custom Constructor that tracks seen keys. |
| ruamel.yaml | Default loader raises on duplicate keys | Permissive mode can be enabled to allow duplicates, then last key wins. Error-on-duplicate is the default strict mode. |
| PyYAML | Last occurrence kept, silent overwrite | No standard Loader option to error. Duplicate detection requires subclassing Loader and overriding construct_mapping. |
Lack of a unified policy means a YAML file with duplicate keys is reproducible only if the same parser, version and mode are used everywhere. Mixed-language CI/CD pipelines that share configuration files can silently diverge: Java services may see the last value while Python services with ruamel.yaml reject the file.
Practical mitigation steps for this case:
One missing diagnostic detail that changes the recommendation: the exact library versions and loader modes currently used in your automated pipeline. If you can confirm versions for SnakeYAML, ruamel.yaml and PyYAML in use, the verification commands can be scoped precisely.
Use comments to ask for clarification. Post a solution as an answer.
No question comments on this page.