StAX Streaming Parser and DOM Tree Builder Interoperability: Memory Footprint at Integration Boundary
0 reputation · 04 Feb 2021, 14:52 UTC
0 reputation · 04 Feb 2021, 14:52 UTC
StAX provides a low‑memory, event‑driven API while DOM builds a complete in‑memory tree, creating a mismatch when the two are combined at an integration point.
Goal: design a pattern that allows StAX to supply parsed fragments to a DOM builder without causing unbounded memory growth, especially for documents exceeding 100 MB.
Uncertainty remains about whether the DOM implementation can accept incremental node insertion and how character‑encoding handling interacts with the streaming cursor, given JVM‑specific memory management and GC tuning. How can StAX be used to feed DOM construction while keeping peak heap usage bounded? Is there an API to limit DOM node creation to a subset of the document during streaming? Which JVM GC configurations reduce OutOfMemoryError risk when both parsers operate concurrently?
35775 reputation · 04 Feb 2021, 19:35 UTC
StAX supplies events on demand while DOM retains every node until removed; the integration boundary risks unbounded heap growth if every parsed element is converted to a DOM node without discarding processed subtrees. No standard DOM API limits node creation during streaming—the filter must live in the StAX layer. Whether peak heap stays bounded depends on the size and retention pattern of the DOM subset you need to keep in memory.
When code converts every START_ELEMENT into a DOM node and retains references, the tree grows proportionally to the full document size, regardless of the streaming nature of StAX.
XMLStreamReader to iterate events and test each START_ELEMENT against the exact subset of elements you must retain as DOM.DocumentFragment, copy attributes and text content, process the fragment immediately, then remove it from its parent and null all references before advancing the cursor.-Xmx sized for the expected fragment set plus parser buffers, not for the full 100 MB document.JVM GC configurations that have been observed to reduce OutOfMemoryError risk in similar concurrent parsing scenarios include using G1GC or ZGC, and tuning -XX:InitiatingHeapOccupancyPercent to trigger concurrent marking earlier. Exact pause and throughput characteristics vary by JVM vendor and workload.
Missing diagnostic detail: What is the expected size and retention pattern of the DOM subset you need to keep in memory? If you must retain most of the document as DOM, no streaming hybrid can keep peak heap below the DOM tree size, and a different architecture is required.
Use comments to ask for clarification. Post a solution as an answer.
No question comments on this page.