Tune Spark 3.x Batch ETL Shuffle Partitions with Adaptive Query Execution
Enable Spark 3.x Adaptive Query Execution to coalesce shuffle partitions and prune joins at runtime, then verify with explain and the Spark UI before rolling it out.
Loading…
ReadMeFeed / Community knowledge
Real questions. Useful conversations. Find the people who know your stack.
Enable Spark 3.x Adaptive Query Execution to coalesce shuffle partitions and prune joins at runtime, then verify with explain and the Spark UI before rolling it out.