#partition-coalescing

Spark AQE post-shuffle partition count target

Goal: Constrain the number of partitions produced after an Apache Spark Adaptive Query Execution shuffle coalescing stage in Spark 3.x. spark.sql.adaptive.coalescePartitions.enabled treats spark.sql.adaptive.advisoryPartitionSizeInBytes as a size target, not a fixed partition count. The resulting partition count depends on runtime data statistics and skew an