Spark 4.0 drops Scala 2.12 support, breaking Scala binary compatibility
Spark 4.0 changes the Scala binary compatibility boundary compared with Spark 3.5. Spark 3.5 ships Scala 2.12 and 2.13 builds, while Spark 4.0 is documented as Scala 2.13 only and requires Java 17 at runtime. The goal is to determine the impact on existing codebases compiled against Spark 3.5 with Scala 2.12 and Java 8/11 toolchains. The constraints include