AWS Glue 6.0: Apache Iceberg v3 and a 30% Price Cut — What Changes for Data Teams
Cloud Platform EngineeringAWS just made two arguments for upgrading your Glue jobs simultaneously — one financial, one technical.
What Happened
AWS Glue 6.0 is now generally available across all supported regions, bringing three things at once: a 30% reduction in hourly billing for ETL jobs and crawlers, a runtime modernization to Apache Spark 4.1, Python 3.13, and Scala 2.13, and full support for Apache Iceberg v3 through version 1.11.0. No API changes are required for the upgrade — teams can switch by setting --glue-version to 6.0.
The pricing reduction applies to billed-by-the-second job execution, not Data Catalog storage. For data engineering teams running significant Glue workloads, this is a same-day FinOps win requiring no code changes.
Why It Matters
The Iceberg v3 additions are where the release has longer legs. Four capabilities stand out:
VARIANT with shredding enables querying JSON, event logs, and semi-structured data without schema-flattening preprocessing. Instead of copying repeated fields into new columns, shredding optimizes access paths internally — relevant for any team processing high-cardinality event streams or API logs.
Geometry and Geography types bring native spatial processing into Glue ETL. Until now, spatial analytics required preprocessing outside Glue or custom UDFs. Teams building geospatial pipelines for logistics, retail location data, or satellite-derived signals now have a native path.
Nanosecond-precision timestamps matter for IoT telemetry and high-frequency financial workloads where microsecond-level ordering affects correctness. The previous microsecond ceiling silently degraded data in edge cases.
UNKNOWN type handling is the quietest but most operational addition: pipelines no longer need to be hardened against unexpected schema fields. The runtime manages them without failure, which reduces the toil of maintaining schema contracts in environments where upstream producers change their payloads.
The runtime modernization (Spark 4.1, Python 3.13) eliminates the serialization overhead from Python UDFs through Arrow-native execution — a meaningful throughput gain for PySpark workloads, especially at scale.
The So What
The migration path is unusually low-friction for a major version bump: change one parameter, capture the 30% cost reduction, and get Iceberg v3 and the new runtime. Teams that have been waiting for a forcing function to upgrade their Glue version now have one. Start with a non-critical job, validate outputs, and roll forward. The VARIANT and spatial types will matter most to teams actively building new pipelines — existing workloads get the price benefit and runtime improvement without schema changes.
Source: AWS News Blog
Content created with AI assistance and reviewed for accuracy.
Join the conversation
Stack Insiders is our free community for readers who want to go deeper — share resources, ask questions, and connect with others across every vertical we cover.
Join Stack Insiders →