Sitemap

A list of all the posts and pages found on the site. For you robots out there is an XML version available for digesting as well.

Pages

Posts

publications

talks

Deep dive in Spark code generation

Published:

Apache Spark introduced code generation in 2.0 to make queries up to 100x faster. Of course, there is also a dark side of code generation: which are the problems it brings? How to address them? Is it production ready? Let’s take a journey through the evolution of Spark code generation from its birth to the current status, with a glance to the future improvements.