The Battle of the Compressors: Optimizing Spark Workloads with

Hello! Hope you’re having a wonderful time working with challenging issues around Data and Data Engineering. In this article let’s look at the different compression algorithms Apache Spark offers…

Optimizing genomic data processing on Apache Spark, by Johan Nyström-Persson

Spark catalyst optimizer and query optimization, by krishnaprasad k

Easy Guide to Create a Custom Read Data Source in Apache Spark 3, by Amar Gajbhiye

Pyspark — save vs. saveToTable. A cautionary tale of side effects that…, by Ivelina Yordanova

The Battle of the Compressors: Optimizing Spark Workloads with ZStd, Snappy and More for Parquet, by Siraj

Announcing: Spark Performance Advisor, by Vladimir Prus

Article on compression techniques for Apache Spark, Sirajudeen A posted on the topic

Performance Optimization in Apache Spark, by Harun Raseed Basheer

Organize your data lake using Lighthouse, by Gergely Soti, datamindedbe

The Battle of the Compressors: Optimizing Spark Workloads with ZStd, Snappy and More for Parquet, by Siraj

Spark partitioning: full control. In this post, we'll learn how to…, by Vladimir Prus