Applying Software Development Principles for Data Processing with Spark

Sergiy Lunyakin

In this session, we will discuss using software development principles in data processing through Apache Spark and Scala. You will learn how to apply best practices such as code maintainability, readability, and testability to your Spark-based workflows. I will provide practical examples of code structure, patterns, and snippets so you can develop efficient and scalable data processing applications using Scala for Spark. This session is perfect for data engineers and software developers who want to learn how to use Spark for data processing and apply software development principles in their workflows. Whether you are new to Scala for Spark or an experienced developer, you will gain a deeper understanding of developing high-quality, efficient, and scalable data processing applications using Scala for Spark.