There has been an increasing push to load data incrementally throughout the day or even within minutes. Apache Spark and Delta Lake are a great option to do this at a large scale. These tools also integrate well with other Azure data platform capabilities. Using Azure Databricks for this type of processing gives us the power of Apache Spark and Delta Lake, plus added benefits like auto-loader and Delta Live Tables. In this session you will learn best practices for incremental data processing and see several techniques for building these data pipelines using Azure Databricks along with Azure Event Hubs and Synapse Analytics.
Sign up to stay up to date with news, special announcements and educational content.
Redgate will only contact you about PASS Data Community Summit (in line with our Privacy Policy) unless you separately request emails about Redgate. You can unsubscribe from these updates at any time.
Summit video library