An Introduction to S3 Data Lake for SQL Server 2022

Chris Adkin

SQL Server 2022 ushers in data virtualization for one of the world’s most popular object storage formats in the form of S3, thus creating new opportunities to use SQL Server in conjunction with Data Lake. But what is S3? How can you leverage S3 in popular data engineering languages such as Python? What options are their for building data pipelines that leverage S3? This session aims to address all of these questions with answers which will include:

– An S3 101 style primer
– Leveraging the Boto3 package in Python for S3
– Useful tools for manipulating data in S3 such as s5cmd and cyberduck
– Building out elements of a data pipeline in Docker containers

. . . and much more!