DataFrames & SQL in Databricks: Reading, Writing, and Transforming Data This is where things get real. So far we've set up our environment, understood clusters, learned how Spark works under the hood, and connected to cloud storage. Now we actually touch data. In this article you'll learn how to read data in multiple formats, transform it using PySpark and SQL, and write results back to stor