Here is step by step guide to creating Pipeline in Azure Databricks.
Define a Notebook Task You Want To Run
To start creating a pipeline in Databricks, you define the tasks you want to include in your pipeline. These tasks will typically be notebooks that contain the code you want to execute. For example, you can create a new Python 3 notebook and write the code you provided in your question.