From the course: Essentials of MLOps with Azure: 4 Spark MLflow Models and Model Registry

Unlock this course with a free trial

Join today to access over 24,100 courses taught by industry experts.

End-to-end machine learning on Databricks

End-to-end machine learning on Databricks

- [Instructor] Let's take a look at this process of end-to-end machine learning with Databricks. It is as relatively simple as this process of first importing your data. In the case of the Vino Verde Wine Set from the wine that grows in Portugal, you would take that data, pull it into the DBFS or the Databricks file system. This allows us to work in a cluster based workflow. Next step, inside of your notebook you would go through and visualize your data, doing traditional exploratory data analysis. And now next, when you do a hyperparameter sweep this would be where you could really leverage the power of a cluster, go through and pick a bunch of different hyperparameters to select the best model. Once you've selected that best model you would then register the model with MLflow. As we've talked about earlier, you can go through and download the model, or in the case of the Databricks platform, you can also just register it…

Contents