Introduction
DVC 3.0: Elevating Experiment Tracking, Model Management, and Cloud Storage for MLOps
DVC 3.0 is here, bringing significant improvements to experiment tracking, model management, and cloud storage for MLOps. With this release, we are committed to making DVC the solid choice for your MLOps workflows. In this article, we will explore the new features and improvements that make DVC 3.0 a game-changer.
Experiment Tracking and Beyond
<h3 Experiment Tracking and Beyond
With DVC 2.0, we introduced the concept of experiments, providing a way to track experiments as hidden, lightweight Git commits, so you don’t have to separately manage your experiments and code. Now, with DVC 3.0, you can start tracking experiments from your Python script or notebook (see examples). You only need a Git repo and DVC’s Python logging library, DVCLive. You don’t need prior DVC knowledge or an existing DVC project.
Model Management
<h3 Model Management
DVC can also help you manage your entire model lifecycle inside your Git workflow, from creating the model to deploying it in any deployment system. Our ethos for model management is consistent with everything else we do – it’s all about integrating with your existing stack and tools, and empowering you to tie your workflows around GitOps principles and automation.
Cloud Experiments (Alpha Release)
<h3 Cloud Experiments (Alpha Release)
When we released DVC 2.0, we also launched the cml runner command to run continuous integration (CI) on your own cloud instances so you could automate large ML jobs. Cloud experiments build on this technology without CI, meaning less setup (you can configure directly in Studio). With the alpha release of Studio Cloud Experiments, you can run DVC experiments on your own cloud infrastructure in a few clicks, including with GPU and spot instance support.
Hyperparameter Optimization
<h3 Hyperparameter Optimization
DVC can also help you do hyperparameter optimization by integrating with other tools. You can queue an entire grid search of experiments, configure multiple complex model architectures with Hydra integration, and track your Optuna studies.
Smarter Cloud/Remote Storage
<h3 Smarter Cloud/Remote Storage
We are committed to building the best data versioning experience. This means making DVC work with your existing data stack and not trying to replace it. We have focused on working more closely with cloud storage (and non-cloud storage) by making DVC not only faster but smarter.
Faster Performance
<h3 Faster Performance
Sometimes you just need faster performance, especially for large data downloads and uploads. We have focused on improving performance where it matters most. For example, pushing data to S3 is 2.5x faster in DVC 3.0 than in early versions of DVC 2.x according to our benchmarks.
Conclusion
<h3 Conclusion
Our constant interaction with the DVC community gives us feedback on what should be improved. We heard from you that the ML landscape is already complex and you want to keep your tools simple. That’s why many of the new "features" are improvements to existing functionality, and why we are building this stack of tools to make DVC easier, more flexible, and the solid choice for your MLOps workflows.
Frequently Asked Questions
<h4 Frequently Asked Questions
Q1: What are the key features of DVC 3.0?
<h5 What are the key features of DVC 3.0?
The key features of DVC 3.0 include experiment tracking, model management, cloud experiments, hyperparameter optimization, and faster performance.
Q2: How do I start tracking experiments with DVC 3.0?
<h5 How do I start tracking experiments with DVC 3.0?
To start tracking experiments with DVC 3.0, you need a Git repo and DVC’s Python logging library, DVCLive. You can start tracking experiments from your Python script or notebook (see examples).
Q3: What is the cloud experiment feature in DVC 3.0?
<h5 What is the cloud experiment feature in DVC 3.0?
The cloud experiment feature in DVC 3.0 allows you to run DVC experiments on your own cloud infrastructure in a few clicks, including with GPU and spot instance support.
Q4: How does DVC 3.0 improve performance?
<h5 How does DVC 3.0 improve performance?
DVC 3.0 improves performance by making DVC not only faster but smarter. For example, pushing data to S3 is 2.5x faster in DVC 3.0 than in early versions of DVC 2.x according to our benchmarks.
Q5: What is the future of DVC 3.0?
<h5 What is the future of DVC 3.0?
The future of DVC 3.0 is to continue building a stack of tools that make DVC easier, more flexible, and the solid choice for your MLOps workflows. We will continue to improve performance, add new features, and integrate with other tools to make DVC the best choice for your MLOps needs.
