Introduction
Always-on machine learning models have become increasingly popular in recent years, thanks to their ability to continuously learn and adapt to new data. However, these models require a very low memory and compute footprint, which can be a challenge when it comes to training and deploying them. In this article, we’ll explore a technique that can help overcome this challenge by refactoring a small convolutional model into a larger redundant multi-branched architecture, and then re-parameterizing it for inference.
Overcoming the Limitations of Always-On Machine Learning Models
Always-on machine learning models require a very low memory and compute footprint. Their restricted parameter count limits the model’s capacity to learn, and the effectiveness of the usual training algorithms to find the best parameters. This can make it difficult to achieve the desired level of accuracy and performance.
A Novel Approach to Training Always-On Models
Our approach involves refactoring a small convolutional model into a larger redundant multi-branched architecture. This allows the model to learn more complex patterns and relationships in the data, which can improve its accuracy and performance. For inference, we algebraically re-parameterize the trained model into the single-branched form with fewer parameters, which reduces the memory footprint and compute cost.
Experimental Results
We tested our approach using an always-on wake-word detector model, which we call RepCNN. Our results show that RepCNN provides a good trade-off between latency and accuracy during inference. Compared to a uni-branch convolutional model, RepCNN is 43% more accurate while having the same runtime. Additionally, RepCNN meets the accuracy of complex architectures like BC-ResNet, while having 2x lesser peak memory usage and 10x faster runtime.
Conclusion
In conclusion, our approach provides a novel way to train always-on machine learning models with a low memory and compute footprint. By refactoring a small convolutional model into a larger redundant multi-branched architecture, and then re-parameterizing it for inference, we can achieve better accuracy and performance while reducing the memory footprint and compute cost. This technique has the potential to be widely applicable to a variety of always-on machine learning applications.
Frequently Asked Questions
Q: What is the main challenge in training always-on machine learning models?
The main challenge in training always-on machine learning models is their restricted parameter count, which limits the model’s capacity to learn and the effectiveness of the usual training algorithms to find the best parameters.
Q: How does the refactoring technique work?
The refactoring technique involves transforming a small convolutional model into a larger redundant multi-branched architecture, which allows the model to learn more complex patterns and relationships in the data.
Q: What are the benefits of re-parameterizing the trained model?
Re-parameterizing the trained model reduces the memory footprint and compute cost, making it more suitable for deployment on devices with limited resources.
Q: How does RepCNN compare to other always-on machine learning models?
RepCNN is 43% more accurate than a uni-branch convolutional model while having the same runtime, and meets the accuracy of complex architectures like BC-ResNet, while having 2x lesser peak memory usage and 10x faster runtime.
Q: What are the potential applications of this technique?
This technique has the potential to be widely applicable to a variety of always-on machine learning applications, such as voice assistants, smart home devices, and autonomous vehicles.
