Lecture
3.11 How to use NVIDIA TAO Models that run on Orin to work on Embedded ARM based MCU
- 14.11.2024 at 14:20 - 14:40
- Future Control Stage (C5.259)Artificial Intelligence & ML
- Language: English
- Type: Lecture
Lecture description
Nvidia stands as the undisputed leader in its domain, pioneering groundbreaking advancements in graphics processing units (GPUs) and increasingly dominating the AI and
machine learning sectors. Renowned for its cutting-edge technologies, Nvidia continuously pushes the boundaries of what’s possible in visual computing, parallel processing, and artificial
intelligence. Its GPUs are the backbone of high-performance computing and modern AI.
GPUs have historically been the preferred hardware for deploying models for inference due to their parallel processing capabilities, which excel at handling the computational demands
of deep learning algorithms. Nvidia’s Model Zoo, a repository of pre-trained deep learning models, has primarily catered to this paradigm, offering a wealth of models optimized for
deployment on GPUs.
However, with the emergence of edge computing and the growing demand for deploying models on resource-constrained devices such as microcontrollers (MCUs), there has been a
shift towards enabling inference on these platforms as well. Nvidia has addressed this demand through its TAO (Train, Adapt, Optimize) Toolkit AutoML and MLOps environment, which
facilitates the adaptation and optimization of deep learning models for deployment on a variety of hardware, including MCUs.
Edge Impulse, the leading MLOps platform for EdgeML, enhances Nvidia’s TAO Toolkit by streamlining the development and deployment of machine learning models on resource constrained
devices. Through an end-to-end workflow, Edge Impulse integrates seamlessly with TAO, allowing users to import pre-trained models or train custom ones, and optimize
them for deployment on MCUs. Additionally, Edge Impulse offers the capability to fine-tune models from TAO before deployment, further refining and customizing them to suit the
constraints of embedded platforms.
machine learning sectors. Renowned for its cutting-edge technologies, Nvidia continuously pushes the boundaries of what’s possible in visual computing, parallel processing, and artificial
intelligence. Its GPUs are the backbone of high-performance computing and modern AI.
GPUs have historically been the preferred hardware for deploying models for inference due to their parallel processing capabilities, which excel at handling the computational demands
of deep learning algorithms. Nvidia’s Model Zoo, a repository of pre-trained deep learning models, has primarily catered to this paradigm, offering a wealth of models optimized for
deployment on GPUs.
However, with the emergence of edge computing and the growing demand for deploying models on resource-constrained devices such as microcontrollers (MCUs), there has been a
shift towards enabling inference on these platforms as well. Nvidia has addressed this demand through its TAO (Train, Adapt, Optimize) Toolkit AutoML and MLOps environment, which
facilitates the adaptation and optimization of deep learning models for deployment on a variety of hardware, including MCUs.
Edge Impulse, the leading MLOps platform for EdgeML, enhances Nvidia’s TAO Toolkit by streamlining the development and deployment of machine learning models on resource constrained
devices. Through an end-to-end workflow, Edge Impulse integrates seamlessly with TAO, allowing users to import pre-trained models or train custom ones, and optimize
them for deployment on MCUs. Additionally, Edge Impulse offers the capability to fine-tune models from TAO before deployment, further refining and customizing them to suit the
constraints of embedded platforms.
Further lectures
- 10:00 - 10:203.1 Demystifying the Open Source Graphics Stack on Linux: From Kernel to User Space with a Focus on Embedded Systems
- 10:20 - 10:403.2 Building and maintaining embedded Linux distributions with Yocto and Mender
- 10:40 - 11:003.3 Embedded Linux: Armbian or Yocto?
- 11:00 - 11:203.4 Getting Up to Speed with RTOS-based Firmware Development
- 11:20 - 11:403.5 Managing compliance: Open Source license obligations checklists
- 11:40 - 12:003.6 Meeting the cybersecurity needs of embedded systems
- 12:20 - 12:403.8 Mesa3D Unveiled: From glDrawArrays(…) to GPU Magic
- 13:40 - 14:003.9 Enhancing interoperability and performance for low-power Wi-Fi 6/6E IoT devices
- 14:00 - 14:203.10 Embedded-AI in a nutshell - process, use cases and a look in to the future
- 14:20 - 14:403.11 How to use NVIDIA TAO Models that run on Orin to work on Embedded ARM based MCU