Starting with Machine Learning on devices like MCU - Part 13 (TensorFlow Lite)

 

We mentioned about the TensorFlow framework which is going to be the background for all the deep learning models development. We said this is like Arduino where we use the inbuilt libraries for all the hobby projects we run. But Tensor Flow is meant to be used for memory intensive devices like the processor based systems, bigger SoCs. To work on microcontroller boards, we need TensorFlow Lite. Especially, when we talk about Edge Computing, real-time AI experiences like image classification, speech recognition, and anomaly detection can be done directly on devices without requiring cloud connectivity, leading to lower latency, reduced power consumption, and improved privacy

TensorFlow Lite is a lightweight version of TensorFlow designed for deploying machine learning models on mobile, embedded, and edge devices with limited computing resources. It provides tools to convert and run TensorFlow models efficiently in such environments while maintaining acceptable accuracy and speed.

While converting the TensorFlow models to a MCU compatible model, we need

• TensorFlow Lite Converter
• TensorFlow Lite Interpreter

The TensorFlow Lite Converter is responsible for converting trained TensorFlow models into a compact format (.tflite file) that can be executed by the TensorFlow Lite runtime. During conversion, it applies several optimizations like quantization, pruning, and weight clustering to reduce model size and improve performance. The converter takes models from TensorFlow’s formats such as SavedModel or Keras and transforms them into a flat buffer format that is optimized for on-device inference. This step ensures the model runs efficiently on devices like MCU or any DSP.

The TensorFlow Lite Interpreter is the runtime component that executes the converted .tflite models on the target device. It is a lightweight, optimized inference engine designed to minimize memory usage and latency. The interpreter loads the model, allocates tensors, and executes inference based on the input data. Developers can use it in languages such as Python, C++, Java, and Swift depending on the deployment platform.

The workflow is like this,

• models are first trained using TensorFlow on high-performance hardware
• models are converted using the TensorFlow Lite Converter
• models are finally deployed on resource-constrained devices using the TensorFlow Lite Interpreter