Embedded AI: Running TinyML on ESP32-S3 and Other Microcontrollers

By informic1

Keywords: TinyML, embedded AI, ESP32 machine learning, edge AI, MCU ML

Introduction

Running machine learning models on microcontrollers was once considered impractical—the memory, compute, and power constraints of typical MCUs seemed fundamentally incompatible with the demands of neural networks. TinyML has changed that perception. By combining model compression techniques (quantization, pruning, knowledge distillation) with runtime environments designed for kilobyte-scale memory footprints, TinyML enables on-device inference on microcontrollers costing less than $5. The ESP32-S3, with its vector instructions and dual-core architecture, has emerged as the most popular TinyML platform, but it is far from the only option. This guide provides a practical overview of TinyML deployment on microcontrollers, with deep coverage of the ESP32-S3, comparisons with alternative MCUs, and hands-on guidance for model quantization, deployment, and optimization.

What Is TinyML? Architecture and Ecosystem

TinyML refers to the field of machine learning inference on resource-constrained devices—typically microcontrollers with 32-512 KB of RAM, 128 KB-4 MB of flash storage, and power consumption in the milliwatt range. The TinyML ecosystem consists of three layers:

Model Layer

TinyML models are compressed versions of standard neural networks, reduced to fit within microcontroller memory. Common model architectures include:

Keyword Spotting (KWS): Convolutional or depthwise separable models (2-50 KB) for detecting wake words ("Hey Siri," "Alexa")
Visual Wake Words (VWW): Binary image classifiers (~100-250 KB) that detect person presence
Anomaly Detection: Autoencoder-based models (5-20 KB) for industrial predictive maintenance
Gesture Recognition: Time-series classifiers (10-50 KB) for IMU-based gesture detection

Framework Layer

Framework	Supported MCUs	Model Format	Min RAM	Min Flash	Language
TensorFlow Lite for Microcontrollers (TFLM)	ESP32-S3, RP2040, STM32, Apollo3	.tflite	16 KB	64 KB	C++
Edge Impulse	ESP32-S3, RP2040, STM32, nRF52	.tflite / custom	16 KB	64 KB	C++
NanoEdge AI	STM32	Proprietary	4 KB	16 KB	C
NNoM	ESP32, STM32, RP2040	.h (header-only)	8 KB	32 KB	C

Hardware Layer

TinyML-capable MCUs share several characteristics: sufficient SRAM for model weights and activations, flash or external PSRAM for model storage, and integer arithmetic units capable of efficient INT8 multiply-accumulate (MAC) operations. The ESP32-S3 stands out by adding vector instructions specifically designed for neural network computation.

TensorFlow Lite for Microcontrollers (TFLM) Deep Dive

TFLM is the dominant TinyML runtime, ported to over 20 MCU platforms. It is a C++ library that interprets .tflite model files—flatbuffer-serialized neural networks—and executes them using platform-optimized operator implementations.

TFLM Architecture

The TFLM runtime consists of: 1. Interpreter: Reads the .tflite flatbuffer and schedules operator execution 2. Kernel implementations: Platform-specific optimized code for each operator (Conv2D, DepthwiseConv2D, FullyConnected, Softmax, etc.) 3. Memory planner: Allocates tensor buffers in a pre-allocated arena, avoiding dynamic memory allocation (no malloc) 4. MicroOpResolver: Maps model operators to kernel implementations at compile time, stripping unused operators to minimize binary size

Memory Arena Sizing

The TFLM arena holds all tensor activations during inference. Arena size depends on model architecture and is typically 1.5-3x the largest layer's output tensor size. For a typical 20 KB INT8 keyword spotting model, the arena requires 8-15 KB of SRAM. The total memory footprint (model weights + arena + runtime overhead) must fit within available SRAM.

ESP32-S3 TFLM Integration

The ESP32-S3 TFLM port leverages Espressif's ESP-NN library, which provides assembly-optimized kernels for the ESP32-S3's vector extension instructions. Key optimizations include:

INT8 matrix multiplication using vector MAC instructions (2-4x speedup over scalar)
Padding and activation functions using vector load/store
Quantized convolution with optimized im2col transformation

Model Quantization: From Float to INT8

Quantization is the process of converting 32-bit floating-point model weights and activations to 8-bit integers (INT8), reducing model size by 4x and enabling inference on MCUs without floating-point units.

Post-Training Quantization (PTQ)

PTQ is the simplest approach: a pre-trained float model is converted to INT8 using a representative dataset to calibrate activation ranges. The TensorFlow Lite Converter performs this conversion:

# Python: Convert Keras model to INT8 TFLite
import tensorflow as tf

converter = tf.lite.TFLiteConverter.from_keras_model(model)
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.representative_dataset = representative_dataset_generator
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
converter.inference_input_type = tf.int8
converter.inference_output_type = tf.int8

tflite_model = converter.convert()

PTQ typically incurs less than 1% accuracy loss for classification tasks and 2-5% for detection tasks.

Quantization-Aware Training (QAT)

For models sensitive to quantization (e.g., small models with aggressive compression), QAT simulates INT8 quantization during training, allowing the model to adapt to quantization noise. QAT typically recovers 50-80% of the accuracy lost by PTQ.

INT8 vs INT4 Quantization

Metric	FP32 (Baseline)	INT8 (PTQ)	INT4 (PTQ)
Model Size	100%	25%	12.5%
Accuracy	100%	98-99%	92-96%
Inference Speed	1×	3-4×	5-8×
RAM Usage	100%	30-40%	15-20%
Hardware Support	All	Most MCUs	Limited (ESP32-S3 partial)

INT4 quantization is an active research area. While it halves memory usage compared to INT8, accuracy drops can be significant for complex models, and hardware support remains limited.

ESP32-S3 AI Capabilities and Practical Deployment

ESP32-S3 Hardware for AI

The ESP32-S3 is uniquely suited for TinyML among low-cost MCUs:

Specification	ESP32-S3	ESP32 (original)	STM32F4	RP2040
CPU	Xtensa LX7 dual-core @ 240 MHz	Xtensa LX6 dual-core @ 240 MHz	ARM Cortex-M4 @ 168 MHz	ARM Cortex-M0+ dual-core @ 133 MHz
SRAM	512 KB	520 KB	192 KB	264 KB
Flash	4-16 MB (QSPI)	4-16 MB (QSPI)	1-2 MB	2 MB (external)
PSRAM	Up to 8 MB (octal SPI)	Up to 4 MB	No	No
Vector Instructions	Yes (AI-optimized)	No	No	No
FPU	Yes	No	Yes	No
AI Accelerator	No (vector instructions only)	No	No	No
Price (1K qty)	~$2.50	~$2.00	~$8.00	~$1.00

The vector instructions are the ESP32-S3's key differentiator. These instructions operate on 128-bit data widths, performing 16 INT8 multiply-accumulate operations in a single cycle—effectively delivering 3.8 GOPS of INT8 compute at 240 MHz.

Practical Project 1: Keyword Spotting on ESP32-S3

A keyword spotting model detects wake words using mel-frequency cepstral coefficients (MFCCs) as input features. The typical pipeline:

Audio Capture: I2S MEMS microphone (INMP441) captures 16 kHz, 16-bit audio
Feature Extraction: 49 MFCC features computed per 30 ms window with 20 ms stride
Inference: Depthwise separable CNN (8 layers, ~18 KB INT8) classifies 12 keywords
Response: LED or GPIO output on detection

Performance on ESP32-S3: Inference time ~15 ms per window, memory footprint ~35 KB (model + arena), accuracy ~91% on the Google Speech Commands dataset.

Practical Project 2: Person Detection (Visual Wake Words)

Using an OV2640 camera module connected via DVP interface:

Image Capture: 96×96 RGB image
Preprocessing: Resize and quantize to INT8
Inference: MobileNetV2-based binary classifier (~250 KB INT8)
Output: Person present / not present

Performance on ESP32-S3 with PSRAM: Inference time ~800 ms, memory footprint ~400 KB (requires PSRAM), accuracy ~88% on the Visual Wake Words dataset.

Practical Project 3: Anomaly Detection for Industrial Vibration Monitoring

Sensor: ADXL345 accelerometer via I2C, 100 Hz sampling
Feature Extraction: 128-point FFT on 1-second vibration windows
Inference: Autoencoder-based anomaly detector (~8 KB INT8)
Output: Reconstruction error threshold for anomaly flagging

Performance on ESP32-S3: Inference time ~5 ms, memory footprint ~16 KB, detects bearing faults with 95% accuracy.

ESP32-S3 vs Other Microcontrollers for TinyML

ESP32-S3 vs RP2040

The Raspberry Pi RP2040 ($1.00) is significantly cheaper than the ESP32-S3 ($2.50) and has a strong community and excellent C/C++ SDK. However, it lacks vector instructions, FPU, and Wi-Fi/Bluetooth connectivity. For TinyML workloads, the ESP32-S3 is 2-3x faster for INT8 convolution due to vector instructions, and its integrated wireless enables over-the-air model updates.

Choose RP2040 when: Cost is the primary constraint, wireless is not needed, and model inference time is not critical (>500 ms acceptable).

ESP32-S3 vs STM32

STM32 MCUs (particularly the STM32H7 series) offer higher single-thread CPU performance (Cortex-M7 at 480 MHz), larger SRAM (up to 1 MB), and proven reliability for industrial applications. However, they cost 3-5x more than the ESP32-S3 and lack the vector instruction optimizations for AI workloads. STMicroelectronics offers the STM32Cube.AI tool, which converts neural networks to optimized C code—a different approach from TFLM's interpreter model.

Choose STM32 when: Industrial certification (IEC 61508, AEC-Q100) is required, the application needs high-speed ADCs/DACs, or the project benefits from STM32Cube.AI's ahead-of-time compilation approach.

ESP32-S3 vs nRF5340

The Nordic nRF5340 is the preferred choice for Bluetooth Low Energy (BLE) applications, with superior RF performance and lower power consumption in sleep modes. However, its Cortex-M33 at 128 MHz is significantly slower than the ESP32-S3 for AI inference, and its 512 KB flash / 128 KB RAM limits model size.

Choose nRF5340 when: BLE is the primary communication protocol, battery life is critical (<1 mA average current), and the model is small (<50 KB).

Toolchain and Development Workflow

ESP-IDF + TFLM Setup

# 1. Clone ESP-IDF
git clone --recursive https://github.com/espressif/esp-idf.git
cd esp-idf && ./install.sh esp32s3
. ./export.sh

# 2. Clone TFLM ESP32 example
git clone https://github.com/espressif/esp-tflite-micro.git
cd esp-tflite-micro/examples/person_detection

# 3. Configure target
idf.py set-target esp32s3

# 4. Build and flash
idf.py build flash monitor

Edge Impulse Workflow

Edge Impulse provides a cloud-based platform that automates the entire TinyML pipeline:

Data acquisition: Connect sensors via mobile app or upload datasets
Feature extraction: Automatic MFCC, spectral, or image feature generation
Model training: Auto-selected architecture based on target MCU and latency requirements
Deployment: Export as C++ library for ESP-IDF, Arduino, or Mbed OS

Edge Impulse is the recommended starting point for teams new to TinyML, as it handles quantization, optimization, and deployment automatically.

Limitations and Practical Considerations

Memory Ceiling

The ESP32-S3's 512 KB SRAM limits model size to approximately 300 KB for INT8 models (after accounting for TFLM arena, FreeRTOS, and application code). Models requiring more memory must use PSRAM via the octal SPI interface, which has higher latency (10-20x slower than SRAM) and increases power consumption by 30-50%.

Inference Latency

While the ESP32-S3 delivers impressive performance for its price, inference times for image-based models (person detection, object classification) are measured in hundreds of milliseconds—suitable for periodic sensing but not real-time video processing. Audio-based models (keyword spotting) achieve real-time performance at 15-30 ms per inference window.

Power Consumption

Mode	ESP32-S3 Current	Duration (per inference cycle)
Active (240 MHz, inference)	95 mA	15-800 ms
Light sleep	130 μA	Between inferences
Deep sleep	7 μA	Idle periods

For battery-powered applications with periodic inference (e.g., every 10 seconds), the average current can be reduced to 1-3 mA, enabling weeks of operation on a 1000 mAh Li-Po battery.

Model Update Challenges

Deploying model updates to field devices requires either wired reprogramming or wireless OTA (over-the-air) updates. The ESP32-S3's Wi-Fi capability simplifies OTA, but the model binary must be signed and verified to prevent adversarial model substitution.

FAQ

What is TinyML and how does it differ from edge AI?

TinyML is a subset of edge AI that specifically targets microcontroller-class devices (32-512 KB RAM, milliwatt power). While edge AI encompasses all on-device inference—including powerful edge devices like NVIDIA Jetson with GBs of RAM—TinyML focuses on deeply resource-constrained devices using specialized techniques like INT8 quantization, operator fusion, and static memory allocation to fit neural networks into kilobytes of memory.

Can I run TensorFlow Lite models on ESP32-S3?

Yes. TensorFlow Lite for Microcontrollers (TFLM) is fully ported to the ESP32-S3 with optimized kernels using the ESP32-S3's vector instructions. Espressif maintains the esp-tflite-micro library, which includes ready-to-run examples for keyword spotting, person detection, and anomaly detection. Models must be converted to INT8 .tflite format before deployment.

How much memory does a TinyML model need on a microcontroller?

A typical TinyML model requires 1.5-3x its model weight size in total memory (weights + activation arena + runtime overhead). For example, a 20 KB INT8 keyword spotting model needs approximately 35-50 KB of total memory. The ESP32-S3 with 512 KB SRAM can comfortably run models up to 300 KB. Larger models require external PSRAM, which increases latency and power consumption.

Which microcontroller is best for TinyML in 2026?

The ESP32-S3 is the most popular TinyML MCU due to its vector instructions, 512 KB SRAM, integrated Wi-Fi/BLE, and low cost ($2.50). For industrial applications requiring certification, the STM32H7 offers higher performance and reliability at a higher cost. For ultra-low-power BLE applications, the nRF5340 is preferred. The Raspberry Pi RP2040 is the lowest-cost option ($1.00) but lacks wireless and vector instructions.

How do I quantize a machine learning model for ESP32-S3?

Use TensorFlow Lite Converter with post-training quantization (PTQ). Provide a representative dataset generator that samples typical inputs, set optimization to DEFAULT, and specify INT8 input/output types. The converter automatically quantizes weights and activations. For models sensitive to quantization, use quantization-aware training (QAT) in TensorFlow, which simulates INT8 during training to minimize accuracy loss.

What is the inference speed of TinyML models on ESP32-S3?

Inference speed depends on model complexity and input type. Audio keyword spotting models (18 KB, 12 classes) run in ~15 ms per 30 ms window—faster than real-time. Person detection models (250 KB, 96×96 image) run in ~800 ms. Anomaly detection models (8 KB, vibration FFT) run in ~5 ms. The ESP32-S3's vector instructions provide 2-4x speedup over scalar execution for INT8 operations.

References

TensorFlow. (2025). TensorFlow Lite for Microcontrollers — Official Documentation. https://www.tensorflow.org/lite/microcontrollers
Espressif Systems. (2025). ESP32-S3 Datasheet and Technical Reference Manual. https://www.espressif.com/en/products/socs/esp32-s3
Edge Impulse. (2025). TinyML with ESP32-S3 — Getting Started Guide. https://docs.edgeimpulse.com/docs/development-platforms/officially-supported-mcu-targets/esp32-s3
Pete Warden & Daniel Situnayake. (2019). TinyML: Machine Learning with TensorFlow Lite on Arduino and Ultra-Low-Power Microcontrollers. O'Reilly Media. https://www.oreilly.com/library/view/tinyml/9781492052036/
Harvard TinyML. (2025). TinyML Benchmark Suite — MCU Performance Comparisons. https://github.com/harvard-edge/tinyml-benchmark

Meta Description: Practical guide to running TinyML on ESP32-S3 and other microcontrollers. Covers TensorFlow Lite for Microcontrollers, INT8 model quantization, keyword spotting and person detection deployment, MCU comparisons (RP2040, STM32, nRF5340), toolchain setup, and performance benchmarks for embedded AI applications.