🛠 Inference IA accélérée par les instructions vectorielles

Objectives

Understand what the LX7 vector instructions (absent in the ESP32 and S2) bring to int8 quantized neural network inference: the ESP-NN (layers) and ESP-DSP (signal) libraries exploit them, enabling the ESP32-S3 to run ESP-WHO (face detection) or ESP-Skainet (offline voice recognition). You will measure the gain yourself with a fully connected layer computed by esp_nn_fully_connected_s8, the flagship primitive of ESP-NN.

Hardware

• 1 ESP32-S3 board (N8R8/N16R8 variants with octal PSRAM are comfortable for large models)
• Arduino IDE with esp32 core >= 2.0.14 (ESP-NN is provided as a core component) or an ESP-IDF v5.x project with the esp-nn component
• 1 USB cable

Full Code

// Couche fully-connected int8 acceleree (ESP-NN, cote Arduino via esp32 core)
// Outils -> Board: "ESP32S3 Dev Module" ; ajouter "esp_nn" aux includes paths
#include "esp_nn.h"

#define ENTREES 64
#define SORTIES 8

// Poids et biases quantifies int8 (ici des valeurs demo)
static int8_t  poids[SORTIES][ENTREES];
static int32_t biais[SORTIES];
static int8_t  entree[ENTREES];
static int8_t  sortie[SORTIES];

void setup {
  Serial.begin(115200);
  for (int i = 0; i < SORTIES; i++) {
    biais[i] = 0;
    for (int j = 0; j < ENTREES; j++)
      poids[i][j] = (int8_t)((i * 7 + j * 13) % 251 - 125);
  }
  for (int j = 0; j < ENTREES; j++) entree[j] = (int8_t)(j % 127);

  setCpuFrequencyMhz(240);
  dl_nn_version;   // version d'ESP-NN liee

  uint32_t t0 = micros;
  for (int k = 0; k < 1000; k++) {
    esp_nn_fully_connected_s8(
      entree, ENTREES, 0,            // activations, taille, zero-point
      poids, SORTIES, 0,             // poids, sortie, zero-point poids
      biais, 1033, 12,               // biais, mult, shift (quantification)
      sortie, -128, 127);            // tampon sortie, plage int8
  }
  uint32_t t1 = micros;
  Serial.printf("1000 FC %dx8 : %lu us (ESP-NN vectoriel)\n",
                ENTREES, (unsigned long)(t1 - t0));
  Serial.printf("sortie[0] = %d\n", sortie[0]);
}

void loop { delay(1000); }

Expected Verification

The console displays the ESP-NN version followed by the time taken for one thousand 64×8 int8 layers — on the order of a few hundred microseconds on a core running at 240 MHz, a throughput that allows audio or sensor models to perform several dozen inferences per second. Comparing this with a hand-written reference loop (without esp_nn), the visible difference demonstrates the effect of the LX7 vector instructions. For a real-world case, use TensorFlow Lite for Microcontrollers with the ESP-NN delegate, or directly ESP-WHO/ESP-Skainet.

Common Errors

• Believing that simply being on the S3 accelerates everything: only vectorized libraries (ESP-NN, ESP-DSP, TFLite ESP-NN delegate) take advantage of the LX7; naive float code gains almost nothing.
• Poorly configured quantization: inconsistent multiply/shift (mult/shift) values between training and deployment produce incorrect outputs without runtime errors.
• Forgetting PSRAM for large models: a face model (ESP-WHO) saturates the 512 KB internal RAM; choose an R variant (octal PSRAM on GPIO35/36/37) and enable "Support of PSRAM" in the Tools menu.
• Not increasing the frequency: some builds run at 160 MHz by default; call setCpuFrequencyMhz(240) before measuring.
• Wi-Fi during inference: the radio adds latency and cache/PSRAM contention; measure performance with the radio turned off.