ESP32 Keyword Spotting: Build a Wake-Word Edge AI Device Without a Cloud Connection

ESP32 Keyword Spotting: Build a Wake-Word Edge AI Device Without a Cloud Connection

If you've built a few ESP32 projects already, you've probably hit the same wall I did: every "smart" feature seems to need Wi-Fi, a cloud API key, and a monthly bill. Voice control is usually the worst offender. Most tutorials wire a mic up to a Raspberry Pi, ship the audio to Google or AWS, and call it "AI."

That's not what we're doing here. This guide walks through building an actual offline keyword spotter — a device that listens for a specific word (like "hey lamp" or "start") and reacts instantly, entirely on an ESP32-S3, with no internet connection at all. No round-trip latency, no privacy leakage, no recurring API cost.

This is Edge AI in its purest form: the inference happens on a $6 chip sitting on your desk.

Why keyword spotting is the right first Edge AI project

Full speech recognition is genuinely hard to run on a microcontroller. Keyword spotting (KWS) is the tamed-down version — the model only has to answer one question: "was that word said, yes or no?" That narrow scope is exactly why it fits inside the ESP32-S3's memory and CPU budget, and why it's the standard entry point into embedded machine learning.

A few reasons this is worth your weekend:

• It's a real, working demo of TinyML — not a toy blink-an-LED example.
• It reuses skills you likely already have from other ESP32 projects (I2S, interrupts, FreeRTOS tasks).
• It scales into genuinely useful things: hands-free smart home triggers, accessibility devices, offline dictation triggers, security systems that arm on a spoken code word.

What you'll need

• ESP32-S3 dev board — needs the vector instructions and RAM the S3 has over the original ESP32
• I2S MEMS microphone (e.g. INMP441) — digital I2S mics are far cleaner than analog for this
• USB-C cable + 5V supply
• Arduino IDE or ESP-IDF — either works; examples below assume Arduino IDE with the ESP32 board package
• Edge Impulse account (free tier) — handles the model training so you don't need a TensorFlow background

You do not need a GPU, a Raspberry Pi, or any paid cloud service anywhere in this pipeline.

Step 1: Wire the I2S microphone

INMP441 to ESP32-S3, standard I2S wiring:

• VDD → 3.3V
• GND → GND
• SD (data out) → GPIO 4
• SCK (bit clock) → GPIO 5
• WS (word select / LR clock) → GPIO 6
• L/R → GND (locks the mic to the left channel)

Keep these wires short. I2S is timing-sensitive, and long jumper wires on a breadboard are a common source of static or dropped samples — if your recordings sound "crunchy" later, check this first before blaming the code.

Step 2: Capture training audio

This is the step most tutorials skip, and it's the one that actually determines whether your device works. You need:

• 30–60 short (1-second) recordings of your wake word, spoken by different people, at different distances from the mic, in different rooms.
• An equal amount of "noise" and "not the wake word" clips — background chatter, TV audio, other random words, silence.

Edge Impulse's ESP32 firmware has a built-in audio capture mode that streams samples straight to your project dashboard over serial, so you're not juggling WAV files by hand. Label each clip as you go: wake_word or noise.

The single biggest cause of a flaky keyword spotter isn't the model — it's a training set that's too small or too clean. Real rooms have fridges humming and keyboards clacking. Your dataset should too.

Step 3: Train the model

In Edge Impulse:

1. Create an Audio (MFCC) processing block — this converts raw audio into the frequency-based features the model actually learns from.
2. Add a small Neural Network (Keras) learning block. For KWS you don't need depth — 2–3 layers is standard and keeps inference fast.
3. Train, then check the confusion matrix. You're looking for high accuracy on wake_word specifically — false positives on "noise" matter less than missing the actual word.
4. Use Edge Impulse's built-in quantization to convert the model to int8. This step alone is often what makes the difference between a model that fits on the ESP32-S3 and one that doesn't.

Step 4: Deploy to the ESP32-S3

Export as an Arduino library (Edge Impulse builds this automatically) and drop it into your sketch. The core loop is simple in structure even though the model underneath isn't:

if (microphone_inference_record()) {
  run_classifier(&signal, &result, false);
  if (result.classification[WAKE_WORD_IDX].value > 0.85) {
    trigger_action(); // your GPIO, relay, MQTT publish, whatever
  }
}

Everything after trigger_action() is up to you — light a strip, unlock a servo, publish to a local MQTT broker (if you've already followed the MQTT tutorial on this site, this slots right in).

Common failure points

• Model triggers on everything. Your confidence threshold is too low, or your "noise" training set is too small. Push the threshold to 0.85–0.9 and add more negative samples.
• Model never triggers. Usually a mic gain or wiring issue — verify raw audio levels with a simple serial-print test before blaming the model.
• Works on your desk, fails across the room. Distance-varied training data fixes this almost every time. It's tempting to record 40 clips sitting right next to the mic; don't.
• Board runs out of memory. Confirm you're on int8 quantization, not float32. This one setting is the difference between "fits" and "doesn't."

Where this goes next

Once wake-word detection is solid, the natural next step is chaining it with the WiFi CSI presence detection approach or ESP-NOW mesh setup covered elsewhere on this site — a device that only "wakes up" its network stack when both motion and a voice trigger are present is a genuinely power-efficient design, and it's a good showcase project if you're building a portfolio around embedded ML.

FAQ

Does this need internet access at all?
No. After training (which happens once, in the cloud dashboard), the compiled model runs fully offline on the chip.

Can the original ESP32 (not S3) run this?
It's possible with a very small model, but the S3's extra RAM and vector instructions give you meaningfully more headroom before you start hitting reliability problems.

How much does this cost to build?
Under $15 in hardware if you already have a dev board — the mic is the only new part for most people who've been following ESP32 projects already.

---

If you're building out a niche blog like this one and want to see how a smaller site turns consistent posting into actual income, petlifehacks.page is worth a look as a case study in monetizing a focused niche site.