Idea 1: Hierarchical Always-On Sensing via ULP Coprocessor
Overview
Design a two-tier hierarchical inference engine where the ESP32’s Ultra-Low-Power (ULP) coprocessor runs a tiny quantized model while the main CPU is in deep sleep. The ULP performs coarse “wake-word” detection; only when triggered does the main CPU boot and run fine-grained classification.
Research Gap
- All existing TinyML on ESP32 runs inference on the main CPU, keeping it awake and burning ~100-500 mA
- No prior work has demonstrated machine learning on the ESP32 ULP coprocessor
- The ULP runs at ~17.5 MHz with only 8 KB RTC SRAM, making model design a non-trivial constraint
Key Innovation
Hardware-software co-design for ESP32’s asymmetric compute architecture. Formalize the energy-delay trade-off and demonstrate 10-50x average power reduction versus always-on main-CPU inference.
Hardware Requirements
| Component | Specification | Purpose |
|---|---|---|
| Main MCU | ESP32-S3 (ULP RISC-V) | Easier to program than FSM-based ULP |
| Sensor | I2S MEMS microphone or IMU | Input signal source |
| Power Monitor | INA219 or INA3221 | Profiling energy consumption |
| Logic Analyzer | Saleae or DSLogic | Measuring wake-up latency |
Technical Approach
Phase 1: ULP Model Design
- Target model size: <4 KB (fits in ULP’s 8 KB RTC SRAM)
- Use extreme quantization: binary or ternary weights where possible
- Design a tiny decision tree or 1-2 layer perceptron for coarse filtering
- Implement in ULP RISC-V assembly or C with ESP-IDF ULP toolchain
Phase 2: Main CPU Model
- Standard TFLite Micro model stored in flash
- Loaded into RAM only upon ULP wake interrupt
- Can be a full CNN or RNN for fine-grained classification
Phase 3: Wake Policy
- ULP samples sensor at low frequency (e.g., 16 Hz for audio, 10 Hz for IMU)
- If confidence > threshold, trigger GPIO wake interrupt
- Main CPU boots from deep sleep (~100-300 ms boot time)
- Include hysteresis to prevent rapid wake/sleep cycles
Phase 4: Energy Modeling
- Measure: ULP active current, ULP sleep current, main CPU active current, boot energy
- Derive optimal wake threshold as function of event arrival rate
- Compare against: always-on main CPU, always-on ULP (no main CPU), cloud offloading
Expected Results
- 10-50x average power reduction in low-activity scenarios
- <5% accuracy loss compared to always-on main CPU
- Wake-up latency <500 ms end-to-end
- Formal model of energy-optimal threshold selection
Target Venues
- IEEE Internet of Things Journal
- ACM Transactions on Embedded Computing Systems (TECS)
- ACM SenSys
- IEEE Transactions on Mobile Computing
Related Work to Cite
- ESP32 ULP coprocessor documentation (Espressif)
- Always-on keyword spotting papers (e.g., Hello Edge, Hello Edge 2)
- Hierarchical sensing in mobile systems (e.g., NeuroZero, Capybara)
- Energy-proportional computing literature
Risk Mitigation
| Risk | Mitigation |
|---|---|
| ULP memory too small | Use feature extraction on sensor; ULP only does threshold comparison |
| Boot latency too high | Keep model in RTC memory; use lighter wake-up modes |
| ULP programming complexity | Use ESP32-S3 ULP RISC-V (C compiler available) instead of ESP32 FSM |
Timeline (Approximate)
- Weeks 1-2: ULP toolchain setup, basic GPIO wake demo
- Weeks 3-4: ULP model design and deployment
- Weeks 5-6: Main CPU model integration
- Weeks 7-8: Energy profiling and threshold optimization
- Weeks 9-10: Evaluation and paper writing