# Waveshare ESP32-S3 Touch AMOLED 1.75C Arduino CLI
This workspace automates build, upload, and serial monitoring for the Waveshare ESP32-S3-Touch-AMOLED-1.75C using `arduino-cli`.
## Current Tested Baseline
- `arduino-cli` 1.5.1
- `esp32:esp32` core 3.3.5
- Port: `/dev/cu.usbmodem83101`
- Sketch: `sketches/codex_hello_world`
- Serial diagnostic sketch: `sketches/serial_probe`
- Visual OCR sketch: `sketches/display_ocr_check`
- Vendor examples/libraries: `.vendor/ESP32-S3-Touch-AMOLED-1.75C/examples/arduino` (upstream renamed this from `examples/Arduino-v3.3.5`)
The installed ESP32 core does not currently expose a dedicated `ESP32-S3-Touch-AMOLED-1.75C` FQBN. The scripts use `esp32:esp32:esp32s3` with explicit board options and the Waveshare 1.75C `pin_config.h`.
## Workflow Map
Use the Arduino lane first when validating local toolchain, display, touch, PMU, IMU, and audio examples. Use the XiaoZhi lane only when you intend to replace the currently flashed Arduino sketch with the XiaoZhi firmware or inspect that firmware route.
## Arduino Commands
```bash
make setup
make build
make upload
make monitor
make smoke
make visual-smoke
make camera-aligner
make camera-diagnose
make camera-ready
make camera-reset
make ok-qoder-evidence
make feature-matrix-check
make hardware-evidence-audit
make hardware-evidence-doc
make evidence-index
make evidence-index-doc
make remaining-gates-doc
make remaining-gates-runbook
make hardware-smoke-list
make hardware-smoke-suite HARDWARE_SMOKE_ARGS="--dry-run"
make official-demos
make official-build DEMO=01-helloworld
make official-smoke DEMO=01-helloworld
```
Override defaults with environment variables or `.env`:
```bash
ARDUINO_PORT=/dev/cu.usbmodem83101 make upload
WAVESHARE_VENDOR_DIR=/path/to/ESP32-S3-Touch-AMOLED-1.75C-main make build
```
`make smoke` builds with a dedicated build directory, uploads, then records a short serial log under `.logs/`.
On this macOS USB Serial/JTAG port, raw `stty` + `cat` captures data reliably; set `ARDUINO_CLI_MONITOR=1` to force `arduino-cli monitor`.
`make visual-smoke` uploads a static high-contrast OCR screen and then captures one camera frame with `ffmpeg` before checking it with macOS Vision OCR. Defaults:
```bash
CAMERA_CAPTURE_ENGINE=auto
CAMERA_DEVICE=0
CAMERA_SIZE=1280x720
CAMERA_CROP="iw*0.55:ih*0.65:(iw-ow)/2:(ih-oh)/2"
OCR_ROTATE=0
DISPLAY_ROTATION=0
DISPLAY_BRIGHTNESS=128
OCR_EXPECTED="OK"
OCR_ENGINE=vision
CAMERA_CAPTURE_TIMEOUT=15
CAMERA_EXPOSURE_POINT=""
CAMERA_FOCUS_POINT=""
CAMERA_WARMUP_FRAMES=3
OCR_PREPROCESS_MODE=gray
COLOR_SWATCH_CHECK=0
COLOR_SWATCH_ROI="0.35,0.35,0.40,0.40"
```
Point the camera at the AMOLED before running the command. If macOS asks for camera permission, allow the terminal/Codex process and rerun.
If the board appears upside down in the camera, prefer `DISPLAY_ROTATION=2 make visual-smoke` so the sketch renders OCR text upright for the camera. Use `OCR_ROTATE` only when you cannot change the displayed orientation.
Camera capture is bounded by `CAMERA_CAPTURE_TIMEOUT` so automation fails clearly instead of hanging when the selected camera device is unavailable or already owned by another app. `CAMERA_CAPTURE_ENGINE=auto` prefers the SwiftPM `CameraSnapshot` tool and falls back to ffmpeg. For bright AMOLED frames, lower `DISPLAY_BRIGHTNESS`, keep `OCR_PREPROCESS_MODE=color`, set `CAMERA_EXPOSURE_POINT` / `CAMERA_FOCUS_POINT` to a normalized point on the display, for example `0.5,0.65`, and enable `COLOR_SWATCH_CHECK=1` to verify the red/green/blue/yellow calibration swatches by largest connected color block, average RGB, centroid, and row/order geometry.
The OCR script can also reprocess an existing raw frame without touching the camera:
```bash
CAMERA_RAW_IMAGE=.logs/camera-ocr-YYYYMMDD-HHMMSS.jpg OCR_PREPROCESS_MODE=amoled ./scripts/camera-ocr.sh
```
The visual calibration sketch includes large `OK` text plus red/green/blue/yellow swatches. This is the preferred camera setup target before trying vendor demos with white backgrounds or small LVGL fonts. To check swatches on an existing image directly, run `make camera-color-check IMAGE=.logs/camera-ocr-YYYYMMDD-HHMMSS.jpg`; tune strict placement checks with `COLOR_SWATCH_MIN_X_GAP`, `COLOR_SWATCH_MAX_Y_SPREAD`, or `COLOR_SWATCH_GEOMETRY=0`.
`make ok-qoder-evidence` is the article-ready self-verification chain for the default `sketches/codex_hello_world` sketch. It clean-builds the sketch, uploads it, records serial frames, captures the screen with large `Qoder` branding, uses OCR on the stable `OK` marker, and writes a committed evidence pack under `docs/evidence/ok-qoder-<timestamp>/`. The full chain is only complete when the serial smoke passes, a camera image is saved, and OCR sees the stable marker. Use `ALLOW_PARTIAL=1 make ok-qoder-evidence` when you want to preserve a partial run for debugging or writing: the summary will still say whether the visual proof is missing. This is the smallest loop for explaining AI self-verification and self-iteration:
1. change firmware source
2. build with deterministic board options
3. flash the board
4. prove control flow through serial logs
5. prove the AMOLED through camera image and OCR
6. write artifacts and a machine-readable summary
7. let the next agent iterate only from failed evidence, not from guesses
Latest saved run: `docs/evidence/ok-qoder-20260614-120532/summary.md`. It proves build, upload, serial runtime, camera capture, and OCR for the large `Qoder` screen with the stable `OK` marker. That pack records `display_rotation=0` and `ocr_rotation=180` because the current camera mount sees the board upside down while the physical display layout remains readable on the desk.
`make claude-skill-smoke` asks the local `claude` CLI to read the repo-owned Waveshare Skill and invoke its helper script for the same non-audio visual smoke path. This validates the agent-facing Skill wiring from a second agent process. The default `CLAUDE_SKILL_SMOKE_MODE=visual` uploads the calibration sketch, captures camera OCR, and checks the color swatches. Use `CLAUDE_SKILL_SMOKE_MODE=audit make claude-skill-smoke` for a no-upload feature-matrix-only fallback. Details and local evidence are in `docs/skill-automation.md`.
`make camera-aligner` opens a SwiftPM macOS camera tuning tool. Use it to:
- preview the selected camera live
- adjust the OCR crop rectangle with sliders
- set OCR rotation for boards that appear upside down in the camera
- see Vision OCR results update live
- copy the generated `CAMERA_CROP` and `OCR_ROTATE` values for `make visual-smoke`
`make camera-diagnose` writes a bounded diagnostic bundle under `.logs/camera-diagnose-*` with macOS camera inventory, Swift device status, camera-related processes, and optional Swift/ffmpeg video-only capture probes. Use `CAMERA_DIAGNOSE_FFMPEG=0 make camera-diagnose` to avoid ffmpeg device enumeration. `make camera-ready` is the stricter preflight for visual smokes: it uses Swift capture only and fails unless a frame is saved. If Swift diagnostics show `running=true frames=0 drops=0`, the camera session started but macOS/USB delivered no video buffers; fix the host camera path before rerunning board visual smokes. `make camera-reset` restarts macOS camera services and then runs the same strict camera-ready check; it does not quit chat apps unless `CAMERA_RESET_QUIT_CHAT_APPS=1` is set.
`make feature-matrix-check` verifies that each requested feature direction has matching Makefile, script/sketch, documentation, and Skill helper coverage. `make feature-matrix-doc` regenerates `docs/hardware-verification-matrix.md`.
`make hardware-evidence-audit` reports which feature lanes have documented verification evidence and latest smoke-suite evidence. `make hardware-evidence-doc` regenerates `docs/hardware-evidence-audit.md`.
`make evidence-index` prints a consolidated cross-reference of strict completion, feature docs, hardware-suite status, visual OCR status, and remaining-gate status. `make evidence-index-doc` regenerates `docs/evidence-index.md`; this is the quickest article and handoff map, while `make goal-completion-audit` remains the strict completion gate.
`make remaining-gates-preflight` refreshes the safe side of the still-incomplete gates without audio, XiaoZhi flashing, or unsupervised physical taps. `make remaining-gates-doc` regenerates `docs/remaining-gates-preflight.md` from the latest summary so these blocked completion requirements stay explicit. `make remaining-gates-runbook` regenerates `docs/remaining-gates-supervised-runbook.md`, which is the operator checklist for the four gates that still need an allowed audio window, explicit XiaoZhi flash approval, or a supervised AMOLED tap.
`make hardware-smoke-list` shows which feature smokes are selected by the default non-audio suite. `make hardware-smoke-suite` runs those selected smokes serially and writes logs under `.logs/hardware-smoke-suite/`; pass `HARDWARE_SMOKE_ARGS="--target iot-panel --skip-build"` for a narrow lane. When `--with-visual` is set, the suite runs `make camera-ready` first and aborts before any upload if the host camera cannot save a frame. Details are in `docs/hardware-smoke-suite.md`.
`make official-demos` lists the Waveshare official Arduino examples tracked in `config/official-demos.tsv`.
Use `make official-build DEMO=<id>` for compile-only validation, and `make official-smoke DEMO=<id>` to upload a vendor demo and verify its expected serial output. Set `OFFICIAL_VISUAL_STABLE_MARKER=1 OFFICIAL_VISUAL_SMOKE=1 make official-smoke DEMO=01-helloworld` when you also want camera/OCR proof for the display baseline; this applies a staged-only large `OK` marker and leaves the vendor source untouched. Start with `DEMO=01-helloworld`, then move through PMU, IMU, LVGL, and audio demos.
See `docs/p0-official-demos.md` for the current P0 bring-up matrix and local verification notes.
## XiaoZhi AI Commands
```bash
make xiaozhi-latest
make xiaozhi-download
make xiaozhi-inspect
make xiaozhi-preflight
make xiaozhi-readiness
make xiaozhi-backup
make xiaozhi-idf-env
make xiaozhi-idf-build
CONFIRM=--yes make xiaozhi-flash
make xiaozhi-source-clone
make xiaozhi-source-check
```
`make xiaozhi-latest`, `make xiaozhi-download`, `make xiaozhi-inspect`, `make xiaozhi-preflight`, and `make xiaozhi-backup` automate the non-destructive prebuilt XiaoZhi AI firmware route for `waveshare-esp32-s3-touch-amoled-1.75c`. `make xiaozhi-readiness` bundles preflight, source check, ESP-IDF env/build, and latest rollback-image verification without flashing or using audio; set `XIAOZHI_READINESS_BACKUP=1` to refresh the 16MB board backup before an approved flash. `make xiaozhi-backup` reads the current board flash into `.vendor/xiaozhi/backups/` with `destructive=0 audio=0`. `CONFIRM=--yes make xiaozhi-flash` writes the downloaded merged binary to the board and is intentionally explicit because it replaces the Arduino sketch currently on the device.
The source route uses `make xiaozhi-idf-env`, `make xiaozhi-idf-build`, `scripts/xiaozhi.sh idf-flash`, and `scripts/xiaozhi.sh idf-monitor`. See `docs/p0-xiaozhi-ai.md` for the current XiaoZhi acceptance notes.
## Git Hooks
```bash
make install-hooks
make hook-smoke
```
The versioned `pre-push` hook detects outgoing push refs or commit subjects that include `feat`. When it finds one, it updates the generated Feature Push Notes block in this README and stops the push so the README change can be reviewed and committed before pushing again.
`make hook-smoke` validates that behavior in a temporary git worktree without modifying the current checkout.
## Cloud AI Terminal Commands
```bash
make cloud-ai-build
make cloud-ai-smoke
CLOUD_AI_PIPELINE=1 make cloud-ai-smoke
make cloud-ai-pipeline-smoke
make cloud-ai-cache-smoke
CLOUD_AI_VISUAL_SMOKE=1 DISPLAY_ROTATION=2 make cloud-ai-smoke
make local-ai-server
make web-ai-button-build
make web-ai-button-smoke
make web-ai-button-tap-smoke
make audio-vad-build
make audio-afe-readiness
make audio-vad-preflight
make audio-vad-smoke
make speaker-output-build
make speaker-output-smoke
make sensor-status-build
make sensor-status-smoke
make power-lifecycle-build
make power-lifecycle-smoke
make wifi-connectivity-build
make wifi-connectivity-smoke
make touch-status-build
make touch-status-smoke
make interaction-dashboard-build
make interaction-dashboard-smoke
make imu-interaction-build
make imu-interaction-smoke
make desk-widget-build
make desk-widget-smoke
make desk-widget-relay-smoke
make iot-panel-build
make iot-panel-smoke
make iot-panel-relay-smoke
make tinyml-imu-build
make tinyml-imu-smoke
make esp-claw-agent-build
make esp-claw-agent-smoke
make offline-voice-build
make offline-voice-smoke
make lvgl-visual-agent-build
make lvgl-visual-agent-smoke
make nes-contra-preflight
make nes-contra-build
make nes-contra-smoke
make hardware-evidence-audit
make hardware-evidence-doc
make evidence-index
make evidence-index-doc
make remaining-gates-doc
make remaining-gates-runbook
make hardware-smoke-list
make hardware-smoke-suite HARDWARE_SMOKE_ARGS="--dry-run"
```
`make cloud-ai-smoke` uploads the self-developed `cloud_ai_terminal` sketch, runs the host serial relay in mock mode, and verifies the board displays an AI response. `make cloud-ai-pipeline-smoke` drives the silent ASR -> LLM -> TTS serial pipeline and verifies `PIPELINE_DONE` without using the microphone or speaker. `make cloud-ai-cache-smoke` additionally validates board-local NVS cache and runtime state commands. These slices validate the display and host/cloud protocol shape; real audio capture and speaker playback are tracked in `docs/p0-cloud-ai-terminal.md`.
`make web-ai-button-smoke` starts a local Mac HTTP AI server, uploads `sketches/web_ai_button`, sends Wi-Fi credentials from the ignored `.env` file over serial, and verifies that the board can trigger the server and display the returned AI text. The AMOLED shows `Qoder` above a large `ASK AI` touch button. Use `make web-ai-button-tap-smoke` for the supervised physical tap gate; it waits for `WEB_AI_TOUCH_EVENT`, `WEB_AI_TRIGGER source=touch`, and the HTTP AI response. Use `WEB_AI_KEEP_SERVER=1 SKIP_BUILD=1 make web-ai-button-smoke` when the server should stay alive for manual taps after automation exits; details are in `docs/p1-web-ai-button.md`.
Latest saved web-button evidence: `docs/evidence/web-ai-button-qoder-20260615-081421/summary.md`.
`make audio-afe-readiness` rebuilds the ES7210/VAD probe and reports no-audio readiness for the AFE lane: implemented ES7210 capture and ESP-SR VAD source/build/checker status, plus planned AEC, NS, and WakeNet source-integration/physical-audio requirements. `make audio-vad-preflight` includes the same readiness report and remains safe when audio should stay quiet. `make audio-vad-smoke` uploads the ES7210 microphone probe, plays a host-side `say` stimulus, and validates serial RMS/peak metrics from the board. This is the microphone capture gate before full ASR streaming; details are in `docs/p0-audio-vad-probe.md`.
`make speaker-output-smoke` uploads the ES8311 speaker probe, sends `PLAY` over serial, records the board output through a host microphone, and validates the active audio window against baseline energy. Use `SPEAKER_VISUAL_SMOKE=1 DISPLAY_ROTATION=2 make speaker-output-smoke` when camera OCR should also verify `SPK OK`; details are in `docs/p0-speaker-output-probe.md`. Avoid running audible audio smokes late at night unless explicitly requested.
`make sensor-status-smoke` uploads the AXP2101 + QMI8658 probe and validates PMU/IMU serial metrics without using any audio device. Use `SENSOR_STATUS_VISUAL_SMOKE=1 DISPLAY_ROTATION=2 make sensor-status-smoke` when camera OCR should also verify `SENS OK`; details are in `docs/p1-sensor-status-probe.md`.
`make power-lifecycle-smoke` uploads the AXP2101 power lifecycle probe and validates DIM, STANDBY, ACTIVE, brightness, capacity, load profile, wake, and runtime-estimate serial behavior without using audio devices. Use `POWER_LIFECYCLE_VISUAL_SMOKE=1 DISPLAY_ROTATION=2 make power-lifecycle-smoke` when camera OCR should also verify the screen reaches `OK`; the visual path defaults to color preprocessing plus centered focus/exposure for this probe. The default smoke does not require a connected battery because the battery connector may be unused; set `POWER_REQUIRE_BATTERY=1` only when a battery is connected. The automated standby keeps USB serial alive instead of entering true deep sleep; details are in `docs/p1-power-lifecycle-probe.md`.
`make wifi-connectivity-smoke` uploads the Wi-Fi connectivity probe and validates the ESP32-S3 radio with a scan-only default. It reports AP count, best RSSI, and connection state without printing SSID names or storing credentials. Set `WIFI_TEST_SSID` and `WIFI_TEST_PASSWORD` only for a supervised join check; details are in `docs/p1-wifi-connectivity-probe.md`.
`make touch-status-smoke` uploads the CST9217 touch-controller probe and validates that the controller is online. Use `TOUCH_STATUS_VISUAL_SMOKE=1 DISPLAY_ROTATION=2 make touch-status-smoke` for camera OCR, or `TOUCH_REQUIRE_EVENT=1 make touch-status-smoke` when a human can tap the screen during the smoke window; details are in `docs/p1-touch-status-probe.md`.
`make interaction-dashboard-smoke` uploads the combined non-audio dashboard and drives it with serial commands across HOME, IMU, PWR, and TOUCH pages. It validates display control flow, CST9217 controller presence, AXP2101 PMU metrics, QMI8658 IMU metrics, serial-simulated gesture handling, brightness, standby, and wake transitions in one sketch. Use `INTERACTION_DASHBOARD_VISUAL_SMOKE=1 DISPLAY_ROTATION=2 make interaction-dashboard-smoke` when camera OCR should also verify the screen reaches `OK`; details are in `docs/p1-interaction-dashboard.md`.
`make imu-interaction-smoke` uploads the dedicated QMI8658 interaction probe and validates wrist wake, shake-to-switch, posture menu, and step counting through deterministic serial-injected IMU samples. Use `IMU_INTERACTION_VISUAL_SMOKE=1 DISPLAY_ROTATION=2 make imu-interaction-smoke` when camera OCR should also verify the screen reaches `OK`; details are in `docs/p1-imu-interaction-probe.md`.
`make desk-widget-smoke` uploads the serial-driven desk widget and validates CI/GitHub/alert/timer/AI-summary pages without network credentials or audio devices. `make desk-widget-relay-smoke` validates the host adapter that turns mock, JSON, or HTTP events into the same widget protocol. Use `DESK_WIDGET_VISUAL_SMOKE=1 DISPLAY_ROTATION=2 make desk-widget-smoke` when camera OCR should also verify the screen reaches `OK`; details are in `docs/p1-desk-widget.md`.
`make iot-panel-smoke` uploads the serial-driven IoT control panel and validates device state changes, MQTT-style inbound updates, HTTP-style outbound actions, and scene changes without Wi-Fi credentials. `make iot-panel-relay-smoke` validates the host adapter that turns mock, JSON, or HTTP smart-home events into the same panel protocol. Use `IOT_PANEL_VISUAL_SMOKE=1 DISPLAY_ROTATION=2 make iot-panel-smoke` when camera OCR should also verify the screen reaches the stable `IOT` marker; details are in `docs/p1-iot-control-panel.md`.
`make tinyml-imu-smoke` uploads the TinyML IMU classifier scaffold, disables live mode, injects deterministic accelerometer/gyroscope feature vectors over serial, and verifies `REST`, `TILT_LEFT`, `TILT_RIGHT`, `FACE_UP`, and `SHAKE` labels. `TINYML_IMU_VISUAL_SMOKE=1 DISPLAY_ROTATION=2 OCR_ROTATE=180 make tinyml-imu-smoke` adds camera OCR against the stable `TINY` marker. This is the model automation harness before replacing the embedded classifier with ESP-DL or a trained model; details are in `docs/p2-tinyml-imu-classifier.md`.
`make esp-claw-agent-smoke` uploads the ESP-Claw/OpenClaw Arduino harness and validates an agent control loop over serial: local rule add, event sensing, rule decision, MCP-style tool call, IM chat input, tagged memory write, and LLM fallback routing. Use `ESP_CLAW_AGENT_VISUAL_SMOKE=1 DISPLAY_ROTATION=2 DISPLAY_BRIGHTNESS=96 make esp-claw-agent-smoke` when camera OCR should also verify the screen reaches `OK`. This is a deterministic compatibility slice before replacing it with the official ESP-Claw firmware route; details are in `docs/p2-esp-claw-agent.md`.
`make offline-voice-smoke` uploads the offline voice-control harness and validates the WakeNet/MultiNet-facing state machine without using the microphone: pre-wake rejection, wake event, command recognition, runtime command add, continuous mode, sleep/wake, and local actions. `OFFLINE_VOICE_VISUAL_SMOKE=1 DISPLAY_ROTATION=2 OCR_ROTATE=180 make offline-voice-smoke` adds camera OCR against the stable `VOICE` marker. This is the serial control-plane gate before wiring real ESP-SR audio; details are in `docs/p1-offline-voice-control.md`.
`make lvgl-visual-agent-smoke` uploads the repo-owned LVGL visual-agent surface and validates LVGL tabview pages for chat bubbles, cards, settings, and agent thoughts over serial. `LVGL_VISUAL_AGENT_VISUAL_SMOKE=1 DISPLAY_ROTATION=2 DISPLAY_BRIGHTNESS=96 make lvgl-visual-agent-smoke` also captures the AMOLED and OCRs the stable large `OK` marker; serial remains the proof for LVGL behavior. This complements the official LVGL widgets demo with an automatable agent UI; details are in `docs/p1-lvgl-visual-agent.md`.
`make nes-contra-preflight` inspects the local `/Users/phodal/hardware/nes-contra-us` disassembly checkout, cc65 tools, and legally supplied ROM state. `make nes-contra-build` compiles the diagnostic emulator scaffold, and `make nes-contra-smoke` uploads it and validates the `NES_CONTRA_*` serial protocol, display readiness, touch readiness, input injection, and frame progress. This lane is partial until a reviewed emulator core boots a Mapper 2 ROM; details are in `docs/p2-nes-contra-emulator.md`.
<!-- feat-push-readme:start -->
## Feature Push Notes
This section is maintained by `.githooks/pre-push` when outgoing push refs or commit subjects include `feat`.
Outgoing feature commits:
| Commit | Subject |
| --- | --- |
| `6123df7` | feat: add NES Contra emulator scaffold |
Commit this README update, then run `git push` again.
<!-- feat-push-readme:end -->