A few years ago, building a camera that spots birds at a feeder, sorts them by species and shows the results on a private website meant learning firmware, image models, web servers and hosting, one after another. Today some people build exactly that by describing it to an AI coding agent and connecting the pieces it produces. ESP32 flight trackers, voice desk pets that talk to coding agents and usage meters for AI plans are now being built the same way, with most of the code written by AI.
This is often called vibe coding. It works better on hardware than many people expect, and it also fails in ways that are different from web apps. This guide explains how to do it well, using a bird feeder camera as the running example.
Why hardware is different
On a web app, a bug shows an error in the browser. On a microcontroller, a bug can mean a board that resets every few seconds, a sensor that returns zeros or a screen that stays black. The AI cannot see your desk. It does not know which pins you used, whether your cable carries data or whether your camera module is the one it assumed.
So the main job is to give the agent eyes. You do that with a clear description of your hardware, a way for it to build and flash and a way for it to read the serial log.
The tools
An AI code editor such as Cursor, or a terminal agent such as Claude Code or Codex, open in your project folder.
PlatformIO or ESP-IDF on the command line, so the agent can build, flash and open the serial monitor with commands. PlatformIO works as an extension in VS Code-based editors and is a common choice for this kind of work.
Git, so every working state is saved.
Step one: write the hardware down
Before the first prompt, write a short project file in plain words. Many agents read files like AGENTS.md or CLAUDE.md automatically. Include:
The exact board name and chip, such as an ESP32-S3 camera board, with its flash and PSRAM sizes.
Every part and the pins it uses.
How the board is powered.
What the device should do, in a few sentences.
What it must not do, such as never storing Wi-Fi passwords in the code.
For the bird feeder, that might read: an ESP32-S3 camera board takes a photo when motion is detected, uploads it to a small server, the server runs a bird classifier and a web page shows recent visitors to family members only.
Step two: split the system
A device like this has several parts. Ask the agent to plan them separately.
The firmware: camera setup, motion detection, Wi-Fi, uploading.
The server: receiving images, running a classifier, storing results.
The website: showing results, with a login.
Deployment: putting the server on a host.
Build them in that order, one at a time. Do not ask for everything in one prompt.
Step three: smallest working firmware first
Ask for firmware that only does one thing: connect to Wi-Fi and print the IP address to the serial monitor. Build, flash and check. Then ask for a version that takes one photo and prints its size. Then one that uploads a test image. Each step proves one piece of hardware or one connection.
Let the agent run the build and flash commands, and approve each one. After flashing, have it open the serial monitor and read the log. This loop of build, flash and read is what lets an agent fix firmware without seeing the board.
Step four: the AI parts
For a bird feeder, the classifier can run in two places. It can run on the server, where you have plenty of memory and can use an existing bird species model. Or a small detection model can run on the camera board itself, which saves uploads and works offline but limits accuracy. For a first version, keep the ESP32 simple: detect motion, take the photo and send it. Let the server do the heavy thinking.
Step five: deploy and protect
Ask the agent to help you deploy the server to a hosting service and set up a domain. Then ask it, separately, to review the security: login on the website, a secret key so only your camera can upload and limits on upload size. A family photo page should not be open to the world by accident.
Pitfalls that show up again and again
Invented pins and APIs. Agents sometimes use pin numbers from a different board or functions from an older library version. Your project file and the compile errors are the cure. Paste exact errors back instead of describing them.
Brownouts. Cameras and Wi-Fi draw bursts of current. If the board resets when it takes a photo, the power supply is a likely cause, not the code. Tell the agent the reset reason from the log.
Memory limits. Camera frames are large. Boards without PSRAM struggle. Choose a board with PSRAM for camera work.
Secrets in code. Agents often place Wi-Fi passwords and API keys directly in source files. Ask for a separate configuration file that Git ignores, and check before every commit.
Going in circles. If the agent tries the same fix three times, stop. Ask it to list what it knows and what it has ruled out, or start a fresh session with a clear summary.
Not learning anything. If you only paste errors, you will be stuck the first time the agent is. Ask it to explain each new piece of code in a sentence or two. Over time, you will start to spot its mistakes yourself.
Ideas for first projects
A desk flight tracker that shows aircraft overhead using a public flight data feed.
A plant monitor that reads soil moisture and sends a daily message.
A usage meter that shows how much of your AI coding plan you have used.
A voice desk pet that speaks when your coding agent needs approval.
A weather or calendar display on an e-ink screen.
The short version
AI coding agents can write a large share of the firmware and server code for an ESP32 gadget, but they cannot see your hardware. Write down your board, parts and pins in a project file, give the agent build, flash and serial log commands, and build in small steps that each prove one thing. Keep the board simple and push heavy AI work to a server, watch power and memory, keep secrets out of the code and ask the agent to explain what it writes so you keep learning.