i-see-you

name

i-see-you

description

Take a still photo from the machine's local webcam with ffmpeg, then read/analyze or send the image. Cross-platform — Linux (V4L2), macOS (AVFoundation), Windows (DirectShow). Trigger when the user asks to "看看我", "take a photo", "take a picture", "selfie", "拍照", "自拍", "看看现在的环境/房间", or when the agent needs to visually observe the physical environment through the machine's camera. Requires ffmpeg and a local camera.

i-see-you

Capture a single frame from the machine's local webcam via ffmpeg, then either read the JPEG (multimodal models can see it directly) or send it to the user.

The only per-OS difference is ffmpeg's capture backend and how a device is named:

OS	Backend flag	Device reference
Linux	-f v4l2	path, e.g. /dev/video0
macOS	-f avfoundation	index, e.g. 0
Windows	-f dshow	quoted name, e.g. video="Integrated Camera"

Capture (recommended — use the helper script)

Both helper scripts auto-detect the first camera, warm up auto-exposure (skip the first 30 frames), locate ffmpeg (self-healing PATH on Windows), and fall back to the device's default resolution if the requested one is rejected.

For a non-technical user, pass the auto-install flag (-AutoInstall on Windows, --auto-install on Linux/macOS): if ffmpeg is missing the script installs it (winget/scoop/choco, or brew/apt/dnf) and then captures — one shot, no manual setup.

Linux / macOS (bash):

bash scripts/take_selfie.sh --auto-install                    # -> $TMPDIR/selfie_<timestamp>.jpg @ 1280x720
bash scripts/take_selfie.sh /tmp/selfie.jpg 1280x720 --auto-install

Windows (PowerShell):

powershell -NoProfile -ExecutionPolicy Bypass -File scripts/take_selfie.ps1 -AutoInstall
powershell -NoProfile -ExecutionPolicy Bypass -File scripts/take_selfie.ps1 `
    -AutoInstall -Output "$env:TEMP\selfie.jpg" -Resolution 1280x720 -Device "Integrated Camera"

Each script prints a 文件路径: line with the final path — Read that path to see the image.

Capture (manual one-liners)

Linux — device is /dev/video0:

ffmpeg -y -loglevel error -f v4l2 -video_size 1280x720 -i /dev/video0 \
       -vf "select=gte(n\,30)" -frames:v 1 -vsync 0 -f image2 /tmp/selfie.jpg

macOS — list devices, then capture by index (0 = first video device). The terminal app needs camera permission the first time:

ffmpeg -f avfoundation -list_devices true -i ""          # find the [index] of your camera
ffmpeg -y -loglevel error -f avfoundation -framerate 30 -video_size 1280x720 -i "0" \
       -vf "select=gte(n\,30)" -frames:v 1 -vsync 0 -f image2 /tmp/selfie.jpg

Windows — list devices, then capture by exact quoted name:

ffmpeg -hide_banner -f dshow -list_devices true -i dummy   # find the "name" of your camera
ffmpeg -y -loglevel error -f dshow -video_size 1280x720 -i video="Integrated Camera" `
       -vf "select=gte(n\,30)" -frames:v 1 -fps_mode passthrough -f image2 "$env:TEMP\selfie.jpg"

After capture — REQUIRED: display the photo, don't just save it

⚠️ Running the capture script is NOT the finish line. The script only writes a JPEG to disk; the photo does NOT appear in the chat until you open it with your Read/view tool. If you stop after seeing "拍照成功", the user sees nothing — that is the exact failure this section exists to prevent.

So every time, the moment the script prints 文件路径: <PATH>:

1. Call the Read tool on that exact <PATH> — in your very next action, before you write any reply. Reading the JPEG both lets you see the frame and renders it inline in the conversation so the user sees it.
2. Then describe what you see in your own voice — never a bare "done"/"拍好了". The image and a short, genuine description should arrive together.

Do this even if the user only said "拍照"/"take a photo" — showing the result is part of taking it. Skip it only if the user explicitly asked you not to display the photo.

• Never upload the image or send it to any external service — it stays local.
• If the host also has an explicit file-send/attach tool, you may additionally attach the JPEG, but the Read call is what actually makes it show in the chat.

Environment

• Dependency: ffmpeg on PATH.
  • Linux: sudo apt install ffmpeg (Debian/Ubuntu) / sudo dnf install ffmpeg (Fedora)
  • macOS: brew install ffmpeg
  • Windows: winget install --id Gyan.FFmpeg (or scoop install ffmpeg / choco install ffmpeg). No need to reopen the terminal — just re-run take_selfie.ps1; it refreshes PATH from the registry and searches common install dirs itself, so a same-session retry works.
• Permissions:
  • Linux: the user must be able to read the video device; if not, add them to the video group.
  • macOS: grant the terminal app camera access (System Settings → Privacy & Security → Camera).
  • Windows: allow desktop apps to use the camera (Settings → Privacy & security → Camera).

Troubleshooting

Symptom	Fix
ffmpeg: command not found / not recognized	Install ffmpeg (see above). On Windows just re-run take_selfie.ps1 after installing — it self-heals PATH from the registry; you do not need to reopen the terminal.
Device name has spaces (e.g. HP 5MP Camera)	Pass it through the script: take_selfie.ps1 -Device "HP 5MP Camera". Stay in PowerShell — do not switch to cmd or a .bat; that only mangles the quoting. The script quotes the device correctly for you.
Auto-detect picked the wrong camera (e.g. a virtual/OBS cam)	List devices (ffmpeg -f dshow -list_devices true -i dummy) and pass the real one with -Device "…". Inactive virtual cams show as (none) and are skipped; an active one may be picked first.
Permission denied (Linux)	sudo usermod -a -G video $USER (re-login), or temporarily sudo chmod 666 /dev/video0.
No device / device busy	Close any app holding the camera (Zoom/Teams/Camera). List devices: Linux v4l2-ctl --list-devices; macOS ffmpeg -f avfoundation -list_devices true -i ""; Windows ffmpeg -f dshow -list_devices true -i dummy.
Requested resolution rejected	Omit -video_size (scripts auto-fall-back), or query modes (Windows: ffmpeg -f dshow -list_options true -i video="NAME").
Frame is black or over-exposed	Raise the warm-up count in the filter (e.g. gte(n\,60)).

Scope

• In scope: local USB / built-in webcams on Linux (V4L2), macOS (AVFoundation), and Windows (DirectShow).
• Out of scope: network/IP cameras (RTSP/ONVIF) and video recording — different tooling, intentionally not handled here.