Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 19 additions & 6 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -8,12 +8,18 @@ VIBE_STICK_PROVIDER=auto
VIBE_STICK_BRIDGE_TOKEN=
# Set to on only if you want real Claude 5H/7D usage and Claude Code is installed/logged in.
VIBE_STICK_CLAUDE_USAGE=off
# Recommended ASR for China: SiliconFlow, OpenAI-compatible.
# Get a key at https://cloud.siliconflow.cn/i/7ZCoy9fU
VIBE_STICK_ASR_PROVIDER=openai-compatible
VIBE_STICK_ASR_BASE_URL=https://api.siliconflow.cn/v1
# Optional server-side Sub2-Usage source. Keep this token on the Bridge;
# never compile it into the StickS3 firmware.
VIBE_STICK_SUB2_USAGE_TOKEN=
VIBE_STICK_SUB2_USAGE_BASE_URL=https://88api.ai/sub2-usage
VIBE_STICK_SUB2_CODEX_ACCOUNT_ID=52
VIBE_STICK_SUB2_CLAUDE_ACCOUNT_ID=29
VIBE_STICK_SUB2_USAGE_INTERVAL_SECONDS=60
# Recommended for a LAN FunASR server: no API key is required.
VIBE_STICK_ASR_PROVIDER=local-funasr
VIBE_STICK_ASR_BASE_URL=ws://192.168.31.100:10095
VIBE_STICK_ASR_API_KEY=
VIBE_STICK_ASR_MODEL=FunAudioLLM/SenseVoiceSmall
VIBE_STICK_ASR_MODEL=paraformer-zh-streaming

# OPTIONAL -- defaults are fine, leave empty to use defaults.
VIBE_STICK_PROJECT_ROOT=
Expand All @@ -24,6 +30,13 @@ CLAUDE_CODE_OAUTH_TOKEN=
VIBE_STICK_ASR_LANGUAGE=zh
VIBE_STICK_ASR_TIMEOUT_SECONDS=15
VIBE_STICK_ASR_ATTEMPTS=2
VIBE_STICK_FUNASR_HOTWORDS_URL=http://192.168.31.100:10096/api/hotwords
VIBE_STICK_FUNASR_CHUNK_DELAY_MS=2

# Optional SiliconFlow fallback (OpenAI-compatible).
# VIBE_STICK_ASR_PROVIDER=openai-compatible
# VIBE_STICK_ASR_BASE_URL=https://api.siliconflow.cn/v1
# VIBE_STICK_ASR_MODEL=FunAudioLLM/SenseVoiceSmall

# Optional overseas Groq ASR preset.
# VIBE_STICK_ASR_PROVIDER=groq
Expand All @@ -40,6 +53,6 @@ VIBE_STICK_GROQ_LANGUAGE=zh
VIBE_STICK_TRANSCRIBE_CMD=
VIBE_STICK_TRANSCRIBE_TIMEOUT_SECONDS=120

# Recording behavior.
# Recording behavior. Mac microphone capture is macOS-only; StickS3 audio upload works on Windows.
VIBE_STICK_RECORDING_USE_MAC_MIC=1
VIBE_STICK_AUTO_ENTER=0
11 changes: 11 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,16 @@
# Changelog

## v0.1.6 - 2026-07-15

- Added low-latency StickS3-to-FunASR streaming with stable segment locking, tail-only correction, silence filtering, hotwords, and URL/developer-term normalization.
- Added the Windows input-relay protocol used to stream provisional and final text into the focused local or remote-desktop application.
- Added server-side Sub2-Usage polling for Codex/Claude 5-hour and 7-day remaining quota plus account-name display; credentials remain on the Bridge.
- Restored front-button Send/Enter and Codex/Claude task completion, approval, and error sounds, including deferred playback while recording.
- Added BMI270 pickup/flip display control, 60-minute battery idle shutdown, PMIC rail cleanup, and deep-sleep fallback.
- Added Unraid Docker deployment assets, Windows PowerShell helpers, a larger firmware partition, and expanded automated tests.

See [the complete 0.1.6 release record](docs/releases/0.1.6.md).

## v0.1.4

Initial public release of VibeStick — a tiny desktop companion for coding agents on M5Stack StickS3.
Expand Down
43 changes: 40 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,10 @@ VibeStick turns an M5Stack StickS3 into a tiny desktop companion for coding agen

VibeStick targets M5Stack StickS3 hardware and is not an official M5Stack project. Third-party agent names such as Codex and Claude describe compatible local tools and integrations only.

This branch can fetch Codex and Claude 5H/7D remaining quota through Sub2-Usage on the server-side Bridge. The token never enters the firmware; StickS3 shows `account_name` when available and falls back to the configured account id.

Current integrated firmware/Bridge record: [VibeStick 0.1.6](docs/releases/0.1.6.md).

## What you'll need (prepare first)

- [ ] M5Stack StickS3 and a USB-C data cable.
Expand All @@ -24,6 +28,21 @@ Building the firmware needs ESP-IDF v5.5.x — a one-time toolchain install (~1

You can do this manually, or hand the command steps to an AI coding agent such as Claude Code and Codex.

### Windows bridge quick start

Windows supports the Python bridge, local Codex / Claude observation, StickS3 audio uploads, and automatic paste into the focused window. The macOS HUD and Mac microphone fallback remain macOS-only; on Windows, voice input uses audio uploaded by the StickS3.

Run in PowerShell:

```powershell
git clone https://github.com/GaryGaryyy/VibeStick.git
cd VibeStick
.\scripts\setup.ps1
.\scripts\dev.ps1
```

Edit Wi-Fi values in `firmware\sticks3\include\vibe_stick_secrets.h` and the ASR key in `.env` before using the device. Allow private-network access if Windows Firewall prompts, then open `http://127.0.0.1:8765/health` to verify the bridge. Firmware builds still require ESP-IDF v5.5.x; use Espressif's Windows Installation Manager and run `idf.py build flash` from an ESP-IDF PowerShell.

> Legend: steps marked 👤 are PHYSICAL steps that need a human to act directly, such as plugging in the cable, long-pressing or short-pressing the power button, and granting macOS permissions in System Settings. AI agents should run the shell steps in order, then pause at each 👤 step and ask the user to complete it before continuing.

1. Clone the repo and create local config files:
Expand Down Expand Up @@ -169,7 +188,25 @@ Empty values in `.env` generally mean "use the built-in default". `scripts/dev.s
- `VIBE_STICK_RECORDING_USE_MAC_MIC`: set to `0` to disable Mac microphone fallback.
- `VIBE_STICK_AUTO_ENTER`: set to `1` to press Return after pasting.

### ASR option 1: SiliconFlow (recommended default)
### StickS3 power behavior

- The screen backlight turns off after 30 seconds; this is standby, not power-off.
- On battery, 60 minutes without interaction triggers PMIC power-off. Wi-Fi, audio, LCD and the unused IMU rail are released first; deep sleep is used if the PMIC rail doesn't cut successfully.
- Double-click the side reset/power button for hardware power-off. Long-pressing it enters download mode and still consumes power while waiting to be flashed.

### ASR option 1: LAN FunASR (lowest latency)

This sends the completed StickS3 WAV to a FunASR WebSocket service on the LAN. It needs no API key and can load hotwords and normalization rules from the companion hotword API.

```sh
VIBE_STICK_ASR_PROVIDER=local-funasr
VIBE_STICK_ASR_BASE_URL=ws://192.168.31.100:10095
VIBE_STICK_FUNASR_HOTWORDS_URL=http://192.168.31.100:10096/api/hotwords
VIBE_STICK_FUNASR_CHUNK_DELAY_MS=2
VIBE_STICK_ASR_LANGUAGE=zh
```

### ASR option 2: SiliconFlow

```sh
VIBE_STICK_ASR_PROVIDER=openai-compatible
Expand All @@ -183,7 +220,7 @@ VIBE_STICK_ASR_ATTEMPTS=2

Audio sent to a cloud ASR provider leaves the Mac.

### ASR option 2: any OpenAI-compatible provider
### ASR option 3: any OpenAI-compatible provider

Use any provider that accepts `POST {base_url}/audio/transcriptions`.

Expand All @@ -203,7 +240,7 @@ VIBE_STICK_ASR_API_KEY=your-groq-key

The legacy aliases `VIBE_STICK_GROQ_API_KEY`, `VIBE_STICK_GROQ_MODEL`, and `VIBE_STICK_GROQ_LANGUAGE` remain supported.

### ASR option 3: local command (offline)
### ASR option 4: local command (offline)

```sh
VIBE_STICK_TRANSCRIBE_CMD=/path/to/transcribe-command
Expand Down
50 changes: 47 additions & 3 deletions README.zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,10 @@ VibeStick 把 M5Stack StickS3 变成一个桌面 AI agent 小终端:显示状

VibeStick 面向 M5Stack StickS3,不是 M5Stack 官方项目。Codex、Claude 等第三方 agent 名称只用于说明本地兼容工具和集成。

本分支支持由 Bridge 在服务器端调用 Sub2-Usage,缓存 Codex 与 Claude 的 5H/7D 剩余额度后再发送给 StickS3。Token 不进入固件;固件会优先显示接口的 `account_name`,字段暂缺时显示账号编号。

当前固件与 Bridge 的完整更新记录:[VibeStick 0.1.6](docs/releases/0.1.6.md)。

## 开始前的准备

- [ ] M5 StickS3|一根 USB-C 数据线|一台电脑(最好是Mac)
Expand All @@ -22,6 +26,28 @@ VibeStick 面向 M5Stack StickS3,不是 M5Stack 官方项目。Codex、Claude

你可以手动执行,也可以交给 AI 编程 agent,例如 Claude Code 和 Codex。

### Windows bridge 快速启动

Windows 支持运行 Python bridge、读取本机 Codex / Claude 状态、接收 StickS3 上传的音频,以及把转写结果粘贴到当前窗口。macOS HUD 和 Mac 本机麦克风回退仍然仅限 macOS;Windows 使用 StickS3 自带麦克风上传的音频。

在 PowerShell 中运行:

```powershell
git clone https://github.com/GaryGaryyy/VibeStick.git
cd VibeStick
.\scripts\setup.ps1
```

然后填写 `firmware\sticks3\include\vibe_stick_secrets.h` 中的 Wi-Fi 信息,并在 `.env` 中填写 ASR API key。启动 bridge:

```powershell
.\scripts\dev.ps1
```

Windows 防火墙首次询问时,需要允许专用网络访问,否则 StickS3 无法连接电脑的 8765 端口。可在浏览器打开 `http://127.0.0.1:8765/health` 检查 bridge;看到 `"ok": true` 表示服务已启动。

固件仍需 ESP-IDF v5.5.x。Windows 可使用 Espressif 官方 ESP-IDF Installation Manager 安装工具链,然后在 ESP-IDF PowerShell 中进入 `firmware\sticks3` 执行 `idf.py build flash`。

> 说明:标 👤 的步骤是需要人亲自动手的物理操作,例如插线、长按/短按电源键、在系统设置里授权。AI agent 请按顺序执行 shell 步骤,执行到 👤 步骤时暂停,让用户完成后再继续。

1. 克隆仓库并创建本地配置文件:
Expand Down Expand Up @@ -167,7 +193,25 @@ open -e .env
- `VIBE_STICK_RECORDING_USE_MAC_MIC`:设为 `0` 可关闭 Mac 麦克风兜底。
- `VIBE_STICK_AUTO_ENTER`:设为 `1` 会在粘贴后自动按 Return。

### ASR 方案 1:SiliconFlow(默认推荐)
### StickS3 电源行为

- 屏幕背光会在 30 秒后熄灭;这只是待机,不是关机。
- 使用电池时,连续 60 分钟无操作会触发 PMIC 关机。关机前先释放 Wi-Fi、音频、LCD 和未使用的 IMU 电源轨;若 PMIC 没有真正切断 ESP32-S3 电源,则自动进入深度睡眠兜底。
- 双击侧面的复位/电源键才是硬件关机。长按会进入下载模式,等待刷机期间仍会耗电。

### ASR 方案 1:局域网 FunASR(最低延迟)

StickS3 录音结束后,bridge 会把 WAV 直接发送到局域网 FunASR WebSocket 服务。无需 API Key,并可从配套接口定期读取热词和标准化规则。

```sh
VIBE_STICK_ASR_PROVIDER=local-funasr
VIBE_STICK_ASR_BASE_URL=ws://192.168.31.100:10095
VIBE_STICK_FUNASR_HOTWORDS_URL=http://192.168.31.100:10096/api/hotwords
VIBE_STICK_FUNASR_CHUNK_DELAY_MS=2
VIBE_STICK_ASR_LANGUAGE=zh
```

### ASR 方案 2:SiliconFlow

```sh
VIBE_STICK_ASR_PROVIDER=openai-compatible
Expand All @@ -181,7 +225,7 @@ VIBE_STICK_ASR_ATTEMPTS=2

使用云端 ASR 时,音频会离开本机 Mac。

### ASR 方案 2:任意 OpenAI 兼容服务
### ASR 方案 3:任意 OpenAI 兼容服务

只要服务支持 `POST {base_url}/audio/transcriptions` 即可。

Expand All @@ -201,7 +245,7 @@ VIBE_STICK_ASR_API_KEY=your-groq-key

旧别名 `VIBE_STICK_GROQ_API_KEY`、`VIBE_STICK_GROQ_MODEL`、`VIBE_STICK_GROQ_LANGUAGE` 仍然支持。

### ASR 方案 3:本地命令(离线)
### ASR 方案 4:本地命令(离线)

```sh
VIBE_STICK_TRANSCRIBE_CMD=/path/to/transcribe-command
Expand Down
81 changes: 81 additions & 0 deletions bridge/deploy/unraid/apply-sub2-update.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
#!/bin/bash
set -euo pipefail

if [[ -z "${SUB2_TOKEN:-}" ]]; then
echo "SUB2_TOKEN is required" >&2
exit 2
fi

APP=/mnt/user/appdata/FunASR-2pass/gateway
TEMPLATE=/boot/config/plugins/dockerMan/templates-user/my-Vibe-ASR-Gateway.xml
WORK=/www/wwwroot/work
TIMESTAMP=$(date +%Y%m%d-%H%M%S)
BACKUP="$WORK/backups/vibestick-sub2/$TIMESTAMP"
RELEASE="$WORK/vibestick-sub2-release"
ARCHIVE="$WORK/vibestick-sub2-release.tar.gz"

backup_file() {
local source=$1
if [[ -f "$source" ]]; then
local destination="$BACKUP$source"
mkdir -p "$(dirname "$destination")"
cp -a "$source" "$destination"
fi
}

mkdir -p "$BACKUP"
backup_file "$APP/src/vibe_stick/__init__.py"
backup_file "$APP/src/vibe_stick/protocol/state.py"
backup_file "$APP/src/vibe_stick/providers/base.py"
backup_file "$APP/src/vibe_stick/server/app.py"
backup_file "$APP/start.sh"
backup_file "$TEMPLATE"

rm -rf "$RELEASE"
mkdir -p "$RELEASE"
tar -xzf "$ARCHIVE" -C "$RELEASE"

install -d "$APP/src/vibe_stick/usage"
install -m 0644 "$RELEASE/bridge/src/vibe_stick/__init__.py" "$APP/src/vibe_stick/__init__.py"
install -m 0644 "$RELEASE/bridge/src/vibe_stick/protocol/state.py" "$APP/src/vibe_stick/protocol/state.py"
install -m 0644 "$RELEASE/bridge/src/vibe_stick/providers/base.py" "$APP/src/vibe_stick/providers/base.py"
install -m 0644 "$RELEASE/bridge/src/vibe_stick/server/app.py" "$APP/src/vibe_stick/server/app.py"
install -m 0644 "$RELEASE/bridge/src/vibe_stick/usage/__init__.py" "$APP/src/vibe_stick/usage/__init__.py"
install -m 0644 "$RELEASE/bridge/src/vibe_stick/usage/sub2.py" "$APP/src/vibe_stick/usage/sub2.py"
install -m 0755 "$RELEASE/bridge/deploy/unraid/start.sh" "$APP/start.sh"

umask 077
printf '%s\n' \
"VIBE_STICK_SUB2_USAGE_TOKEN=$SUB2_TOKEN" \
'VIBE_STICK_SUB2_USAGE_BASE_URL=https://88api.ai/sub2-usage' \
'VIBE_STICK_SUB2_CODEX_ACCOUNT_ID=52' \
'VIBE_STICK_SUB2_CLAUDE_ACCOUNT_ID=29' \
'VIBE_STICK_SUB2_USAGE_INTERVAL_SECONDS=60' \
> "$APP/sub2-usage.env"

if ! grep -q 'VIBE_STICK_SUB2_USAGE_TOKEN' "$TEMPLATE"; then
awk -v token="$SUB2_TOKEN" '
{ print }
/Target="VIBE_STICK_BRIDGE_TOKEN"/ {
print " <Config Name=\"Sub2 Usage Token\" Target=\"VIBE_STICK_SUB2_USAGE_TOKEN\" Default=\"\" Mode=\"\" Description=\"Server-side Codex and Claude quota token\" Type=\"Variable\" Display=\"always\" Required=\"false\" Mask=\"true\">" token "</Config>"
print " <Config Name=\"Sub2 Usage Base URL\" Target=\"VIBE_STICK_SUB2_USAGE_BASE_URL\" Default=\"https://88api.ai/sub2-usage\" Mode=\"\" Description=\"Sub2-Usage API base URL\" Type=\"Variable\" Display=\"advanced\" Required=\"true\" Mask=\"false\">https://88api.ai/sub2-usage</Config>"
print " <Config Name=\"Codex Usage Account\" Target=\"VIBE_STICK_SUB2_CODEX_ACCOUNT_ID\" Default=\"52\" Mode=\"\" Description=\"Sub2 gpt-plus account id\" Type=\"Variable\" Display=\"advanced\" Required=\"true\" Mask=\"false\">52</Config>"
print " <Config Name=\"Claude Usage Account\" Target=\"VIBE_STICK_SUB2_CLAUDE_ACCOUNT_ID\" Default=\"29\" Mode=\"\" Description=\"Sub2 Claude account id\" Type=\"Variable\" Display=\"advanced\" Required=\"true\" Mask=\"false\">29</Config>"
print " <Config Name=\"Usage Refresh Seconds\" Target=\"VIBE_STICK_SUB2_USAGE_INTERVAL_SECONDS\" Default=\"60\" Mode=\"\" Description=\"Background quota refresh interval\" Type=\"Variable\" Display=\"advanced\" Required=\"true\" Mask=\"false\">60</Config>"
}
' "$TEMPLATE" > "$WORK/my-Vibe-ASR-Gateway.xml.new"
install -m 0600 "$WORK/my-Vibe-ASR-Gateway.xml.new" "$TEMPLATE"
rm -f "$WORK/my-Vibe-ASR-Gateway.xml.new"
else
sed -i -E "s#^(.*Target=\"VIBE_STICK_SUB2_USAGE_TOKEN\"[^>]*>)[^<]*(</Config>)#\1${SUB2_TOKEN}\2#" "$TEMPLATE"
fi

docker restart Vibe-ASR-Gateway >/dev/null
sleep 6
echo "backup=$BACKUP"
printf 'health='
curl -fsS http://127.0.0.1:8765/health
printf '\n'

rm -rf "$RELEASE"
rm -f "$ARCHIVE" "$WORK/apply-sub2-update.sh"
44 changes: 44 additions & 0 deletions bridge/deploy/unraid/apply-task-events-update.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
#!/bin/bash
set -euo pipefail

APP=/mnt/user/appdata/FunASR-2pass/gateway
WORK=/www/wwwroot/work
TIMESTAMP=$(date +%Y%m%d-%H%M%S)
BACKUP="$WORK/backups/vibestick-task-events/$TIMESTAMP"
RELEASE="$WORK/vibestick-task-events-release"
ARCHIVE="$WORK/vibestick-task-events-release.tar.gz"

backup_file() {
local source=$1
if [[ -f "$source" ]]; then
local destination="$BACKUP$source"
mkdir -p "$(dirname "$destination")"
cp -a "$source" "$destination"
fi
}

mkdir -p "$BACKUP"
backup_file "$APP/src/vibe_stick/__init__.py"
backup_file "$APP/src/vibe_stick/paste/input_relay.py"
backup_file "$APP/src/vibe_stick/server/app.py"

rm -rf "$RELEASE"
mkdir -p "$RELEASE"
tar -xzf "$ARCHIVE" -C "$RELEASE"

install -m 0644 "$RELEASE/bridge/src/vibe_stick/__init__.py" \
"$APP/src/vibe_stick/__init__.py"
install -m 0644 "$RELEASE/bridge/src/vibe_stick/paste/input_relay.py" \
"$APP/src/vibe_stick/paste/input_relay.py"
install -m 0644 "$RELEASE/bridge/src/vibe_stick/server/app.py" \
"$APP/src/vibe_stick/server/app.py"

docker restart Vibe-ASR-Gateway >/dev/null
sleep 6
echo "backup=$BACKUP"
printf 'health='
curl -fsS http://127.0.0.1:8765/health
printf '\n'

rm -rf "$RELEASE"
rm -f "$ARCHIVE" "$WORK/apply-task-events-update.sh"
Loading