feat(dross): voice Phase 2a — local whisper transcribe + mic (2.12.0)
faster-whisper (small.en, GPU+CPU fallback) on CT 102 → POST /api/voice/transcribe (multer→whisper client) → mic in the bubble records (MediaRecorder), uploads, drops the transcript into the input to review-and-send. Infra scripts in deploy/whisper/. Retention (P2b) next. NOTE: mic needs a secure context (the https domain), not the LAN IP. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
16
deploy/whisper/README.md
Normal file
16
deploy/whisper/README.md
Normal file
@@ -0,0 +1,16 @@
|
||||
# faster-whisper service (Dross voice STT)
|
||||
|
||||
Runs on **CT 102** (the Ollama box, `192.168.1.185`), bare-metal (no Docker), on the
|
||||
RTX A2000 with CPU fallback. OpenAI-style `/transcribe` consumed by void-app
|
||||
`lib/voice/whisper.js` (`WHISPER_URL=http://192.168.1.185:8001`).
|
||||
|
||||
## Install (on CT 102)
|
||||
```
|
||||
scp deploy/whisper/{server.py,setup.sh} root@192.168.1.185:/opt/whisper_server.py /root/setup.sh
|
||||
ssh root@192.168.1.185 'bash /root/setup.sh && install -m644 /opt/whisper_server.py /opt/whisper/server.py && systemctl enable --now whisper'
|
||||
curl http://192.168.1.185:8001/health # {"ok":true,"model":"small.en","device":"cuda"}
|
||||
```
|
||||
- venv at `/opt/whisper/venv`; model `small.en` (env `WHISPER_MODEL`); CUDA libs via
|
||||
`nvidia-cublas-cu12`/`nvidia-cudnn-cu12` pip wheels (LD_LIBRARY_PATH in the unit).
|
||||
- GPU → CPU fallback is in `server.py` `load()`.
|
||||
- **CT 102 disk was expanded +20G** (was 89% full) before install.
|
||||
35
deploy/whisper/server.py
Normal file
35
deploy/whisper/server.py
Normal file
@@ -0,0 +1,35 @@
|
||||
import os, tempfile
|
||||
from fastapi import FastAPI, UploadFile, File, HTTPException
|
||||
from faster_whisper import WhisperModel
|
||||
|
||||
MODEL = os.environ.get("WHISPER_MODEL", "small.en")
|
||||
app = FastAPI()
|
||||
model = None
|
||||
device_used = None
|
||||
|
||||
def load():
|
||||
global model, device_used
|
||||
try:
|
||||
model = WhisperModel(MODEL, device="cuda", compute_type="int8_float16")
|
||||
device_used = "cuda"
|
||||
except Exception:
|
||||
model = WhisperModel(MODEL, device="cpu", compute_type="int8")
|
||||
device_used = "cpu"
|
||||
|
||||
load()
|
||||
|
||||
@app.get("/health")
|
||||
def health():
|
||||
return {"ok": True, "model": MODEL, "device": device_used}
|
||||
|
||||
@app.post("/transcribe")
|
||||
async def transcribe(file: UploadFile = File(...)):
|
||||
data = await file.read()
|
||||
if not data:
|
||||
raise HTTPException(400, "empty audio")
|
||||
with tempfile.NamedTemporaryFile(suffix=".bin") as f:
|
||||
f.write(data); f.flush()
|
||||
segments, info = model.transcribe(f.name, beam_size=1, vad_filter=True)
|
||||
text = "".join(s.text for s in segments).strip()
|
||||
return {"text": text, "language": info.language,
|
||||
"duration": round(info.duration, 2), "device": device_used}
|
||||
26
deploy/whisper/setup.sh
Normal file
26
deploy/whisper/setup.sh
Normal file
@@ -0,0 +1,26 @@
|
||||
#!/usr/bin/env bash
|
||||
set -e
|
||||
export DEBIAN_FRONTEND=noninteractive
|
||||
apt-get update -qq
|
||||
apt-get install -y -qq python3-pip python3-venv ffmpeg >/dev/null
|
||||
mkdir -p /opt/whisper
|
||||
python3 -m venv /opt/whisper/venv
|
||||
/opt/whisper/venv/bin/pip install -q --upgrade pip
|
||||
/opt/whisper/venv/bin/pip install -q faster-whisper fastapi "uvicorn[standard]" python-multipart nvidia-cublas-cu12 nvidia-cudnn-cu12
|
||||
SITE=/opt/whisper/venv/lib/python3.12/site-packages
|
||||
cat > /etc/systemd/system/whisper.service <<UNIT
|
||||
[Unit]
|
||||
Description=faster-whisper transcription server (Dross voice)
|
||||
After=network.target
|
||||
[Service]
|
||||
Type=simple
|
||||
WorkingDirectory=/opt/whisper
|
||||
Environment=WHISPER_MODEL=small.en
|
||||
Environment=LD_LIBRARY_PATH=${SITE}/nvidia/cublas/lib:${SITE}/nvidia/cudnn/lib
|
||||
ExecStart=/opt/whisper/venv/bin/uvicorn server:app --host 0.0.0.0 --port 8001
|
||||
Restart=on-failure
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
UNIT
|
||||
systemctl daemon-reload
|
||||
echo "deps+unit installed"
|
||||
Reference in New Issue
Block a user