docs(dross): floating Dross chat design spec + mockup
Brainstormed design: global floating bubble companion (replaces the per-Space right rail), draggable orb+panel, bottom collapse + top close, 3 selectable violet avatars, tunable persona, local faster-whisper voice (CT 102) review-and-send. Phased P1 UI/global → P2 voice → P3 modes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
root
221
docs/mockups/dross-chat.html
Normal file
221
docs/mockups/dross-chat.html
Normal file
@@ -0,0 +1,221 @@
|
||||
<!doctype html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="utf-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=1">
|
||||
<title>Dross floating chat — mockup v2</title>
|
||||
<link rel="preconnect" href="https://fonts.googleapis.com">
|
||||
<link href="https://fonts.googleapis.com/css2?family=Cinzel:wght@500;600&family=Cormorant+Garamond:ital@0;1&family=JetBrains+Mono&display=swap" rel="stylesheet">
|
||||
<style>
|
||||
:root{
|
||||
--bg:#0a0a0e; --panel:#14141c; --panel-2:#1c1c26; --border:#2a2a36;
|
||||
--text:#e8e6ed; --muted:#888094; --accent:#ff4f2e;
|
||||
--dross:#a86adf; --dross-dim:#5a2e8a; --dross-soft:#1e1030; --dross-glow:#c79bff;
|
||||
--font-display:'Cinzel',serif; --font-body:'Cormorant Garamond',serif; --font-mono:'JetBrains Mono',monospace;
|
||||
}
|
||||
*{box-sizing:border-box;margin:0;padding:0}
|
||||
html,body{height:100%}
|
||||
body{background:radial-gradient(80% 60% at 50% 0%, #16131b 0%, #0a0a0e 60%),var(--bg);color:var(--text);
|
||||
font-family:var(--font-mono);min-height:100%;overflow-x:hidden;padding:18px 16px 120px}
|
||||
h2{font-family:var(--font-display);letter-spacing:.16em;text-transform:uppercase;font-size:13px;color:#cdbbe6;margin:6px 0 4px}
|
||||
.sub{color:var(--muted);font-size:11px;margin-bottom:14px}
|
||||
|
||||
/* ---------- avatar options ---------- */
|
||||
.avs{display:grid;grid-template-columns:repeat(2,1fr);gap:14px;margin-bottom:26px}
|
||||
.avcard{border:1px solid var(--border);border-radius:14px;padding:16px 10px 12px;display:flex;flex-direction:column;align-items:center;gap:9px;
|
||||
background:linear-gradient(160deg,#16131a,#0f0d12)}
|
||||
.avname{font-family:var(--font-mono);font-size:11px;color:#cdbbe6;letter-spacing:.05em}
|
||||
.avdesc{font-family:var(--font-body);font-size:13px;color:var(--muted);text-align:center;line-height:1.25}
|
||||
.orb{width:62px;height:62px;border-radius:50%;position:relative;flex:none;
|
||||
background:radial-gradient(circle at 38% 30%, #2a1640, #1a0f2a 70%, #120a1e);
|
||||
box-shadow:0 0 0 1px #ffffff12, 0 6px 22px -6px #000, 0 0 26px -4px var(--dross-dim);
|
||||
display:grid;place-items:center;overflow:hidden;animation:bob 5s ease-in-out infinite}
|
||||
@keyframes bob{0%,100%{transform:translateY(0)}50%{transform:translateY(-4px)}}
|
||||
|
||||
/* A — soft eye (friendlier) */
|
||||
.a-eye{width:34px;height:34px;border-radius:50%;background:radial-gradient(circle at 50% 40%, #2a1c3a, #140b20);
|
||||
display:grid;place-items:center;box-shadow:inset 0 0 10px #000}
|
||||
.a-pupil{width:15px;height:15px;border-radius:50%;position:relative;
|
||||
background:radial-gradient(circle at 38% 32%, #fff, var(--dross-glow) 50%, var(--dross) 100%);
|
||||
box-shadow:0 0 10px var(--dross-glow);animation:look 7s ease-in-out infinite}
|
||||
.a-pupil::after{content:"";position:absolute;right:2px;bottom:3px;width:4px;height:4px;border-radius:50%;background:#fff;opacity:.8}
|
||||
@keyframes look{0%,45%{transform:translate(0,0)}58%{transform:translate(4px,-2px)}72%{transform:translate(-3px,1px)}88%,100%{transform:translate(0,0)}}
|
||||
|
||||
/* B — wisp / plasma core */
|
||||
.b-core{position:absolute;inset:8px;border-radius:50%;
|
||||
background:conic-gradient(from 0deg, var(--dross-dim), var(--dross-glow), var(--dross), var(--dross-soft), var(--dross-dim));
|
||||
filter:blur(3px);animation:spin 7s linear infinite}
|
||||
.b-bright{position:absolute;inset:20px;border-radius:50%;background:radial-gradient(circle,#fff,var(--dross-glow) 60%,transparent 75%);
|
||||
animation:pulse 3s ease-in-out infinite}
|
||||
@keyframes spin{to{transform:rotate(360deg)}}
|
||||
@keyframes pulse{0%,100%{opacity:.55;transform:scale(.9)}50%{opacity:1;transform:scale(1.1)}}
|
||||
|
||||
/* C — rune sigil */
|
||||
.c-sigil{width:30px;height:30px;filter:drop-shadow(0 0 6px var(--dross-glow));animation:sigil 8s ease-in-out infinite}
|
||||
@keyframes sigil{0%,100%{opacity:.7;transform:rotate(0)}50%{opacity:1;transform:rotate(20deg)}}
|
||||
|
||||
/* D — orbiting motes */
|
||||
.d-core{width:14px;height:14px;border-radius:50%;background:radial-gradient(circle,#fff,var(--dross-glow) 60%,var(--dross));box-shadow:0 0 12px var(--dross-glow)}
|
||||
.d-ring{position:absolute;inset:0;animation:spin 5s linear infinite}
|
||||
.d-ring.r2{animation-duration:8s;animation-direction:reverse}
|
||||
.d-mote{position:absolute;top:7px;left:50%;width:7px;height:7px;margin-left:-3.5px;border-radius:50%;background:var(--dross-glow);box-shadow:0 0 8px var(--dross-glow)}
|
||||
.d-ring.r2 .d-mote{top:auto;bottom:9px;background:var(--dross);width:5px;height:5px}
|
||||
|
||||
/* ---------- live orb (bottom-right, draggable) ---------- */
|
||||
#live{position:fixed;right:20px;bottom:20px;cursor:grab;z-index:40;touch-action:none}
|
||||
#live:active{cursor:grabbing}
|
||||
.ping{position:absolute;right:-2px;top:-2px;width:17px;height:17px;border-radius:50%;background:var(--accent);
|
||||
color:#0a0a0e;font-size:10px;display:grid;place-items:center;box-shadow:0 0 0 2px var(--bg);z-index:2}
|
||||
|
||||
/* ---------- panel ---------- */
|
||||
.panel{position:fixed;right:20px;bottom:20px;width:340px;max-width:calc(100vw - 24px);height:480px;max-height:calc(100vh - 24px);
|
||||
display:none;flex-direction:column;z-index:41;border:1px solid var(--dross-dim);border-radius:16px;overflow:hidden;
|
||||
background:linear-gradient(180deg, rgba(30,16,48,.6), rgba(20,20,28,.96) 22%);
|
||||
box-shadow:0 24px 70px -18px #000, 0 0 0 1px #00000060, 0 0 40px -16px var(--dross-dim);backdrop-filter:blur(6px)}
|
||||
.panel.open{display:flex;animation:rise .18s ease}
|
||||
@keyframes rise{from{opacity:0;transform:translateY(8px) scale(.98)}to{opacity:1;transform:none}}
|
||||
.hd{display:flex;align-items:center;gap:10px;padding:11px 12px;cursor:grab;
|
||||
background:linear-gradient(180deg, var(--dross-soft), transparent);border-bottom:1px solid var(--border);touch-action:none}
|
||||
.mini{width:30px;height:30px;border-radius:50%;flex:none;position:relative;overflow:hidden;
|
||||
background:radial-gradient(circle at 38% 30%, #2a1640, #160d24)}
|
||||
.mini .b-core{inset:4px;filter:blur(2px)}.mini .b-bright{inset:11px}
|
||||
.who{font-family:var(--font-display);letter-spacing:.16em;text-transform:uppercase;font-size:13px;color:#efe9f6;flex:1}
|
||||
.who small{display:block;font-family:var(--font-mono);letter-spacing:0;text-transform:none;font-size:10px;color:var(--dross-glow);opacity:.85}
|
||||
.xbtn{background:none;border:0;color:var(--muted);font-size:18px;cursor:pointer;padding:4px 6px}
|
||||
.xbtn:hover{color:var(--text)}
|
||||
.log{flex:1;overflow:auto;padding:14px 13px;display:flex;flex-direction:column;gap:11px}
|
||||
.msg{max-width:88%;padding:9px 12px;border-radius:13px;font-family:var(--font-body);font-size:16px;line-height:1.35}
|
||||
.msg.d{align-self:flex-start;background:var(--dross-soft);border:1px solid var(--dross-dim);border-bottom-left-radius:4px;color:#efe9f6}
|
||||
.msg.u{align-self:flex-end;background:var(--panel-2);border:1px solid var(--border);border-bottom-right-radius:4px}
|
||||
.msg .nm{font-family:var(--font-mono);font-size:9px;letter-spacing:.14em;text-transform:uppercase;color:var(--dross-glow);opacity:.7;margin-bottom:2px}
|
||||
|
||||
/* input: textarea on top, big mic + send BELOW */
|
||||
.inwrap{padding:10px;border-top:1px solid var(--border);background:#0d0a12;display:flex;flex-direction:column;gap:9px}
|
||||
textarea{width:100%;background:#0d0d13;border:1px solid var(--border);border-radius:10px;padding:10px 12px;color:var(--text);
|
||||
font-family:var(--font-mono);font-size:13px;resize:none;height:46px;max-height:96px}
|
||||
.btnrow{display:flex;gap:10px}
|
||||
.mic{flex:1;height:50px;border-radius:12px;border:1px solid var(--dross-dim);background:var(--dross-soft);color:var(--dross-glow);
|
||||
cursor:pointer;display:flex;align-items:center;justify-content:center;gap:9px;font-family:var(--font-ui),sans-serif;font-size:13px;letter-spacing:.02em}
|
||||
.mic:hover{border-color:var(--dross)}
|
||||
.mic.rec{background:#3a1010;border-color:var(--accent);color:#fff;animation:recpulse 1.2s infinite}
|
||||
@keyframes recpulse{0%,100%{box-shadow:0 0 0 0 rgba(255,79,46,.5)}50%{box-shadow:0 0 0 8px rgba(255,79,46,0)}}
|
||||
.send{width:64px;height:50px;border-radius:12px;border:1px solid var(--dross-dim);background:linear-gradient(180deg,var(--dross),var(--dross-dim));
|
||||
color:#fff;cursor:pointer;display:grid;place-items:center}
|
||||
.send:hover{filter:brightness(1.1)}
|
||||
/* bottom collapse handle — thumb-friendly minimise-to-orb */
|
||||
.collapsebar{display:flex;align-items:center;justify-content:center;gap:8px;height:34px;cursor:pointer;
|
||||
color:var(--muted);font-family:var(--font-ui),sans-serif;font-size:11px;letter-spacing:.12em;text-transform:uppercase;
|
||||
background:#0b0810;border-top:1px solid var(--border)}
|
||||
.collapsebar:hover{color:var(--dross-glow)}
|
||||
.collapsebar .grip{width:42px;height:4px;border-radius:3px;background:var(--border)}
|
||||
.wave{display:flex;gap:3px;align-items:center;height:16px}
|
||||
.wave span{width:3px;background:#fff;border-radius:2px;animation:wv .8s ease-in-out infinite}
|
||||
@keyframes wv{0%,100%{height:4px}50%{height:15px}}
|
||||
svg{display:block}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<h2>Pick Dross's look</h2>
|
||||
<div class="sub">Four takes on the orb — all violet, all his. Which feels right?</div>
|
||||
<div class="avs">
|
||||
<div class="avcard">
|
||||
<div class="orb"><div class="a-eye"><div class="a-pupil"></div></div></div>
|
||||
<div class="avname">A · Soft Eye</div>
|
||||
<div class="avdesc">The eye, softened — rounder, a glint, a calmer gaze. Still true to character, less stare.</div>
|
||||
</div>
|
||||
<div class="avcard">
|
||||
<div class="orb"><div class="b-core"></div><div class="b-bright"></div></div>
|
||||
<div class="avname">B · Wisp Core</div>
|
||||
<div class="avdesc">A swirling violet madra core. No eye — a contained spirit. Abstract & mystical.</div>
|
||||
</div>
|
||||
<div class="avcard">
|
||||
<div class="orb"><svg class="c-sigil" viewBox="0 0 32 32" fill="none" stroke="#c79bff" stroke-width="1.6">
|
||||
<path d="M16 2 L20 12 L30 16 L20 20 L16 30 L12 20 L2 16 L12 12 Z"/><circle cx="16" cy="16" r="3" fill="#c79bff" stroke="none"/></svg></div>
|
||||
<div class="avname">C · Rune Sigil</div>
|
||||
<div class="avdesc">A glowing arcane glyph that slowly turns. Reads as "a power", not a face.</div>
|
||||
</div>
|
||||
<div class="avcard">
|
||||
<div class="orb"><div class="d-ring"><div class="d-mote"></div></div><div class="d-ring r2"><div class="d-mote"></div></div><div class="d-core"></div></div>
|
||||
<div class="avname">D · Orbiting Motes</div>
|
||||
<div class="avdesc">A bright core with motes circling it. Lively, restless — feels alive & busy.</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<h2>The chat (revised)</h2>
|
||||
<div class="sub">New mic icon · bigger mic + send below the input · opens anchored to the orb. The live orb is bottom-right — drag it, tap to open.</div>
|
||||
|
||||
<!-- live orb (Wisp by default) -->
|
||||
<div id="live"><div class="ping">2</div><div class="orb" style="animation:none"><div class="b-core"></div><div class="b-bright"></div></div></div>
|
||||
|
||||
<div class="panel" id="panel">
|
||||
<div class="hd" id="hd">
|
||||
<div class="mini"><div class="b-core"></div><div class="b-bright"></div></div>
|
||||
<div class="who">Dross <small>always here, regrettably</small></div>
|
||||
<button class="xbtn" id="close">⤬</button>
|
||||
</div>
|
||||
<div class="log">
|
||||
<div class="msg d"><div class="nm">Dross</div>Back already? Your CPU graphs and I were just getting acquainted. Thrilling curves. What do you need?</div>
|
||||
<div class="msg u">how's the farm backup</div>
|
||||
<div class="msg d"><div class="nm">Dross</div>Two days ago — 2.5 gigs, landed on Won, didn't fall over. I'd have woken you otherwise. <span style="opacity:.75">Per-guest breakdown, or shall we keep trusting the universe?</span></div>
|
||||
</div>
|
||||
<div class="inwrap">
|
||||
<textarea id="ta" placeholder="Ask Dross…"></textarea>
|
||||
<div class="btnrow">
|
||||
<button class="mic" id="mic">
|
||||
<svg id="micicon" viewBox="0 0 24 24" width="22" height="22" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round">
|
||||
<rect x="9" y="3" width="6" height="11" rx="3"/><path d="M5 11a7 7 0 0 0 14 0"/><line x1="12" y1="18" x2="12" y2="22"/><line x1="8" y1="22" x2="16" y2="22"/></svg>
|
||||
<span id="miclabel">Hold to talk</span>
|
||||
</button>
|
||||
<button class="send">
|
||||
<svg viewBox="0 0 24 24" width="20" height="20" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M22 2 11 13M22 2l-7 20-4-9-9-4 20-7z"/></svg>
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
<div class="collapsebar" id="collapse" title="Collapse Dross">
|
||||
<span class="grip"></span><span>⌄ collapse</span><span class="grip"></span>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<script>
|
||||
const live=document.getElementById('live'), panel=document.getElementById('panel');
|
||||
function openPanel(){
|
||||
const r=live.getBoundingClientRect();
|
||||
panel.classList.add('open'); live.style.display='none';
|
||||
const pr=panel.getBoundingClientRect();
|
||||
// anchor the panel's bottom-right to roughly where the orb was
|
||||
let left=Math.max(8, Math.min(r.right-pr.width, innerWidth-pr.width-8));
|
||||
let top =Math.max(8, Math.min(r.bottom-pr.height, innerHeight-pr.height-8));
|
||||
panel.style.right='auto'; panel.style.bottom='auto'; panel.style.left=left+'px'; panel.style.top=top+'px';
|
||||
}
|
||||
function closePanel(){ panel.classList.remove('open'); live.style.display='block'; }
|
||||
live.addEventListener('click',()=>{ if(live._moved){live._moved=false;return;} openPanel(); });
|
||||
document.getElementById('close').addEventListener('click',closePanel);
|
||||
document.getElementById('collapse').addEventListener('click',closePanel);
|
||||
|
||||
// mic recording sim
|
||||
const mic=document.getElementById('mic'), label=document.getElementById('miclabel'), ta=document.getElementById('ta'), icon=document.getElementById('micicon');
|
||||
let rec=false;
|
||||
mic.addEventListener('click',()=>{
|
||||
rec=!rec; mic.classList.toggle('rec',rec);
|
||||
if(rec){ label.innerHTML='<span class="wave">'+Array(16).fill('<span></span>').join('')+'</span> 0:03'; icon.style.display='none'; }
|
||||
else { icon.style.display='block'; label.textContent='Hold to talk'; ta.value="what's eating the most disk on the media stack right now"; ta.focus(); }
|
||||
});
|
||||
|
||||
// drag helper
|
||||
function drag(handle,target,isOrb){
|
||||
handle.addEventListener('pointerdown',e=>{
|
||||
if(e.target.closest('.xbtn')||e.target.closest('.mic')||e.target.closest('.send')) return;
|
||||
e.preventDefault();
|
||||
const r=target.getBoundingClientRect(); const sx=e.clientX,sy=e.clientY; let moved=false;
|
||||
target.style.right='auto';target.style.bottom='auto';target.style.left=r.left+'px';target.style.top=r.top+'px';
|
||||
const mv=ev=>{const dx=ev.clientX-sx,dy=ev.clientY-sy; if(Math.abs(dx)+Math.abs(dy)>4)moved=true;
|
||||
target.style.left=Math.max(4,Math.min(innerWidth-r.width-4,r.left+dx))+'px';
|
||||
target.style.top=Math.max(4,Math.min(innerHeight-r.height-4,r.top+dy))+'px';};
|
||||
const up=()=>{document.removeEventListener('pointermove',mv);document.removeEventListener('pointerup',up); if(isOrb)live._moved=moved;};
|
||||
document.addEventListener('pointermove',mv);document.addEventListener('pointerup',up);
|
||||
});
|
||||
}
|
||||
drag(live,live,true); drag(document.getElementById('hd'),panel,false);
|
||||
</script>
|
||||
</body>
|
||||
</html>
|
||||
@@ -0,0 +1,94 @@
|
||||
# Floating Dross Chat — Design
|
||||
|
||||
**Date:** 2026-06-09
|
||||
**Status:** Approved (pending final spec sign-off)
|
||||
**Goal:** Replace the docked, per-Space "Cradle Chat" with a global, movable floating-bubble Dross companion — mobile-first, with voice-clip input transcribed locally into instructions.
|
||||
|
||||
---
|
||||
|
||||
## Background / problem
|
||||
|
||||
Today the companion lives in the right rail (`public/components/rightrail.js`). It is **per-Space**: it binds to the active Space's companion conversation (`/api/spaces/:space_id/companion`) and shows "Open a Space to chat with its companion." everywhere else — Sacred Valley, the apps, etc. Because the user mostly lives on non-Space views, the chat is empty/collapsed most of the time, which is why it "feels closed and not very Dross." The right rail is also cramped on mobile.
|
||||
|
||||
The chat mechanics are already factored into a reusable engine (`public/components/agent_chat.js`, `wireAgentChat({logEl, inputEl, historyUrl, turnUrl, …})`). Turns stream over SSE via `lib/ai/agent/run_turn.js`. Dross is the agent with slug `companion`.
|
||||
|
||||
## Locked decisions (from brainstorming)
|
||||
|
||||
1. **Global Dross** — one always-available companion, summonable on every view; not tied to a Space. He is told what the user is currently looking at (view context) but isn't locked to it.
|
||||
2. **Floating bubble** — a draggable violet orb that opens a draggable chat panel anchored to the orb. Replaces the right-rail companion. Position + open/closed state persist. Mobile = near-full-width panel.
|
||||
3. **Collapse / close** — keep the **close (✕) top-right**, and add a thumb-friendly **"⌄ collapse" bar at the bottom** of the panel. Both minimise back to the orb.
|
||||
4. **Avatar** — default **Soft Eye**; selectable in Settings between **Soft Eye**, **Wisp Core**, **Orbiting Motes** (all violet).
|
||||
5. **Colour** — Dross is **violet** by default, but his accent is **tunable in Settings** (his own vars, independent of the UI theme).
|
||||
6. **Persona** — give him the real Cradle-Dross voice (dry, sardonic, impatient, brilliant, secretly loyal) via an **editable system prompt in Settings** (tunable).
|
||||
7. **Voice** — record a clip → transcribe with **local faster-whisper on the Ollama box (CT 102, GPU, CPU-fallback)** → transcript lands in the input for **review-and-send first (mode 1)**. A *voice-mode* setting allows graduating to **hands-free auto-send (mode 2)**, then **interpret-into-confirmable-action (mode 3)** later.
|
||||
|
||||
## Non-goals (this iteration)
|
||||
|
||||
- Voice modes 2 and 3 are designed-for but not built now (mode setting ships; only mode 1 wired).
|
||||
- Multi-conversation history browser, per-Space companions in the bubble, and wake-word/always-listening are out of scope.
|
||||
|
||||
---
|
||||
|
||||
## Architecture
|
||||
|
||||
### Components
|
||||
|
||||
| Unit | Responsibility |
|
||||
|---|---|
|
||||
| `public/components/dross_bubble.js` (new) | The floating orb + panel: render, drag (orb & panel header), anchored open, collapse/close, avatar switch, voice record UI. Drives chat via `wireAgentChat`. Replaces the `renderRightrail` mount in `app.js`. |
|
||||
| `public/components/dross_avatar.js` (new) | Pure render of the chosen avatar (soft-eye / wisp / motes) at a given size — reused by orb + panel header + settings preview. |
|
||||
| `lib/api/routes/dross.js` (new) | Global (space-less) Dross: `GET /api/dross` (history + conversation id) and `POST /api/dross/turn` (SSE). Mirrors `companion.js` but resolves a **global** conversation for the `companion` agent and injects the persona + view context. |
|
||||
| `lib/api/routes/voice.js` (new) | `POST /api/voice/transcribe` — accepts an audio blob, proxies to the faster-whisper service, returns `{ text }`. Owner-only. |
|
||||
| `public/views/settings.js` (extend) | New **Dross** section: avatar picker, accent colour, persona textarea, voice-mode select. Persists to `app_settings` key `dross`. |
|
||||
| faster-whisper service on **CT 102** (infra) | OpenAI-compatible `/v1/audio/transcriptions` (e.g. `faster-whisper-server`/`speaches`), GPU with CPU fallback, small/base model. Shares the Ollama LXC. |
|
||||
|
||||
### Settings shape (`app_settings` key `dross`)
|
||||
|
||||
```json
|
||||
{
|
||||
"avatar": "soft-eye", // soft-eye | wisp | motes
|
||||
"accent": "#a86adf", // Dross's violet (independent of UI theme)
|
||||
"persona": "<system prompt text>",
|
||||
"voiceMode": "review" // review | handsfree | action(later)
|
||||
}
|
||||
```
|
||||
|
||||
Reuses the generic `app_settings` store (added in 2.9.0) and the `/api/theme`-style read-on-boot pattern. The bubble fetches `dross` settings on mount; the Settings panel writes them.
|
||||
|
||||
---
|
||||
|
||||
## Data flow
|
||||
|
||||
**Text turn:** input → `wireAgentChat` → `POST /api/dross/turn` (body `{ text, view }`) → SSE stream of Dross's reply (+ tool labels) into the panel log. History via `GET /api/dross`.
|
||||
|
||||
**Voice turn (mode 1):** tap mic → `MediaRecorder` captures a clip → on stop, `POST /api/voice/transcribe` (audio blob) → void-app proxies to CT 102 faster-whisper → `{ text }` → text dropped into the input for the user to review/edit → user sends as a normal turn. (Mode 2 would auto-send; mode 3 would route the transcript through an interpret step.)
|
||||
|
||||
**Persona:** the `dross.persona` setting is injected as/with the agent's system prompt in `run_turn` for the global conversation, so his voice is consistent and user-tunable.
|
||||
|
||||
**Context:** `view` (current route/entity) is passed in the turn body so Dross can answer "what am I looking at" questions.
|
||||
|
||||
---
|
||||
|
||||
## Error handling
|
||||
|
||||
- **STT unavailable / GPU absent:** transcribe endpoint returns a clear error; the bubble shows "couldn't transcribe — type instead" and never blocks text input. faster-whisper falls back to CPU on a GPU-less node (per the GPU/CPU-fallback HA rule) — slower but functional.
|
||||
- **Mic permission denied:** show a one-line hint; hide the recording UI, keep typing.
|
||||
- **Turn/stream failure:** existing `agent_chat` error path (surfaces an error bubble); retain the typed/transcribed text so it isn't lost.
|
||||
- **No token / 401:** bubble stays collapsed; opening prompts the normal owner-token flow.
|
||||
|
||||
## Testing
|
||||
|
||||
- **Headless UI:** bubble renders; orb → open (anchored) → drag → collapse (bottom bar) → close (✕); each avatar variant renders; mobile width = near-full panel.
|
||||
- **Settings:** changing avatar/accent/persona/voiceMode persists (`app_settings`) and re-applies on reload.
|
||||
- **API:** `GET /api/dross` returns a global conversation; `POST /api/dross/turn` streams; `POST /api/voice/transcribe` returns `{text}` for a sample WAV (mock the whisper service in the unit test; one live smoke test against CT 102).
|
||||
- **Persona:** a turn reflects the configured system prompt.
|
||||
|
||||
## Build phases
|
||||
|
||||
- **P1 — Floating bubble + global Dross + settings.** New `dross_bubble.js` + `dross_avatar.js`, `dross.js` route (global conversation), Settings → Dross section (avatar/accent/persona/voice-mode). Retire the right-rail companion. *No voice yet.* Ship-able on its own.
|
||||
- **P2 — Voice (review-and-send).** faster-whisper on CT 102, `voice.js` transcribe proxy, record UI + waveform, transcript → input → review → send.
|
||||
- **P3 — Later.** Voice mode 2 (hands-free auto-send), then mode 3 (interpret transcript into a confirmable action via the existing Little Blue action framework).
|
||||
|
||||
## Documentation
|
||||
|
||||
Per the standing rule, ship docs to the Void wiki + Gitea (`Hynes/Void-Homelab`) with each phase; spec + plan under `docs/superpowers/`. Mockup at `docs/mockups/dross-chat.html`.
|
||||
Reference in New Issue
Block a user