Commit Graph

133 Commits

Author SHA1 Message Date
root
10a8b813a5 fix: crash-proofing + small robustness fixes
- pool.js: add pool.on('error') handler — an idle-client error (DB restart /
  .215 failover) previously crashed the process (no 'error' listener → throw)
- context tool: project a SAFE_COLUMNS allow-list for resources (never the
  monitoring/metadata JSON blobs); also add 'resource' to TABLE (was unhandled)
- applyPendingChange: guard the 'upsert' arm so a non-upsertable entity_type
  fails with a clear ValidationError instead of a bare TypeError

Tests: pool_error, context (resource case), pending_extended_actions (guard).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-02 00:17:15 +10:00
root
afbf075d26 docs: security sweep, code review, Yerin design, Plan 6 brainstorm brief
- security-sweep-2026-06-01.md: fresh sweep of alpha.6 (1 fixed, findings + carry-overs)
- code-review-2026-06-01.md: optimisation/cleanliness notes (pool error handler,
  O(n) bcrypt token scan, FTS index alignment, dup auth parsing)
- yerin-security-agent.md: security-agent design + tool roadmap + Orthos role proposal
- plan-6-brainstorm-brief.md: Sacred Valley widget inventory + open design questions
- security-followups.md: marked the pending_changes CHECK finding RESOLVED

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-01 23:26:46 +10:00
root
6c393d8069 feat(security): Yerin security-agent toolset (read-only)
New securityRegistry (separate from companionRegistry) with two read-only,
secret-free tools for the Yerin security agent:
- audit_log: query the redacted audit trail by actor_kind/actor_id
- agent_inventory: list agents + capabilities/scopes (explicit projection,
  never SELECT *, no token material)

Follows the existing createRegistry() pattern. Design + wiring roadmap in
docs/yerin-security-agent.md. Not yet seeded/exposed over MCP (left for review).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-01 23:26:46 +10:00
root
459a7749c9 fix(auth): constant-time owner-token comparison
Owner bearer token was compared with === / !==, which short-circuits on the
first differing byte and leaks token length+prefix via response timing
(security-sweep-2026-06-01.md). New timingSafeStrEqual (crypto.timingSafeEqual
with a length pre-check so it never throws on length mismatch); wired into both
owner.js and agent_auth.js.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-01 23:26:46 +10:00
root
c591b2aed1 fix(pending): allow suggest-tier 'upsert' drafts; make dependency wiring owner-only
The pending_changes.action CHECK only permitted create/update/delete, so a
suggest-tier agent hitting POST /api/refs/upsert (or the resource dependency
routes) 500'd on the INSERT (docs/security-followups.md HIGH finding).

- migration 009: widen CHECK to include 'upsert'
- applyPendingChange: dispatch 'upsert' -> refsRepo.upsertByExternal on approve
- resources.js: add_dependency/remove_dependency are now owner-only (requireOwner),
  infra wiring is never diverted to pending_changes
- tests/api/pending_extended_actions.test.js: regression coverage

Full suite green (278 pass / 1 skip).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-01 23:19:44 +10:00
root
8ce97bbacc feat(companion): Dross persona (Cradle) + migration 008 rename; remove dead API-key path
- system prompt = Dross (Ozriel's construct fragment, per Void 1.0), with tool guidance
- migration 008 renames the seeded agent 'companion' → display name 'Dross'
- removed lib/ai/anthropic.js + lib/ai/agent/runtime.js + tests + @anthropic-ai/sdk dep (companion now runs via the claude CLI; kept lib/ai/secret.js for the Vaultwarden roadmap)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-01 22:58:43 +10:00
root
d35e43a8ad polish(ui): friendly tool-chip labels (strip mcp__void__ prefix)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-01 22:50:27 +10:00
root
f8beff8702 fix(companion): accept view:null in turn schema (rail sends null when no active entity)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-01 22:49:08 +10:00
root
02efff83ce test: isolate tests on void_test DB (stop resetDb wiping prod void)
resetDb() DROPs schema; dev DATABASE_URL pointed at the shared prod void DB on
.215. setup.js now forces a dedicated void_test DB (TEST_DATABASE_URL or derived)
and throws if it would target prod. Created void_test + pg_hba rule on CT 310.
Verified: full suite green, prod void space count unchanged (2→2).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-01 22:45:17 +10:00
root
3988425e67 chore: gitignore python build artifacts (untrack pycache/egg-info)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-01 22:23:06 +10:00
root
16497bd9db chore: version 2.0.0-alpha.6 — companion on Claude CLI subprocess (Max subscription)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-01 22:22:53 +10:00
root
1b8dc91800 fix(companion): emit draft from user-turn tool_result + stamp space_id on created entities
- driver: tool_results arrive as type:'user' content blocks (not bare); parse them
- route: tool_result content is a JSON string; parse it for pending_change_id → draft event
- propose_change: inject ctx.space_id into create payloads (model can't know the uuid; tables require it)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-01 22:21:15 +10:00
root
1e8bbca2a5 fix(claude-cli): --resume for continuing turns (reusing --session-id errored on turn 2)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-01 22:18:13 +10:00
root
c73be6681f fix(claude-cli): feed prompt via stdin (variadic --tools/--allowedTools ate the positional)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-01 22:15:14 +10:00
root
23616d24d6 fix(companion): absolute node path for MCP spawn + restrict to mcp tools
claude resolves the MCP server command against the child env (no PATH), so a
bare 'node' failed to spawn (status:failed). Use process.execPath. Also pass
--tools to drop claude's built-ins (Bash/Read/Write/…) — companion gets only
the four mcp__void__* tools.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-01 22:11:09 +10:00
root
c4b014c15e fix(companion): pass DATABASE_URL/OLLAMA_URL to MCP stdio child explicitly
Don't rely on env inheritance/cwd-dotenv for the claude-spawned MCP server.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-01 22:01:26 +10:00
root
51bc5912ff feat(api): companion route drives claude CLI + MCP tools (subscription auth)
Replaces the runTurn/callModel/Anthropic-API-key path in POST /turn with
runClaudeTurn (claude CLI) backed by a per-turn MCP config that spawns
companion-stdio.js. Extracts pending_change_id from tool_result events
defensively (structuredContent → text-JSON fallback). Rewrites companion
test to inject fake-claude-draft.js via app.locals.claudeExe.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-01 21:57:05 +10:00
root
bc1b820cc8 feat(ai): claude CLI subprocess driver (subscription auth, stream-json)
Implements runClaudeTurn() — spawns the claude CLI for a single companion
turn using subscription/OAuth auth (strips ANTHROPIC_API_KEY +
ANTHROPIC_AUTH_TOKEN from child env), streaming normalised events (delta,
tool, tool_result, result, error) via onEvent callback.

Includes hermetic test + fake-claude.js fixture that mimics real 2.1.159
stream-json output; zero network/CLI calls in the test suite.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-01 21:51:55 +10:00
root
1c03d6c277 feat(mcp): stdio MCP server exposing the four companion tools
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-01 21:45:42 +10:00
root
c7a94f26d1 chore: version 2.0.0-alpha.5 + plan-5 completion doc
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-01 19:41:46 +10:00
root
15d45a8fd6 fix(ui): companion rail loads current space on initial page load
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-01 19:39:18 +10:00
root
f49282b00c feat(ui): right-rail companion chat — streaming, tool chips, inline drafts
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-01 19:34:27 +10:00
root
19a20ba083 feat(ui): authenticated POST->SSE reader
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-01 19:28:25 +10:00
root
df03286415 feat(api): companion SSE turn endpoint + per-Space history
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-01 19:14:44 +10:00
root
d593234904 feat(ai): agent runtime tool-use loop with event streaming
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-01 18:30:46 +10:00
root
09a27e8495 feat(ai): Anthropic client + streaming callModel adapter
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-01 18:24:59 +10:00
root
02e2a633f0 feat(ai): wire the four v1 companion tools into a shared registry
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-01 18:18:44 +10:00
root
7282729654 feat(ai): propose_change tool — drafts to pending_changes, never applies
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-01 18:16:53 +10:00
root
2e121ce6d4 feat(ai): context tool — resolve the active view entity
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-01 18:14:42 +10:00
root
d80c550d2e feat(ai): search + read grounding tools
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-01 18:12:03 +10:00
root
de4b6a8403 feat(ai): extensible agent tool registry
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-01 18:10:11 +10:00
root
f80fd278a5 feat(db): conversations.findOrCreateForSpace for the ambient companion
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-01 18:08:34 +10:00
root
cf0510872e feat(db): migration 007 — conversations.space_id + seed companion agent
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-01 18:06:34 +10:00
root
d0d61575e3 feat(ai): vault_path secret resolver (env:/file:)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-01 18:04:21 +10:00
root
5f601c1a3c chore(deps): add @anthropic-ai/sdk for companion runtime
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-01 18:03:12 +10:00
root
31fb859fa4 docs(plan5): companion chat implementation plan (16 TDD tasks)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-01 18:01:01 +10:00
root
1cc2abf95c docs(plan5): companion chat design spec
Scope B (knowledge assistant + drafting via pending_changes approval chain),
lean Anthropic-SDK runtime (supersedes the top-level spec's Mastra wording),
extensible shared tool registry (search/read/propose_change/context), per-Space
ambient companion, SSE turn lifecycle, inline draft card synced with the Inbox,
structural prompt-injection containment. Ignore .superpowers/ brainstorm dir.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-01 17:49:08 +10:00
root
941df0d0d2 fix(deploy): point deploy targets at CT 311 new IP .216
Post-outage recovery: a rogue OpenWrt device seized 192.168.1.13 after the
power-cut reboot, ARP-poisoning the LAN so CT 311 was unreachable despite being
healthy. Renumbered CT 311 .13 -> .216 (out of the conflict-prone low range,
next to the DB at .215). Update push.sh + push-workers.sh defaults to
root@192.168.1.216; push.sh no longer defaults to the void2-app hostname (that
resolves to the Cloudflare tunnel and can't carry SSH).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-01 17:49:08 +10:00
root
6cba2e82da fix(deploy): exclude venv/ from push-workers rsync
The prod venv at /opt/void-workers/venv was being deleted on every
push because rsync --delete saw no matching dir in the source (which
has .venv/, not venv/). Now both names are excluded.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-01 11:04:21 +10:00
root
a8b2cddcf5 fix(workers): safe_fetch pins IP + manual redirect re-validation
Two real findings from the security reviewer:

1. urllib auto-follows 3xx redirects via the default HTTPRedirectHandler.
   The previous code's hop loop never ran — urllib silently followed.
   Replaced with http.client + a manual hop loop. Every hop re-runs
   _validate_url, so an open-redirect to 127.0.0.1 / RFC1918 / metadata
   gets caught on the second hop.

2. DNS TOCTOU — _resolve() validated but urllib.request re-resolved on
   connect. Now the connection is pinned to the validated IP via a
   PinnedHTTPConn / PinnedHTTPSConn subclass that overrides connect() to
   bind socket.create_connection to (addr, port). For HTTPS, TLS
   server_hostname is set to the original host so SNI + cert
   verification still work against the named host while the TCP
   destination is the pinned IP.

Tests added: redirect-to-loopback short-circuits at validation;
too-many-redirects exhausts max_hops; 2xx returns body; non-2xx raises.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-01 10:28:55 +10:00
root
7707b7eb00 chore: version 2.0.0-alpha.4 + changelog + plan-4 completion doc
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-01 10:25:31 +10:00
root
13fac102dd feat(cron): daily sync.source_doc enqueue
node-cron schedules runSync at 03:00 local time; runSync enqueues
sync.source_doc for every source_docs row with sync_source='url'.
Started from server.js's CLI gate alongside the job queue.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-01 10:14:07 +10:00
root
8fa7f71694 feat(workers): sync.source_doc with sha256 diff
Fetches upstream URL via safe_fetch, sha256-diffs against the prior
body_sha stored in metadata, updates body_text + last_synced only when
content changed. Unchanged syncs just touch last_synced.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-01 10:13:27 +10:00
root
cd1d69c689 feat(workers): safe_fetch Python port
Mirrors lib/ingest/safe_fetch.js. Same scheme + IP-range checks and
VOID_INGEST_ALLOW_PRIVATE env gate. Used by sync.source_doc and any
future Python workers that fetch user-controlled URLs.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-01 10:12:47 +10:00
root
65fd71dc0d fix(workers): yt-dlp argv injection — scheme check + -- separator
The url passed to yt-dlp is user-controllable (via /api/capture). Any
string starting with '-' would be parsed as a flag (e.g.
--config-location=/etc/passwd). Mitigations:
1. Validate scheme is http(s) and hostname is present before subprocess.
2. Pass `--` to yt-dlp so it stops flag parsing before the positional
   URL.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-01 10:11:57 +10:00
root
b10b68582d feat(api): capture routes YouTube/Vimeo URLs to ingest.video
POST /api/capture with a youtube.com / youtu.be / vimeo.com URL
enqueues ingest.video (Python worker) instead of ingest.url
(Node worker). Detection by URL hostname; idempotency_key + response
shape unchanged.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-01 10:08:16 +10:00
root
1ba7aae439 feat(workers): ingest.video via yt-dlp + Whisper
yt-dlp pulls metadata (title, description, uploader, thumbnail) and
bestaudio (opus). faster-whisper transcribes; audio file removed after.
Creates a refs row with kind='video' and source_kind='youtube' for
YouTube URLs, generic 'video' otherwise. Idempotent on
sha256(space_id + url) via refs.external_id.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-01 10:07:33 +10:00
root
e64f1345f6 feat(workers): whisper loader with CUDA detect + CPU fallback
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-01 10:06:50 +10:00
root
2adeae555d fix(deploy): push-workers.sh chowns + preserves .env
Rsync ran as root over SSH so files landed root-owned, but workers run
as voidworkers — the service couldn't even reach the venv binary.
Now: chown -R voidworkers after rsync, run venv create + pip install
under `su voidworkers -c`. Also excludes .env, .gitignore, .pytest_cache
so they survive across deploys.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-01 10:06:29 +10:00
root
3d82f0e5d5 feat(jobs): blob worker fans out to extract.pdf / extract.image
After creating a ref, the Node-side ingest.blob worker enqueues a
follow-up job for the Python void-workers (Plan 4) to OCR / extract
text. Other kinds (file) get no follow-up.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-06-01 09:34:06 +10:00