From 54ba68a11cad0fc966fdb87055ddab1fcf2966b6 Mon Sep 17 00:00:00 2001 From: root Date: Mon, 1 Jun 2026 04:11:32 +1000 Subject: [PATCH] docs: move void-v2 specs + plans into the repo All Void 2.0 superpowers specs and implementation plans now live at docs/superpowers/{specs,plans}/ inside the repo. Previously they were at /project/docs/superpowers/ which was not under git. Co-Authored-By: Claude Opus 4.7 --- .../2026-05-31-void-v2-plan1-foundation.md | 3267 +++++++++++++++++ .../2026-05-31-void-v2-plan1-progress.md | 150 + .../2026-05-31-void-v2-plan2-api-and-shell.md | 536 +++ .../plans/2026-06-01-void-v2-plan3-capture.md | 2430 ++++++++++++ .../specs/2026-05-31-void-v2-design.md | 656 ++++ .../specs/2026-06-01-void-v2-plan3-capture.md | 295 ++ 6 files changed, 7334 insertions(+) create mode 100644 docs/superpowers/plans/2026-05-31-void-v2-plan1-foundation.md create mode 100644 docs/superpowers/plans/2026-05-31-void-v2-plan1-progress.md create mode 100644 docs/superpowers/plans/2026-05-31-void-v2-plan2-api-and-shell.md create mode 100644 docs/superpowers/plans/2026-06-01-void-v2-plan3-capture.md create mode 100644 docs/superpowers/specs/2026-05-31-void-v2-design.md create mode 100644 docs/superpowers/specs/2026-06-01-void-v2-plan3-capture.md diff --git a/docs/superpowers/plans/2026-05-31-void-v2-plan1-foundation.md b/docs/superpowers/plans/2026-05-31-void-v2-plan1-foundation.md new file mode 100644 index 0000000..758933e --- /dev/null +++ b/docs/superpowers/plans/2026-05-31-void-v2-plan1-foundation.md @@ -0,0 +1,3267 @@ +# Void 2.0 — Plan 1: Foundation Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Provision the two Void 2.0 LXCs, stand up Postgres + pgvector, scaffold the `void-server` Node project, build the full schema via migrations, implement all entity repositories with capability checks, ship owner-token auth, and expose a healthcheck. Produces a running-but-empty backend you can hit with `curl`. + +**Architecture:** Two LXCs — `void2-db` (Postgres 16 + pgvector) and `void2-app` (Node 22 + Express). One repo at `/project/src/void-v2`. Strict layer boundaries: HTTP routes → `lib/api/` → `lib/db/repos/` → SQL. Capability checks centralised in `lib/auth/capability.js`; audit log written by every mutating repo call. + +**Tech Stack:** Node 22 LTS, Express 4, `pg` (node-postgres), Zod (validation), Vitest (tests), `bcrypt` (token hashing), `pino` (logs), pgvector (vector type), pg-boss (later plans). PVE 8.x for LXCs. + +**Reference spec:** `/project/docs/superpowers/specs/2026-05-31-void-v2-design.md` + +--- + +## File Structure + +``` +/project/src/void-v2/ +├── README.md +├── CHANGELOG.md +├── docs/ +│ └── VERSION_HISTORY.md +├── .env.example +├── .gitignore +├── package.json +├── vitest.config.js +├── server.js Express bootstrap (entry) +├── lib/ +│ ├── db/ +│ │ ├── pool.js Postgres connection pool +│ │ ├── migrate.js Migration runner (up only for v1) +│ │ ├── migrations/ +│ │ │ ├── 001_core.sql spaces, projects, tasks +│ │ │ ├── 002_knowledge.sql pages, page_revisions, refs +│ │ │ ├── 003_resources.sql resources, dependencies, credentials, source_docs +│ │ │ ├── 004_agents.sql agents, conversations, messages +│ │ │ ├── 005_cross.sql tags, entity_tags, entity_links, attachments +│ │ │ └── 006_audit.sql audit_log, pending_changes +│ │ └── repos/ +│ │ ├── spaces.js +│ │ ├── projects.js +│ │ ├── tasks.js +│ │ ├── pages.js +│ │ ├── refs.js +│ │ ├── resources.js +│ │ ├── source_docs.js +│ │ ├── agents.js +│ │ ├── conversations.js +│ │ ├── messages.js +│ │ ├── tags.js +│ │ ├── links.js +│ │ ├── attachments.js +│ │ ├── audit.js +│ │ └── pending_changes.js +│ ├── auth/ +│ │ ├── owner.js Owner bearer-token middleware +│ │ └── capability.js Per-actor capability check +│ └── log.js pino logger +├── tests/ +│ ├── helpers/ +│ │ └── db.js Test DB bootstrap (drop + migrate + transaction-per-test) +│ └── repos/ One test file per repo +└── deploy/ + ├── void2-app.service systemd unit + └── README.md Deploy steps +``` + +--- + +## Task 1: Repo Scaffolding + Git Init + +**Files:** +- Create: `/project/src/void-v2/.gitignore` +- Create: `/project/src/void-v2/README.md` +- Create: `/project/src/void-v2/CHANGELOG.md` +- Create: `/project/src/void-v2/docs/VERSION_HISTORY.md` +- Create: `/project/src/void-v2/.env.example` + +- [ ] **Step 1: Create repo directory and init git** + +```bash +mkdir -p /project/src/void-v2/docs +cd /project/src/void-v2 +git init -b main +``` + +- [ ] **Step 2: Write `.gitignore`** + +``` +node_modules/ +.env +*.log +coverage/ +.DS_Store +``` + +- [ ] **Step 3: Write `README.md`** + +```markdown +# Void 2.0 + +Homelab orchestrator + canonical knowledge store. Cradle-themed. +Successor to Void 1.x (CT 301). Spec at +`/project/docs/superpowers/specs/2026-05-31-void-v2-design.md`. + +## Layout +- `void-server` (this repo) — Node API, MCP, UI, cron, agent runtime +- `void-workers` — Python ingest workers (separate repo, later plan) + +## Quick start (dev) +1. Provision `void2-db` LXC (see `deploy/README.md`) +2. Install Postgres + pgvector on `void2-db` +3. `npm install` +4. `cp .env.example .env` and edit +5. `npm run migrate` +6. `npm start` +7. `curl -H "Authorization: Bearer $OWNER_TOKEN" http://localhost:3000/health` +``` + +- [ ] **Step 4: Write `CHANGELOG.md`** + +```markdown +# Changelog + +All notable changes to Void 2.0 are documented here. +Format: [Keep a Changelog](https://keepachangelog.com). + +## [Unreleased] + +### Added +- Initial repo scaffolding +``` + +- [ ] **Step 5: Write `docs/VERSION_HISTORY.md`** + +```markdown +# Version History + +Narrative history of major Void releases. + +## Void 2.0 (in development, started 2026-05-31) + +Complete remaster of Void 1.x. Foundation-first rebuild on Postgres + pgvector, +two-LXC HA shape, expanded entity model (Spaces, Projects, Tasks, Pages, +References, Source Docs, Resources, Conversations, Agents), unified MCP +surface for AI agents with per-agent capability tiers. + +See spec: `/project/docs/superpowers/specs/2026-05-31-void-v2-design.md`. + +## Void 1.x (2026-04 — 2026-05, CT 301) + +Cradle-themed dashboard + 7 character agents (Dross, Orthos, Eithan, Mercy, +Lindon, Yerin, Little Blue). Gridstack Sacred Valley layout. node:sqlite +storage. Mastra+Ollama for Orthos; Claude binary subprocess for others. +Path B rebuild (2026-05-13) added FTS5 search, NLP summaries, memory-update +markers, BookStack page import. +``` + +- [ ] **Step 6: Write `.env.example`** + +``` +DATABASE_URL=postgres://void:CHANGE_ME@192.168.X.X:5432/void +OWNER_TOKEN=CHANGE_ME_TO_LONG_RANDOM +PORT=3000 +LOG_LEVEL=info +NODE_ENV=development +``` + +- [ ] **Step 7: Commit** + +```bash +git add -A +git commit -m "chore: initial repo scaffolding" +``` + +--- + +## Task 2: PVE — Provision void2-db and void2-app LXCs + +**Files:** +- Create: `/project/src/void-v2/deploy/README.md` + +**Note:** These commands run on the PVE host (`z`), not inside `/project`. Hand to user to execute, or SSH from agent if PVE shell is reachable. Use unprivileged LXCs with the standard apparmor=unconfined override for this homelab. + +- [ ] **Step 1: Choose CT IDs and IPs** + +Pick two free CT IDs (suggestion: 310 = `void2-db`, 311 = `void2-app`). Pick two free IPs on the 192.168.1.0/24 LAN. + +- [ ] **Step 2: Create `void2-db` LXC on PVE host** + +```bash +# Run on PVE host 'z' as root +pct create 310 local:vztmpl/debian-12-standard_12.7-1_amd64.tar.zst \ + --hostname void2-db \ + --cores 2 --memory 4096 --swap 1024 \ + --net0 name=eth0,bridge=vmbr0,ip=192.168.1.X/24,gw=192.168.1.1 \ + --storage donatello-zfs --rootfs donatello-zfs:32 \ + --unprivileged 1 --features nesting=1 \ + --onboot 1 --start 0 + +# Apparmor override (this LXC env requirement per CLAUDE.md) +cat >> /etc/pve/lxc/310.conf <<'EOF' +lxc.apparmor.profile: unconfined +EOF + +pct start 310 +``` + +- [ ] **Step 3: Create `void2-app` LXC** + +```bash +pct create 311 local:vztmpl/debian-12-standard_12.7-1_amd64.tar.zst \ + --hostname void2-app \ + --cores 4 --memory 4096 --swap 1024 \ + --net0 name=eth0,bridge=vmbr0,ip=192.168.1.Y/24,gw=192.168.1.1 \ + --storage donatello-zfs --rootfs donatello-zfs:16 \ + --unprivileged 1 --features nesting=1 \ + --onboot 1 --start 0 + +cat >> /etc/pve/lxc/311.conf <<'EOF' +lxc.apparmor.profile: unconfined +EOF + +pct start 311 +``` + +- [ ] **Step 4: Verify both LXCs are up** + +```bash +pct list | grep void2 +# Expect: 310 running void2-db, 311 running void2-app +``` + +- [ ] **Step 5: Write deploy/README.md capturing the actual CT IDs + IPs chosen** + +```markdown +# Deploy + +## LXC inventory + +| Role | CT | IP | Specs | +|---|---|---|---| +| Postgres | 310 (`void2-db`) | 192.168.1.X | 2 vCPU, 4 GB RAM, 32 GB disk | +| App | 311 (`void2-app`) | 192.168.1.Y | 4 vCPU, 4 GB RAM, 16 GB disk | + +## Maintenance flow (planned downtime) +1. Notify Void: `curl -X POST .../api/maintenance/start` (later plan) +2. `pct migrate 310 ` +3. `pct migrate 311 ` +4. Service maintenance on source host +5. Migrate back when done +``` + +- [ ] **Step 6: Commit** + +```bash +cd /project/src/void-v2 +git add deploy/README.md +git commit -m "docs: capture LXC provisioning steps + inventory" +``` + +--- + +## Task 3: Postgres 16 + pgvector on void2-db + +**Files:** +- Modify: `/project/src/void-v2/deploy/README.md` (append Postgres setup section) + +**Note:** Runs inside CT 310 (`void2-db`). + +- [ ] **Step 1: SSH into void2-db (or `pct enter 310` on PVE host)** + +```bash +pct enter 310 +``` + +- [ ] **Step 2: Install Postgres 16 + pgvector** + +```bash +apt update +apt install -y curl ca-certificates gnupg +install -d /usr/share/postgresql-common/pgdg +curl -fsSL https://www.postgresql.org/media/keys/ACCC4CF8.asc \ + -o /usr/share/postgresql-common/pgdg/apt.postgresql.org.asc +echo "deb [signed-by=/usr/share/postgresql-common/pgdg/apt.postgresql.org.asc] \ + https://apt.postgresql.org/pub/repos/apt $(. /etc/os-release; echo $VERSION_CODENAME)-pgdg main" \ + > /etc/apt/sources.list.d/pgdg.list +apt update +apt install -y postgresql-16 postgresql-16-pgvector +``` + +- [ ] **Step 3: Configure listen address + pg_hba for LAN access** + +```bash +# Listen on LAN +sed -i "s/^#listen_addresses.*/listen_addresses = '*'/" \ + /etc/postgresql/16/main/postgresql.conf + +# Allow LAN access from void2-app +cat >> /etc/postgresql/16/main/pg_hba.conf <<'EOF' +# Void 2.0 app +host void void 192.168.1.0/24 scram-sha-256 +EOF + +systemctl restart postgresql +``` + +- [ ] **Step 4: Create database + user, enable pgvector** + +```bash +sudo -u postgres psql <<'EOF' +CREATE USER void WITH PASSWORD 'GENERATE_AND_CHANGE_ME'; +CREATE DATABASE void OWNER void; +\c void +CREATE EXTENSION IF NOT EXISTS vector; +CREATE EXTENSION IF NOT EXISTS pgcrypto; -- for gen_random_uuid() +EOF +``` + +Generate password with `openssl rand -base64 24`. Store in your password manager + paste into `.env` on `void2-app` later. + +- [ ] **Step 5: Verify connection from void2-app** + +```bash +# From void2-app +apt install -y postgresql-client +PGPASSWORD= psql -h 192.168.1.X -U void -d void -c 'SELECT version();' +# Expect: PostgreSQL 16.x +``` + +- [ ] **Step 6: Append deploy notes** + +Append to `/project/src/void-v2/deploy/README.md` (back on this LXC): + +```markdown +## Postgres setup (CT 310) + +- Postgres 16 + pgvector + pgcrypto +- Listen on all addresses, LAN-only via firewall +- DB `void`, user `void`, password in user's secrets store +- Test from app: `psql -h -U void -d void -c 'SELECT 1;'` +``` + +- [ ] **Step 7: Commit** + +```bash +git add deploy/README.md +git commit -m "docs: capture Postgres + pgvector setup" +``` + +--- + +## Task 4: Node Project Initialization on void2-app + +**Files:** +- Create: `/project/src/void-v2/package.json` +- Create: `/project/src/void-v2/vitest.config.js` +- Create: `/project/src/void-v2/lib/log.js` + +**Note:** Development happens here on `/project/src/void-v2` (CT 300). Deploy step (rsync to `void2-app`) comes in Task 21. + +- [ ] **Step 1: Install Node 22 if not already present** + +```bash +node --version +# Expect: v22.x.x — if not, install via nvm or nodesource +``` + +- [ ] **Step 2: Init package.json** + +```bash +cd /project/src/void-v2 +npm init -y +``` + +- [ ] **Step 3: Edit package.json — scripts + type=module** + +```json +{ + "name": "void-server", + "version": "2.0.0-alpha.1", + "type": "module", + "private": true, + "scripts": { + "start": "node server.js", + "migrate": "node lib/db/migrate.js up", + "test": "vitest run", + "test:watch": "vitest" + } +} +``` + +- [ ] **Step 4: Install runtime deps** + +```bash +npm install express pg zod dotenv bcrypt pino pino-pretty +``` + +- [ ] **Step 5: Install dev deps** + +```bash +npm install --save-dev vitest @vitest/coverage-v8 +``` + +- [ ] **Step 6: Write `vitest.config.js`** + +```javascript +import { defineConfig } from 'vitest/config'; + +export default defineConfig({ + test: { + globals: true, + testTimeout: 10_000, + coverage: { provider: 'v8', reporter: ['text', 'html'] }, + setupFiles: ['./tests/helpers/setup.js'] + } +}); +``` + +- [ ] **Step 7: Write `lib/log.js`** + +```javascript +import pino from 'pino'; + +const transport = process.env.NODE_ENV === 'production' + ? undefined + : { target: 'pino-pretty' }; + +export const log = pino({ + level: process.env.LOG_LEVEL || 'info', + transport +}); +``` + +- [ ] **Step 8: Commit** + +```bash +git add package.json package-lock.json vitest.config.js lib/log.js +git commit -m "chore: node project init + deps + logger" +``` + +--- + +## Task 5: Postgres Pool + Migration Runner + +**Files:** +- Create: `/project/src/void-v2/lib/db/pool.js` +- Create: `/project/src/void-v2/lib/db/migrate.js` +- Create: `/project/src/void-v2/tests/helpers/setup.js` +- Create: `/project/src/void-v2/tests/helpers/db.js` +- Create: `/project/src/void-v2/tests/db/migrate.test.js` + +- [ ] **Step 1: Write the failing test** + +`tests/db/migrate.test.js`: + +```javascript +import { describe, it, expect, beforeAll } from 'vitest'; +import { resetDb, withClient } from '../helpers/db.js'; +import { migrateUp } from '../../lib/db/migrate.js'; + +describe('migrate', () => { + beforeAll(async () => { await resetDb(); }); + + it('creates schema_migrations table on first run', async () => { + await migrateUp(); + await withClient(async (c) => { + const { rows } = await c.query( + `SELECT to_regclass('public.schema_migrations') AS t;` + ); + expect(rows[0].t).toBe('schema_migrations'); + }); + }); + + it('is idempotent — second run is a no-op', async () => { + await migrateUp(); + await migrateUp(); + await withClient(async (c) => { + const { rows } = await c.query( + `SELECT count(*)::int AS n FROM schema_migrations;` + ); + expect(rows[0].n).toBeGreaterThanOrEqual(0); + }); + }); +}); +``` + +- [ ] **Step 2: Write test helpers** + +`tests/helpers/setup.js`: + +```javascript +import 'dotenv/config'; +``` + +`tests/helpers/db.js`: + +```javascript +import { pool } from '../../lib/db/pool.js'; + +export async function resetDb() { + await pool.query(` + DROP SCHEMA IF EXISTS public CASCADE; + CREATE SCHEMA public; + CREATE EXTENSION IF NOT EXISTS pgcrypto; + CREATE EXTENSION IF NOT EXISTS vector; + `); +} + +export async function withClient(fn) { + const client = await pool.connect(); + try { await fn(client); } + finally { client.release(); } +} +``` + +- [ ] **Step 3: Write `lib/db/pool.js`** + +```javascript +import pg from 'pg'; +import 'dotenv/config'; + +export const pool = new pg.Pool({ + connectionString: process.env.DATABASE_URL, + max: 10, + idleTimeoutMillis: 30_000 +}); +``` + +- [ ] **Step 4: Write `lib/db/migrate.js`** + +```javascript +import fs from 'node:fs/promises'; +import path from 'node:path'; +import { fileURLToPath } from 'node:url'; +import { pool } from './pool.js'; +import { log } from '../log.js'; + +const __dirname = path.dirname(fileURLToPath(import.meta.url)); +const MIG_DIR = path.join(__dirname, 'migrations'); + +export async function migrateUp() { + const client = await pool.connect(); + try { + await client.query(` + CREATE TABLE IF NOT EXISTS schema_migrations ( + name text PRIMARY KEY, + applied_at timestamptz NOT NULL DEFAULT now() + ); + `); + + const files = (await fs.readdir(MIG_DIR)).filter(f => f.endsWith('.sql')).sort(); + const { rows } = await client.query('SELECT name FROM schema_migrations'); + const applied = new Set(rows.map(r => r.name)); + + for (const file of files) { + if (applied.has(file)) continue; + const sql = await fs.readFile(path.join(MIG_DIR, file), 'utf8'); + log.info({ migration: file }, 'applying migration'); + await client.query('BEGIN'); + try { + await client.query(sql); + await client.query( + 'INSERT INTO schema_migrations(name) VALUES ($1)', [file] + ); + await client.query('COMMIT'); + } catch (err) { + await client.query('ROLLBACK'); + throw err; + } + } + } finally { + client.release(); + } +} + +if (process.argv[2] === 'up') { + migrateUp().then(() => process.exit(0)).catch((e) => { + log.error(e); process.exit(1); + }); +} +``` + +- [ ] **Step 5: Create empty migrations directory** + +```bash +mkdir -p lib/db/migrations +``` + +- [ ] **Step 6: Run tests — expect PASS** + +```bash +DATABASE_URL=postgres://void:PWD@192.168.1.X:5432/void npm test -- tests/db/migrate.test.js +``` + +- [ ] **Step 7: Commit** + +```bash +git add lib/db/ tests/ +git commit -m "feat: db pool + migration runner with idempotency" +``` + +--- + +## Task 6: Migration 001 — Core (spaces, projects, tasks) + +**Files:** +- Create: `/project/src/void-v2/lib/db/migrations/001_core.sql` +- Create: `/project/src/void-v2/tests/db/migration_001.test.js` + +- [ ] **Step 1: Write the failing test** + +`tests/db/migration_001.test.js`: + +```javascript +import { describe, it, expect, beforeAll } from 'vitest'; +import { resetDb, withClient } from '../helpers/db.js'; +import { migrateUp } from '../../lib/db/migrate.js'; + +describe('migration 001 — core', () => { + beforeAll(async () => { await resetDb(); await migrateUp(); }); + + it('creates spaces, projects, tasks tables', async () => { + await withClient(async (c) => { + for (const t of ['spaces', 'projects', 'tasks']) { + const { rows } = await c.query( + `SELECT to_regclass('public.' || $1) AS t;`, [t] + ); + expect(rows[0].t).toBe(t); + } + }); + }); + + it('enforces UNIQUE(space_id, slug) on projects', async () => { + await withClient(async (c) => { + const { rows: [s] } = await c.query( + `INSERT INTO spaces(slug, name) VALUES('test', 'Test') RETURNING id;` + ); + await c.query( + `INSERT INTO projects(space_id, slug, name) VALUES($1, 'a', 'A');`, + [s.id] + ); + await expect( + c.query( + `INSERT INTO projects(space_id, slug, name) VALUES($1, 'a', 'B');`, + [s.id] + ) + ).rejects.toThrow(/duplicate key/i); + }); + }); +}); +``` + +- [ ] **Step 2: Write `lib/db/migrations/001_core.sql`** + +```sql +CREATE TABLE spaces ( + id uuid PRIMARY KEY DEFAULT gen_random_uuid(), + slug text NOT NULL UNIQUE, + name text NOT NULL, + description text, + theme text, + created_at timestamptz NOT NULL DEFAULT now(), + updated_at timestamptz NOT NULL DEFAULT now() +); + +CREATE TABLE projects ( + id uuid PRIMARY KEY DEFAULT gen_random_uuid(), + space_id uuid NOT NULL REFERENCES spaces(id) ON DELETE CASCADE, + slug text NOT NULL, + name text NOT NULL, + description text, + status text NOT NULL DEFAULT 'active' + CHECK (status IN ('idea','active','paused','done','abandoned')), + started_at timestamptz, + completed_at timestamptz, + created_at timestamptz NOT NULL DEFAULT now(), + updated_at timestamptz NOT NULL DEFAULT now(), + UNIQUE (space_id, slug) +); + +CREATE TABLE tasks ( + id uuid PRIMARY KEY DEFAULT gen_random_uuid(), + space_id uuid NOT NULL REFERENCES spaces(id) ON DELETE CASCADE, + project_id uuid REFERENCES projects(id) ON DELETE SET NULL, + title text NOT NULL, + body text, + status text NOT NULL DEFAULT 'todo' + CHECK (status IN ('todo','doing','blocked','done')), + priority int, + due_at timestamptz, + position int, + completed_at timestamptz, + created_at timestamptz NOT NULL DEFAULT now(), + updated_at timestamptz NOT NULL DEFAULT now() +); + +CREATE INDEX idx_projects_space ON projects(space_id); +CREATE INDEX idx_tasks_space ON tasks(space_id); +CREATE INDEX idx_tasks_project ON tasks(project_id); +CREATE INDEX idx_tasks_status ON tasks(status) WHERE status <> 'done'; +``` + +- [ ] **Step 3: Run tests — expect PASS** + +```bash +npm test -- tests/db/migration_001.test.js +``` + +- [ ] **Step 4: Commit** + +```bash +git add lib/db/migrations/001_core.sql tests/db/migration_001.test.js +git commit -m "feat(schema): 001 — spaces, projects, tasks with check constraints" +``` + +--- + +## Task 7: Repos — spaces, projects, tasks (TDD) + +**Files:** +- Create: `/project/src/void-v2/lib/db/repos/spaces.js` +- Create: `/project/src/void-v2/lib/db/repos/projects.js` +- Create: `/project/src/void-v2/lib/db/repos/tasks.js` +- Create: `/project/src/void-v2/tests/repos/spaces.test.js` +- Create: `/project/src/void-v2/tests/repos/projects.test.js` +- Create: `/project/src/void-v2/tests/repos/tasks.test.js` + +Each repo exports `create`, `getById`, `getBySlug` (where applicable), `list`, `update`, `del`. Each mutating call returns the row and is responsible for emitting an audit-log entry — but audit_log doesn't exist yet (Task 16). We add a `recordAudit(actor, action, before, after)` placeholder that's a no-op until then. Repos accept an `actor` parameter (object: `{kind, id}`) on mutating ops. + +- [ ] **Step 1: Write spaces repo tests** + +`tests/repos/spaces.test.js`: + +```javascript +import { describe, it, expect, beforeEach } from 'vitest'; +import { resetDb } from '../helpers/db.js'; +import { migrateUp } from '../../lib/db/migrate.js'; +import * as spaces from '../../lib/db/repos/spaces.js'; + +const owner = { kind: 'user', id: null }; + +beforeEach(async () => { await resetDb(); await migrateUp(); }); + +describe('spaces repo', () => { + it('creates and returns a space', async () => { + const s = await spaces.create({ slug: 'homelab', name: 'Homelab' }, owner); + expect(s.id).toBeDefined(); + expect(s.slug).toBe('homelab'); + }); + + it('getBySlug returns the row', async () => { + await spaces.create({ slug: 'a', name: 'A' }, owner); + const got = await spaces.getBySlug('a'); + expect(got.name).toBe('A'); + }); + + it('list returns all', async () => { + await spaces.create({ slug: 'a', name: 'A' }, owner); + await spaces.create({ slug: 'b', name: 'B' }, owner); + const all = await spaces.list(); + expect(all).toHaveLength(2); + }); + + it('update changes fields and bumps updated_at', async () => { + const s = await spaces.create({ slug: 'a', name: 'A' }, owner); + const u = await spaces.update(s.id, { name: 'Renamed' }, owner); + expect(u.name).toBe('Renamed'); + expect(new Date(u.updated_at).getTime()) + .toBeGreaterThanOrEqual(new Date(s.updated_at).getTime()); + }); + + it('del removes the row', async () => { + const s = await spaces.create({ slug: 'a', name: 'A' }, owner); + await spaces.del(s.id, owner); + expect(await spaces.getBySlug('a')).toBeUndefined(); + }); +}); +``` + +- [ ] **Step 2: Implement `lib/db/repos/spaces.js`** + +```javascript +import { pool } from '../pool.js'; +import { recordAudit } from './audit_stub.js'; + +export async function create({ slug, name, description, theme }, actor) { + const { rows: [r] } = await pool.query( + `INSERT INTO spaces(slug, name, description, theme) + VALUES($1,$2,$3,$4) RETURNING *`, + [slug, name, description || null, theme || null] + ); + await recordAudit(actor, 'create', 'space', r.id, null, r); + return r; +} + +export async function getById(id) { + const { rows: [r] } = await pool.query( + `SELECT * FROM spaces WHERE id=$1`, [id]); + return r; +} + +export async function getBySlug(slug) { + const { rows: [r] } = await pool.query( + `SELECT * FROM spaces WHERE slug=$1`, [slug]); + return r; +} + +export async function list() { + const { rows } = await pool.query(`SELECT * FROM spaces ORDER BY name`); + return rows; +} + +export async function update(id, patch, actor) { + const before = await getById(id); + const fields = ['name','description','theme','slug']; + const sets = [], vals = []; + let i = 1; + for (const f of fields) { + if (patch[f] !== undefined) { sets.push(`${f}=$${i++}`); vals.push(patch[f]); } + } + sets.push(`updated_at=now()`); + vals.push(id); + const { rows: [r] } = await pool.query( + `UPDATE spaces SET ${sets.join(', ')} WHERE id=$${i} RETURNING *`, + vals + ); + await recordAudit(actor, 'update', 'space', id, before, r); + return r; +} + +export async function del(id, actor) { + const before = await getById(id); + await pool.query(`DELETE FROM spaces WHERE id=$1`, [id]); + await recordAudit(actor, 'delete', 'space', id, before, null); +} +``` + +- [ ] **Step 3: Create audit stub** + +`lib/db/repos/audit_stub.js`: + +```javascript +// Replaced in Task 16 with real audit_log writes. +export async function recordAudit() { /* noop until 006 migration lands */ } +``` + +- [ ] **Step 4: Run spaces tests — expect PASS** + +```bash +npm test -- tests/repos/spaces.test.js +``` + +- [ ] **Step 5: Write projects repo + tests (mirror spaces pattern)** + +`tests/repos/projects.test.js` — tests: create requires space_id, getBySpace lists by space, status enum enforced, UNIQUE(space, slug) enforced. Then implement `lib/db/repos/projects.js` mirroring `spaces.js` with the additional `space_id` foreign key and a `listBySpace(spaceId, {status})` method. + +```javascript +// tests/repos/projects.test.js +import { describe, it, expect, beforeEach } from 'vitest'; +import { resetDb } from '../helpers/db.js'; +import { migrateUp } from '../../lib/db/migrate.js'; +import * as spaces from '../../lib/db/repos/spaces.js'; +import * as projects from '../../lib/db/repos/projects.js'; + +const owner = { kind: 'user', id: null }; + +beforeEach(async () => { await resetDb(); await migrateUp(); }); + +describe('projects repo', () => { + it('creates a project under a space', async () => { + const s = await spaces.create({ slug: 'h', name: 'H' }, owner); + const p = await projects.create( + { space_id: s.id, slug: 'z2', name: 'Z2 Migration' }, owner + ); + expect(p.space_id).toBe(s.id); + expect(p.status).toBe('active'); + }); + + it('enforces unique slug per space', async () => { + const s = await spaces.create({ slug: 'h', name: 'H' }, owner); + await projects.create({ space_id: s.id, slug: 'a', name: 'A' }, owner); + await expect( + projects.create({ space_id: s.id, slug: 'a', name: 'B' }, owner) + ).rejects.toThrow(); + }); + + it('listBySpace filters by status', async () => { + const s = await spaces.create({ slug: 'h', name: 'H' }, owner); + await projects.create({ space_id: s.id, slug: 'a', name: 'A' }, owner); + await projects.create( + { space_id: s.id, slug: 'b', name: 'B', status: 'done' }, owner + ); + const active = await projects.listBySpace(s.id, { status: 'active' }); + expect(active).toHaveLength(1); + expect(active[0].slug).toBe('a'); + }); +}); +``` + +```javascript +// lib/db/repos/projects.js +import { pool } from '../pool.js'; +import { recordAudit } from './audit_stub.js'; + +export async function create({ space_id, slug, name, description, status = 'active', started_at }, actor) { + const { rows: [r] } = await pool.query( + `INSERT INTO projects(space_id, slug, name, description, status, started_at) + VALUES($1,$2,$3,$4,$5,$6) RETURNING *`, + [space_id, slug, name, description || null, status, started_at || null] + ); + await recordAudit(actor, 'create', 'project', r.id, null, r); + return r; +} + +export async function getById(id) { + const { rows: [r] } = await pool.query(`SELECT * FROM projects WHERE id=$1`, [id]); + return r; +} + +export async function listBySpace(space_id, { status } = {}) { + const sql = status + ? `SELECT * FROM projects WHERE space_id=$1 AND status=$2 ORDER BY name` + : `SELECT * FROM projects WHERE space_id=$1 ORDER BY name`; + const args = status ? [space_id, status] : [space_id]; + const { rows } = await pool.query(sql, args); + return rows; +} + +export async function update(id, patch, actor) { + const before = await getById(id); + const fields = ['slug','name','description','status','started_at','completed_at']; + const sets = [], vals = []; + let i = 1; + for (const f of fields) { + if (patch[f] !== undefined) { sets.push(`${f}=$${i++}`); vals.push(patch[f]); } + } + sets.push(`updated_at=now()`); + vals.push(id); + const { rows: [r] } = await pool.query( + `UPDATE projects SET ${sets.join(', ')} WHERE id=$${i} RETURNING *`, + vals + ); + await recordAudit(actor, 'update', 'project', id, before, r); + return r; +} + +export async function del(id, actor) { + const before = await getById(id); + await pool.query(`DELETE FROM projects WHERE id=$1`, [id]); + await recordAudit(actor, 'delete', 'project', id, before, null); +} +``` + +- [ ] **Step 6: Write tasks repo + tests** + +```javascript +// tests/repos/tasks.test.js +import { describe, it, expect, beforeEach } from 'vitest'; +import { resetDb } from '../helpers/db.js'; +import { migrateUp } from '../../lib/db/migrate.js'; +import * as spaces from '../../lib/db/repos/spaces.js'; +import * as projects from '../../lib/db/repos/projects.js'; +import * as tasks from '../../lib/db/repos/tasks.js'; + +const owner = { kind: 'user', id: null }; + +beforeEach(async () => { await resetDb(); await migrateUp(); }); + +describe('tasks repo', () => { + it('creates a task with optional project', async () => { + const s = await spaces.create({ slug: 'h', name: 'H' }, owner); + const t = await tasks.create({ space_id: s.id, title: 'do it' }, owner); + expect(t.status).toBe('todo'); + expect(t.project_id).toBeNull(); + }); + + it('marking done sets completed_at', async () => { + const s = await spaces.create({ slug: 'h', name: 'H' }, owner); + const t = await tasks.create({ space_id: s.id, title: 'x' }, owner); + const u = await tasks.update(t.id, { status: 'done' }, owner); + expect(u.status).toBe('done'); + expect(u.completed_at).not.toBeNull(); + }); + + it('listByProject returns project tasks only', async () => { + const s = await spaces.create({ slug: 'h', name: 'H' }, owner); + const p = await projects.create({ space_id: s.id, slug: 'p', name: 'P' }, owner); + await tasks.create({ space_id: s.id, project_id: p.id, title: 'a' }, owner); + await tasks.create({ space_id: s.id, title: 'orphan' }, owner); + const list = await tasks.listByProject(p.id); + expect(list).toHaveLength(1); + }); +}); +``` + +```javascript +// lib/db/repos/tasks.js +import { pool } from '../pool.js'; +import { recordAudit } from './audit_stub.js'; + +export async function create({ space_id, project_id = null, title, body, priority, due_at, position }, actor) { + const { rows: [r] } = await pool.query( + `INSERT INTO tasks(space_id, project_id, title, body, priority, due_at, position) + VALUES($1,$2,$3,$4,$5,$6,$7) RETURNING *`, + [space_id, project_id, title, body || null, priority || null, due_at || null, position || null] + ); + await recordAudit(actor, 'create', 'task', r.id, null, r); + return r; +} + +export async function getById(id) { + const { rows: [r] } = await pool.query(`SELECT * FROM tasks WHERE id=$1`, [id]); + return r; +} + +export async function listByProject(project_id) { + const { rows } = await pool.query( + `SELECT * FROM tasks WHERE project_id=$1 ORDER BY position NULLS LAST, created_at`, + [project_id] + ); + return rows; +} + +export async function listBySpace(space_id, { status } = {}) { + const sql = status + ? `SELECT * FROM tasks WHERE space_id=$1 AND status=$2 ORDER BY created_at` + : `SELECT * FROM tasks WHERE space_id=$1 ORDER BY created_at`; + const { rows } = await pool.query(sql, status ? [space_id, status] : [space_id]); + return rows; +} + +export async function update(id, patch, actor) { + const before = await getById(id); + const fields = ['title','body','status','priority','due_at','position','project_id']; + const sets = [], vals = []; + let i = 1; + for (const f of fields) { + if (patch[f] !== undefined) { sets.push(`${f}=$${i++}`); vals.push(patch[f]); } + } + if (patch.status === 'done') { sets.push(`completed_at=now()`); } + sets.push(`updated_at=now()`); + vals.push(id); + const { rows: [r] } = await pool.query( + `UPDATE tasks SET ${sets.join(', ')} WHERE id=$${i} RETURNING *`, + vals + ); + await recordAudit(actor, 'update', 'task', id, before, r); + return r; +} + +export async function del(id, actor) { + const before = await getById(id); + await pool.query(`DELETE FROM tasks WHERE id=$1`, [id]); + await recordAudit(actor, 'delete', 'task', id, before, null); +} +``` + +- [ ] **Step 7: Run all repo tests — expect PASS** + +```bash +npm test -- tests/repos/ +``` + +- [ ] **Step 8: Commit** + +```bash +git add lib/db/repos/ tests/repos/ +git commit -m "feat(repos): spaces, projects, tasks with audit stub" +``` + +--- + +## Task 8: Migration 002 — Knowledge (pages, page_revisions, refs) + +**Files:** +- Create: `/project/src/void-v2/lib/db/migrations/002_knowledge.sql` +- Create: `/project/src/void-v2/tests/db/migration_002.test.js` + +- [ ] **Step 1: Write test** + +```javascript +// tests/db/migration_002.test.js +import { describe, it, expect, beforeAll } from 'vitest'; +import { resetDb, withClient } from '../helpers/db.js'; +import { migrateUp } from '../../lib/db/migrate.js'; + +describe('migration 002 — knowledge', () => { + beforeAll(async () => { await resetDb(); await migrateUp(); }); + + it('creates pages, page_revisions, refs', async () => { + await withClient(async (c) => { + for (const t of ['pages','page_revisions','refs']) { + const { rows } = await c.query( + `SELECT to_regclass('public.' || $1) AS t;`, [t] + ); + expect(rows[0].t).toBe(t); + } + }); + }); + + it('refs.kind check enforces enum', async () => { + await withClient(async (c) => { + const { rows: [s] } = await c.query( + `INSERT INTO spaces(slug,name) VALUES('h','H') RETURNING id;` + ); + await expect(c.query( + `INSERT INTO refs(space_id, kind) VALUES($1, 'invalid');`, [s.id] + )).rejects.toThrow(/check/i); + }); + }); +}); +``` + +- [ ] **Step 2: Write `002_knowledge.sql`** + +```sql +CREATE TABLE pages ( + id uuid PRIMARY KEY DEFAULT gen_random_uuid(), + space_id uuid REFERENCES spaces(id) ON DELETE SET NULL, + slug text NOT NULL, + title text NOT NULL, + body_md text NOT NULL DEFAULT '', + body_html text, + parent_id uuid REFERENCES pages(id) ON DELETE SET NULL, + embedding vector(1024), + created_at timestamptz NOT NULL DEFAULT now(), + updated_at timestamptz NOT NULL DEFAULT now(), + UNIQUE (space_id, slug) +); + +CREATE TABLE page_revisions ( + id uuid PRIMARY KEY DEFAULT gen_random_uuid(), + page_id uuid NOT NULL REFERENCES pages(id) ON DELETE CASCADE, + body_md text NOT NULL, + edited_by text, + created_at timestamptz NOT NULL DEFAULT now() +); + +CREATE TABLE refs ( + id uuid PRIMARY KEY DEFAULT gen_random_uuid(), + space_id uuid REFERENCES spaces(id) ON DELETE SET NULL, + kind text NOT NULL + CHECK (kind IN ('url','video','pdf','image','file')), + source_url text, + title text, + description text, + summary text, + body_text text, + blob_path text, + thumbnail text, + metadata jsonb NOT NULL DEFAULT '{}'::jsonb, + embedding vector(1024), + status text NOT NULL DEFAULT 'ingested' + CHECK (status IN ('ingested','indexed','enriched')), + source_kind text, + external_id text, + captured_at timestamptz NOT NULL DEFAULT now(), + created_at timestamptz NOT NULL DEFAULT now(), + updated_at timestamptz NOT NULL DEFAULT now() +); + +CREATE INDEX idx_pages_space ON pages(space_id); +CREATE INDEX idx_pages_parent ON pages(parent_id); +CREATE INDEX idx_pages_fts ON pages + USING GIN (to_tsvector('english', title || ' ' || body_md)); +CREATE INDEX idx_pages_embed ON pages + USING hnsw (embedding vector_cosine_ops); + +CREATE INDEX idx_refs_space ON refs(space_id); +CREATE INDEX idx_refs_kind ON refs(kind); +CREATE INDEX idx_refs_external ON refs(source_kind, external_id) + WHERE source_kind IS NOT NULL; +CREATE INDEX idx_refs_fts ON refs USING GIN ( + to_tsvector('english', + coalesce(title,'') || ' ' || coalesce(summary,'') || ' ' || coalesce(body_text,'')) +); +CREATE INDEX idx_refs_embed ON refs USING hnsw (embedding vector_cosine_ops); +``` + +- [ ] **Step 3: Run tests** + +```bash +npm test -- tests/db/migration_002.test.js +``` + +- [ ] **Step 4: Commit** + +```bash +git add lib/db/migrations/002_knowledge.sql tests/db/migration_002.test.js +git commit -m "feat(schema): 002 — pages, page_revisions, refs with FTS + vector indexes" +``` + +--- + +## Task 9: Repos — pages, page_revisions, refs + +**Files:** +- Create: `/project/src/void-v2/lib/db/repos/pages.js` +- Create: `/project/src/void-v2/lib/db/repos/refs.js` +- Create: `/project/src/void-v2/tests/repos/pages.test.js` +- Create: `/project/src/void-v2/tests/repos/refs.test.js` + +Follow the same pattern as Task 7 (create / getById / getBySlug / list / update / del with `actor` parameter, audit stub). + +- [ ] **Step 1: Write pages tests** + +```javascript +// tests/repos/pages.test.js +import { describe, it, expect, beforeEach } from 'vitest'; +import { resetDb } from '../helpers/db.js'; +import { migrateUp } from '../../lib/db/migrate.js'; +import * as spaces from '../../lib/db/repos/spaces.js'; +import * as pages from '../../lib/db/repos/pages.js'; + +const owner = { kind: 'user', id: null }; + +beforeEach(async () => { await resetDb(); await migrateUp(); }); + +describe('pages repo', () => { + it('creates a page and auto-snapshots a revision', async () => { + const s = await spaces.create({ slug: 'h', name: 'H' }, owner); + const p = await pages.create( + { space_id: s.id, slug: 'a', title: 'A', body_md: 'hello' }, owner + ); + expect(p.body_md).toBe('hello'); + const revs = await pages.listRevisions(p.id); + expect(revs).toHaveLength(1); + expect(revs[0].body_md).toBe('hello'); + }); + + it('updating body adds a revision', async () => { + const s = await spaces.create({ slug: 'h', name: 'H' }, owner); + const p = await pages.create( + { space_id: s.id, slug: 'a', title: 'A', body_md: 'v1' }, owner + ); + await pages.update(p.id, { body_md: 'v2' }, owner); + const revs = await pages.listRevisions(p.id); + expect(revs).toHaveLength(2); + expect(revs[0].body_md).toBe('v2'); // newest first + }); +}); +``` + +- [ ] **Step 2: Implement `lib/db/repos/pages.js`** + +```javascript +import { pool } from '../pool.js'; +import { recordAudit } from './audit_stub.js'; + +async function snapshot(client, page_id, body_md, edited_by) { + await client.query( + `INSERT INTO page_revisions(page_id, body_md, edited_by) + VALUES($1,$2,$3)`, + [page_id, body_md, edited_by || null] + ); +} + +export async function create({ space_id, slug, title, body_md = '', parent_id }, actor) { + const client = await pool.connect(); + try { + await client.query('BEGIN'); + const { rows: [r] } = await client.query( + `INSERT INTO pages(space_id, slug, title, body_md, parent_id) + VALUES($1,$2,$3,$4,$5) RETURNING *`, + [space_id, slug, title, body_md, parent_id || null] + ); + await snapshot(client, r.id, body_md, actor?.kind); + await client.query('COMMIT'); + await recordAudit(actor, 'create', 'page', r.id, null, r); + return r; + } catch (e) { + await client.query('ROLLBACK'); throw e; + } finally { + client.release(); + } +} + +export async function getById(id) { + const { rows: [r] } = await pool.query(`SELECT * FROM pages WHERE id=$1`, [id]); + return r; +} + +export async function getBySlug(space_id, slug) { + const { rows: [r] } = await pool.query( + `SELECT * FROM pages WHERE space_id=$1 AND slug=$2`, [space_id, slug] + ); + return r; +} + +export async function listBySpace(space_id) { + const { rows } = await pool.query( + `SELECT id, space_id, slug, title, parent_id, updated_at + FROM pages WHERE space_id=$1 ORDER BY title`, [space_id] + ); + return rows; +} + +export async function listRevisions(page_id) { + const { rows } = await pool.query( + `SELECT * FROM page_revisions WHERE page_id=$1 ORDER BY created_at DESC`, + [page_id] + ); + return rows; +} + +export async function update(id, patch, actor) { + const before = await getById(id); + const client = await pool.connect(); + try { + await client.query('BEGIN'); + const fields = ['slug','title','body_md','body_html','parent_id','embedding']; + const sets = [], vals = []; + let i = 1; + for (const f of fields) { + if (patch[f] !== undefined) { sets.push(`${f}=$${i++}`); vals.push(patch[f]); } + } + sets.push(`updated_at=now()`); + vals.push(id); + const { rows: [r] } = await client.query( + `UPDATE pages SET ${sets.join(', ')} WHERE id=$${i} RETURNING *`, + vals + ); + if (patch.body_md !== undefined && patch.body_md !== before.body_md) { + await snapshot(client, id, patch.body_md, actor?.kind); + } + await client.query('COMMIT'); + await recordAudit(actor, 'update', 'page', id, before, r); + return r; + } catch (e) { + await client.query('ROLLBACK'); throw e; + } finally { + client.release(); + } +} + +export async function del(id, actor) { + const before = await getById(id); + await pool.query(`DELETE FROM pages WHERE id=$1`, [id]); + await recordAudit(actor, 'delete', 'page', id, before, null); +} +``` + +- [ ] **Step 3: Write refs tests** + +```javascript +// tests/repos/refs.test.js +import { describe, it, expect, beforeEach } from 'vitest'; +import { resetDb } from '../helpers/db.js'; +import { migrateUp } from '../../lib/db/migrate.js'; +import * as spaces from '../../lib/db/repos/spaces.js'; +import * as refs from '../../lib/db/repos/refs.js'; + +const owner = { kind: 'user', id: null }; +beforeEach(async () => { await resetDb(); await migrateUp(); }); + +describe('refs repo', () => { + it('creates a url ref', async () => { + const s = await spaces.create({ slug: 'h', name: 'H' }, owner); + const r = await refs.create({ + space_id: s.id, kind: 'url', + source_url: 'https://example.com', + title: 'Ex', source_kind: 'manual' + }, owner); + expect(r.kind).toBe('url'); + expect(r.status).toBe('ingested'); + }); + + it('idempotent upsert by (source_kind, external_id)', async () => { + const s = await spaces.create({ slug: 'h', name: 'H' }, owner); + const r1 = await refs.upsertByExternal({ + space_id: s.id, kind: 'url', source_url: 'https://e.com', + source_kind: 'karakeep', external_id: 'kk-123', title: 'v1' + }, owner); + const r2 = await refs.upsertByExternal({ + space_id: s.id, kind: 'url', source_url: 'https://e.com', + source_kind: 'karakeep', external_id: 'kk-123', title: 'v2' + }, owner); + expect(r2.id).toBe(r1.id); + expect(r2.title).toBe('v2'); + }); +}); +``` + +- [ ] **Step 4: Implement `lib/db/repos/refs.js`** + +```javascript +import { pool } from '../pool.js'; +import { recordAudit } from './audit_stub.js'; + +const FIELDS = [ + 'space_id','kind','source_url','title','description','summary', + 'body_text','blob_path','thumbnail','metadata','embedding','status', + 'source_kind','external_id','captured_at' +]; + +export async function create(input, actor) { + const cols = [], vals = [], placeholders = []; + let i = 1; + for (const f of FIELDS) { + if (input[f] !== undefined) { + cols.push(f); vals.push(input[f]); placeholders.push(`$${i++}`); + } + } + const { rows: [r] } = await pool.query( + `INSERT INTO refs(${cols.join(',')}) + VALUES(${placeholders.join(',')}) RETURNING *`, + vals + ); + await recordAudit(actor, 'create', 'ref', r.id, null, r); + return r; +} + +export async function getById(id) { + const { rows: [r] } = await pool.query(`SELECT * FROM refs WHERE id=$1`, [id]); + return r; +} + +export async function upsertByExternal(input, actor) { + const { source_kind, external_id } = input; + if (!source_kind || !external_id) { + throw new Error('upsertByExternal requires source_kind + external_id'); + } + const { rows: [existing] } = await pool.query( + `SELECT * FROM refs WHERE source_kind=$1 AND external_id=$2`, + [source_kind, external_id] + ); + if (existing) { + return update(existing.id, input, actor); + } + return create(input, actor); +} + +export async function list({ space_id, kind, limit = 100, offset = 0 } = {}) { + const where = [], vals = []; + let i = 1; + if (space_id) { where.push(`space_id=$${i++}`); vals.push(space_id); } + if (kind) { where.push(`kind=$${i++}`); vals.push(kind); } + const w = where.length ? `WHERE ${where.join(' AND ')}` : ''; + vals.push(limit, offset); + const { rows } = await pool.query( + `SELECT * FROM refs ${w} ORDER BY captured_at DESC LIMIT $${i++} OFFSET $${i}`, + vals + ); + return rows; +} + +export async function update(id, patch, actor) { + const before = await getById(id); + const sets = [], vals = []; + let i = 1; + for (const f of FIELDS) { + if (patch[f] !== undefined) { sets.push(`${f}=$${i++}`); vals.push(patch[f]); } + } + sets.push(`updated_at=now()`); + vals.push(id); + const { rows: [r] } = await pool.query( + `UPDATE refs SET ${sets.join(', ')} WHERE id=$${i} RETURNING *`, + vals + ); + await recordAudit(actor, 'update', 'ref', id, before, r); + return r; +} + +export async function del(id, actor) { + const before = await getById(id); + await pool.query(`DELETE FROM refs WHERE id=$1`, [id]); + await recordAudit(actor, 'delete', 'ref', id, before, null); +} +``` + +- [ ] **Step 5: Run tests — expect PASS** + +```bash +npm test -- tests/repos/pages.test.js tests/repos/refs.test.js +``` + +- [ ] **Step 6: Commit** + +```bash +git add lib/db/repos/pages.js lib/db/repos/refs.js tests/repos/pages.test.js tests/repos/refs.test.js +git commit -m "feat(repos): pages with auto-revisions, refs with upsertByExternal" +``` + +--- + +## Task 10: Migration 003 — Resources + Source Docs + +**Files:** +- Create: `/project/src/void-v2/lib/db/migrations/003_resources.sql` +- Create: `/project/src/void-v2/tests/db/migration_003.test.js` + +- [ ] **Step 1: Write the test** + +```javascript +// tests/db/migration_003.test.js +import { describe, it, expect, beforeAll } from 'vitest'; +import { resetDb, withClient } from '../helpers/db.js'; +import { migrateUp } from '../../lib/db/migrate.js'; + +describe('migration 003 — resources', () => { + beforeAll(async () => { await resetDb(); await migrateUp(); }); + + it('creates resources, resource_dependencies, resource_credentials, source_docs', async () => { + await withClient(async (c) => { + for (const t of ['resources','resource_dependencies','resource_credentials','source_docs']) { + const { rows } = await c.query( + `SELECT to_regclass('public.' || $1) AS t;`, [t] + ); + expect(rows[0].t).toBe(t); + } + }); + }); +}); +``` + +- [ ] **Step 2: Write `003_resources.sql`** + +```sql +CREATE TABLE resources ( + id uuid PRIMARY KEY DEFAULT gen_random_uuid(), + space_id uuid NOT NULL REFERENCES spaces(id) ON DELETE CASCADE, + slug text NOT NULL, + name text NOT NULL, + runtime_type text NOT NULL + CHECK (runtime_type IN ('lxc','vm','docker','bare-metal')), + host text, + url text, + version text, + status text NOT NULL DEFAULT 'unknown' + CHECK (status IN ('running','stopped','down','unknown')), + monitoring jsonb NOT NULL DEFAULT '{}'::jsonb, + metadata jsonb NOT NULL DEFAULT '{}'::jsonb, + last_check timestamptz, + maintenance_until timestamptz, + created_at timestamptz NOT NULL DEFAULT now(), + updated_at timestamptz NOT NULL DEFAULT now(), + UNIQUE (space_id, slug) +); + +CREATE TABLE resource_dependencies ( + resource_id uuid NOT NULL REFERENCES resources(id) ON DELETE CASCADE, + depends_on uuid NOT NULL REFERENCES resources(id) ON DELETE CASCADE, + kind text, + PRIMARY KEY (resource_id, depends_on), + CHECK (resource_id <> depends_on) +); + +CREATE TABLE resource_credentials ( + id uuid PRIMARY KEY DEFAULT gen_random_uuid(), + resource_id uuid NOT NULL REFERENCES resources(id) ON DELETE CASCADE, + label text NOT NULL, + vault_path text NOT NULL, + kind text, + notes text, + created_at timestamptz NOT NULL DEFAULT now() +); + +CREATE TABLE source_docs ( + id uuid PRIMARY KEY DEFAULT gen_random_uuid(), + resource_id uuid REFERENCES resources(id) ON DELETE CASCADE, + name text NOT NULL, + upstream_url text NOT NULL, + version text, + format text, + sync_source text, + local_path text, + body_text text, + embedding vector(1024), + last_synced timestamptz, + metadata jsonb NOT NULL DEFAULT '{}'::jsonb, + created_at timestamptz NOT NULL DEFAULT now(), + updated_at timestamptz NOT NULL DEFAULT now() +); + +CREATE INDEX idx_resources_space ON resources(space_id); +CREATE INDEX idx_source_docs_resource ON source_docs(resource_id); +CREATE INDEX idx_source_docs_fts ON source_docs + USING GIN (to_tsvector('english', coalesce(body_text,''))); +CREATE INDEX idx_source_docs_embed ON source_docs + USING hnsw (embedding vector_cosine_ops); +``` + +- [ ] **Step 3: Run + commit** + +```bash +npm test -- tests/db/migration_003.test.js +git add lib/db/migrations/003_resources.sql tests/db/migration_003.test.js +git commit -m "feat(schema): 003 — resources, deps, credentials, source_docs" +``` + +--- + +## Task 11: Repos — resources, source_docs + +**Files:** +- Create: `/project/src/void-v2/lib/db/repos/resources.js` +- Create: `/project/src/void-v2/lib/db/repos/source_docs.js` +- Create: `/project/src/void-v2/tests/repos/resources.test.js` +- Create: `/project/src/void-v2/tests/repos/source_docs.test.js` + +Follow the established pattern. Resources additionally exposes `addDependency(resource_id, depends_on, kind)`, `removeDependency`, `listDependencies(resource_id)`, `addCredential`, `listCredentials`. + +- [ ] **Step 1: Write resources tests** + +```javascript +// tests/repos/resources.test.js +import { describe, it, expect, beforeEach } from 'vitest'; +import { resetDb } from '../helpers/db.js'; +import { migrateUp } from '../../lib/db/migrate.js'; +import * as spaces from '../../lib/db/repos/spaces.js'; +import * as resources from '../../lib/db/repos/resources.js'; + +const owner = { kind: 'user', id: null }; +beforeEach(async () => { await resetDb(); await migrateUp(); }); + +describe('resources repo', () => { + it('creates a resource', async () => { + const s = await spaces.create({ slug: 'h', name: 'H' }, owner); + const r = await resources.create({ + space_id: s.id, slug: 'kk', name: 'Karakeep', + runtime_type: 'docker', host: '192.168.1.230', url: 'https://karakeep.hynesy.com' + }, owner); + expect(r.status).toBe('unknown'); + }); + + it('addDependency creates a link, prevents self-dep', async () => { + const s = await spaces.create({ slug: 'h', name: 'H' }, owner); + const a = await resources.create({ space_id: s.id, slug: 'a', name: 'A', runtime_type: 'lxc' }, owner); + const b = await resources.create({ space_id: s.id, slug: 'b', name: 'B', runtime_type: 'lxc' }, owner); + await resources.addDependency(a.id, b.id, 'data'); + const deps = await resources.listDependencies(a.id); + expect(deps).toHaveLength(1); + expect(deps[0].depends_on).toBe(b.id); + await expect(resources.addDependency(a.id, a.id, 'self')).rejects.toThrow(); + }); +}); +``` + +- [ ] **Step 2: Implement `lib/db/repos/resources.js`** + +```javascript +import { pool } from '../pool.js'; +import { recordAudit } from './audit_stub.js'; + +const FIELDS = ['space_id','slug','name','runtime_type','host','url','version','status','monitoring','metadata','last_check','maintenance_until']; + +export async function create(input, actor) { + const cols = [], vals = [], ph = []; + let i = 1; + for (const f of FIELDS) { + if (input[f] !== undefined) { cols.push(f); vals.push(input[f]); ph.push(`$${i++}`); } + } + const { rows: [r] } = await pool.query( + `INSERT INTO resources(${cols.join(',')}) VALUES(${ph.join(',')}) RETURNING *`, + vals + ); + await recordAudit(actor, 'create', 'resource', r.id, null, r); + return r; +} + +export async function getById(id) { + const { rows: [r] } = await pool.query(`SELECT * FROM resources WHERE id=$1`, [id]); + return r; +} + +export async function listBySpace(space_id) { + const { rows } = await pool.query( + `SELECT * FROM resources WHERE space_id=$1 ORDER BY name`, [space_id] + ); + return rows; +} + +export async function update(id, patch, actor) { + const before = await getById(id); + const sets = [], vals = []; + let i = 1; + for (const f of FIELDS) { + if (patch[f] !== undefined) { sets.push(`${f}=$${i++}`); vals.push(patch[f]); } + } + sets.push(`updated_at=now()`); + vals.push(id); + const { rows: [r] } = await pool.query( + `UPDATE resources SET ${sets.join(', ')} WHERE id=$${i} RETURNING *`, + vals + ); + await recordAudit(actor, 'update', 'resource', id, before, r); + return r; +} + +export async function del(id, actor) { + const before = await getById(id); + await pool.query(`DELETE FROM resources WHERE id=$1`, [id]); + await recordAudit(actor, 'delete', 'resource', id, before, null); +} + +export async function addDependency(resource_id, depends_on, kind) { + if (resource_id === depends_on) throw new Error('resource cannot depend on itself'); + await pool.query( + `INSERT INTO resource_dependencies(resource_id, depends_on, kind) + VALUES($1,$2,$3) ON CONFLICT DO NOTHING`, + [resource_id, depends_on, kind || null] + ); +} + +export async function removeDependency(resource_id, depends_on) { + await pool.query( + `DELETE FROM resource_dependencies WHERE resource_id=$1 AND depends_on=$2`, + [resource_id, depends_on] + ); +} + +export async function listDependencies(resource_id) { + const { rows } = await pool.query( + `SELECT * FROM resource_dependencies WHERE resource_id=$1`, [resource_id] + ); + return rows; +} + +export async function addCredential(resource_id, { label, vault_path, kind, notes }) { + const { rows: [r] } = await pool.query( + `INSERT INTO resource_credentials(resource_id, label, vault_path, kind, notes) + VALUES($1,$2,$3,$4,$5) RETURNING *`, + [resource_id, label, vault_path, kind || null, notes || null] + ); + return r; +} + +export async function listCredentials(resource_id) { + const { rows } = await pool.query( + `SELECT id, resource_id, label, vault_path, kind, notes, created_at + FROM resource_credentials WHERE resource_id=$1 ORDER BY label`, + [resource_id] + ); + return rows; +} +``` + +- [ ] **Step 3: Write source_docs repo + tests (mirror pattern)** + +```javascript +// tests/repos/source_docs.test.js +import { describe, it, expect, beforeEach } from 'vitest'; +import { resetDb } from '../helpers/db.js'; +import { migrateUp } from '../../lib/db/migrate.js'; +import * as spaces from '../../lib/db/repos/spaces.js'; +import * as resources from '../../lib/db/repos/resources.js'; +import * as sourceDocs from '../../lib/db/repos/source_docs.js'; + +const owner = { kind: 'user', id: null }; +beforeEach(async () => { await resetDb(); await migrateUp(); }); + +describe('source_docs repo', () => { + it('creates a source doc bound to a resource', async () => { + const s = await spaces.create({ slug: 'h', name: 'H' }, owner); + const r = await resources.create({ space_id: s.id, slug: 'kk', name: 'KK', runtime_type: 'docker' }, owner); + const sd = await sourceDocs.create({ + resource_id: r.id, name: 'Karakeep docs', + upstream_url: 'https://docs.karakeep.app', version: '0.20', format: 'markdown' + }, owner); + expect(sd.resource_id).toBe(r.id); + }); +}); +``` + +```javascript +// lib/db/repos/source_docs.js +import { pool } from '../pool.js'; +import { recordAudit } from './audit_stub.js'; + +const FIELDS = ['resource_id','name','upstream_url','version','format','sync_source','local_path','body_text','embedding','last_synced','metadata']; + +export async function create(input, actor) { + const cols = [], vals = [], ph = []; + let i = 1; + for (const f of FIELDS) { + if (input[f] !== undefined) { cols.push(f); vals.push(input[f]); ph.push(`$${i++}`); } + } + const { rows: [r] } = await pool.query( + `INSERT INTO source_docs(${cols.join(',')}) VALUES(${ph.join(',')}) RETURNING *`, + vals + ); + await recordAudit(actor, 'create', 'source_doc', r.id, null, r); + return r; +} + +export async function getById(id) { + const { rows: [r] } = await pool.query(`SELECT * FROM source_docs WHERE id=$1`, [id]); + return r; +} + +export async function listByResource(resource_id) { + const { rows } = await pool.query( + `SELECT * FROM source_docs WHERE resource_id=$1 ORDER BY name`, [resource_id] + ); + return rows; +} + +export async function update(id, patch, actor) { + const before = await getById(id); + const sets = [], vals = []; + let i = 1; + for (const f of FIELDS) { + if (patch[f] !== undefined) { sets.push(`${f}=$${i++}`); vals.push(patch[f]); } + } + sets.push(`updated_at=now()`); + vals.push(id); + const { rows: [r] } = await pool.query( + `UPDATE source_docs SET ${sets.join(', ')} WHERE id=$${i} RETURNING *`, + vals + ); + await recordAudit(actor, 'update', 'source_doc', id, before, r); + return r; +} + +export async function del(id, actor) { + const before = await getById(id); + await pool.query(`DELETE FROM source_docs WHERE id=$1`, [id]); + await recordAudit(actor, 'delete', 'source_doc', id, before, null); +} +``` + +- [ ] **Step 4: Run tests + commit** + +```bash +npm test -- tests/repos/resources.test.js tests/repos/source_docs.test.js +git add lib/db/repos/resources.js lib/db/repos/source_docs.js tests/repos/resources.test.js tests/repos/source_docs.test.js +git commit -m "feat(repos): resources (+ deps + creds) and source_docs" +``` + +--- + +## Task 12: Migration 004 — Agents + Conversations + Messages + +**Files:** +- Create: `/project/src/void-v2/lib/db/migrations/004_agents.sql` +- Create: `/project/src/void-v2/tests/db/migration_004.test.js` + +- [ ] **Step 1: Write test (existence check on all three tables)** + +```javascript +// tests/db/migration_004.test.js +import { describe, it, expect, beforeAll } from 'vitest'; +import { resetDb, withClient } from '../helpers/db.js'; +import { migrateUp } from '../../lib/db/migrate.js'; + +describe('migration 004 — agents', () => { + beforeAll(async () => { await resetDb(); await migrateUp(); }); + + it('creates agents, agent_tokens, conversations, messages', async () => { + await withClient(async (c) => { + for (const t of ['agents','agent_tokens','conversations','messages']) { + const { rows } = await c.query(`SELECT to_regclass('public.' || $1) AS t;`, [t]); + expect(rows[0].t).toBe(t); + } + }); + }); +}); +``` + +- [ ] **Step 2: Write `004_agents.sql`** + +```sql +CREATE TABLE agents ( + id uuid PRIMARY KEY DEFAULT gen_random_uuid(), + slug text NOT NULL UNIQUE, + name text NOT NULL, + kind text NOT NULL + CHECK (kind IN ('claude','ollama','mastra','mcp-client','external')), + model text, + persona_path text, + capabilities jsonb NOT NULL DEFAULT '{"read":true,"suggest":true,"write":false}'::jsonb, + scopes jsonb NOT NULL DEFAULT '{}'::jsonb, + created_at timestamptz NOT NULL DEFAULT now() +); + +CREATE TABLE agent_tokens ( + id uuid PRIMARY KEY DEFAULT gen_random_uuid(), + agent_id uuid NOT NULL REFERENCES agents(id) ON DELETE CASCADE, + label text, + token_hash text NOT NULL, + last_used timestamptz, + created_at timestamptz NOT NULL DEFAULT now(), + revoked_at timestamptz +); + +CREATE TABLE conversations ( + id uuid PRIMARY KEY DEFAULT gen_random_uuid(), + title text, + agent_id uuid REFERENCES agents(id) ON DELETE SET NULL, + participants text[] NOT NULL DEFAULT '{}', + status text NOT NULL DEFAULT 'open' + CHECK (status IN ('open','summarized','archived')), + summary text, + metadata jsonb NOT NULL DEFAULT '{}'::jsonb, + embedding vector(1024), + started_at timestamptz NOT NULL DEFAULT now(), + ended_at timestamptz +); + +CREATE TABLE messages ( + id uuid PRIMARY KEY DEFAULT gen_random_uuid(), + conversation_id uuid NOT NULL REFERENCES conversations(id) ON DELETE CASCADE, + role text NOT NULL CHECK (role IN ('user','assistant','system','tool')), + agent_id uuid REFERENCES agents(id) ON DELETE SET NULL, + body text NOT NULL, + metadata jsonb NOT NULL DEFAULT '{}'::jsonb, + created_at timestamptz NOT NULL DEFAULT now() +); + +CREATE INDEX idx_messages_conv ON messages(conversation_id, created_at); +CREATE INDEX idx_messages_fts ON messages USING GIN (to_tsvector('english', body)); +CREATE INDEX idx_agent_tokens_hash ON agent_tokens(token_hash) WHERE revoked_at IS NULL; +``` + +- [ ] **Step 3: Run + commit** + +```bash +npm test -- tests/db/migration_004.test.js +git add lib/db/migrations/004_agents.sql tests/db/migration_004.test.js +git commit -m "feat(schema): 004 — agents, tokens, conversations, messages" +``` + +--- + +## Task 13: Repos — agents, conversations, messages + +**Files:** +- Create: `/project/src/void-v2/lib/db/repos/agents.js` +- Create: `/project/src/void-v2/lib/db/repos/conversations.js` +- Create: `/project/src/void-v2/lib/db/repos/messages.js` +- Create tests for each. + +Agents repo additionally exposes: +- `createToken(agent_id, label)` — generates random token, returns plaintext once, stores bcrypt hash +- `verifyToken(plaintext)` — finds matching unrevoked token by hash, returns agent or null +- `revokeToken(token_id)` +- `setCapabilities(agent_id, capabilities, scopes)` + +- [ ] **Step 1: Write agents tests** + +```javascript +// tests/repos/agents.test.js +import { describe, it, expect, beforeEach } from 'vitest'; +import { resetDb } from '../helpers/db.js'; +import { migrateUp } from '../../lib/db/migrate.js'; +import * as agents from '../../lib/db/repos/agents.js'; + +const owner = { kind: 'user', id: null }; +beforeEach(async () => { await resetDb(); await migrateUp(); }); + +describe('agents repo', () => { + it('creates agent with default capabilities', async () => { + const a = await agents.create({ slug: 'mercy', name: 'Mercy', kind: 'claude' }, owner); + expect(a.capabilities.read).toBe(true); + expect(a.capabilities.write).toBe(false); + }); + + it('createToken returns plaintext, verifyToken finds the agent', async () => { + const a = await agents.create({ slug: 'mercy', name: 'Mercy', kind: 'claude' }, owner); + const { token, id } = await agents.createToken(a.id, 'default'); + expect(token).toMatch(/^vk_/); // prefix 'vk_' = void key + const found = await agents.verifyToken(token); + expect(found?.id).toBe(a.id); + }); + + it('revoked token does not verify', async () => { + const a = await agents.create({ slug: 'mercy', name: 'Mercy', kind: 'claude' }, owner); + const { token, id } = await agents.createToken(a.id, 'default'); + await agents.revokeToken(id); + expect(await agents.verifyToken(token)).toBeNull(); + }); + + it('setCapabilities updates the jsonb', async () => { + const a = await agents.create({ slug: 'mercy', name: 'Mercy', kind: 'claude' }, owner); + const u = await agents.setCapabilities(a.id, { read: true, suggest: true, write: true }, { pages: true }); + expect(u.capabilities.write).toBe(true); + expect(u.scopes.pages).toBe(true); + }); +}); +``` + +- [ ] **Step 2: Implement `lib/db/repos/agents.js`** + +```javascript +import crypto from 'node:crypto'; +import bcrypt from 'bcrypt'; +import { pool } from '../pool.js'; +import { recordAudit } from './audit_stub.js'; + +const FIELDS = ['slug','name','kind','model','persona_path','capabilities','scopes']; + +export async function create(input, actor) { + const cols = [], vals = [], ph = []; + let i = 1; + for (const f of FIELDS) { + if (input[f] !== undefined) { cols.push(f); vals.push(input[f]); ph.push(`$${i++}`); } + } + const { rows: [r] } = await pool.query( + `INSERT INTO agents(${cols.join(',')}) VALUES(${ph.join(',')}) RETURNING *`, + vals + ); + await recordAudit(actor, 'create', 'agent', r.id, null, r); + return r; +} + +export async function getById(id) { + const { rows: [r] } = await pool.query(`SELECT * FROM agents WHERE id=$1`, [id]); + return r; +} + +export async function getBySlug(slug) { + const { rows: [r] } = await pool.query(`SELECT * FROM agents WHERE slug=$1`, [slug]); + return r; +} + +export async function list() { + const { rows } = await pool.query(`SELECT * FROM agents ORDER BY name`); + return rows; +} + +export async function setCapabilities(id, capabilities, scopes) { + const { rows: [r] } = await pool.query( + `UPDATE agents SET capabilities=$1, scopes=$2 WHERE id=$3 RETURNING *`, + [capabilities, scopes || {}, id] + ); + return r; +} + +export async function createToken(agent_id, label) { + const plaintext = 'vk_' + crypto.randomBytes(32).toString('base64url'); + const token_hash = await bcrypt.hash(plaintext, 12); + const { rows: [t] } = await pool.query( + `INSERT INTO agent_tokens(agent_id, label, token_hash) + VALUES($1,$2,$3) RETURNING id`, + [agent_id, label || null, token_hash] + ); + return { token: plaintext, id: t.id }; +} + +export async function verifyToken(plaintext) { + if (!plaintext?.startsWith('vk_')) return null; + // Naïve approach: load all unrevoked tokens, bcrypt-compare. Fine at small N. + // Switch to a lookup index if agent count grows. + const { rows } = await pool.query( + `SELECT t.id, t.token_hash, t.agent_id, a.* + FROM agent_tokens t JOIN agents a ON a.id = t.agent_id + WHERE t.revoked_at IS NULL` + ); + for (const row of rows) { + if (await bcrypt.compare(plaintext, row.token_hash)) { + await pool.query( + `UPDATE agent_tokens SET last_used=now() WHERE id=$1`, [row.id] + ); + // Strip token_hash from returned agent record + const { token_hash, ...agent } = row; + return agent; + } + } + return null; +} + +export async function revokeToken(token_id) { + await pool.query( + `UPDATE agent_tokens SET revoked_at=now() WHERE id=$1`, [token_id] + ); +} +``` + +- [ ] **Step 3: Write conversations + messages repos + tests** + +```javascript +// tests/repos/conversations.test.js +import { describe, it, expect, beforeEach } from 'vitest'; +import { resetDb } from '../helpers/db.js'; +import { migrateUp } from '../../lib/db/migrate.js'; +import * as agents from '../../lib/db/repos/agents.js'; +import * as conversations from '../../lib/db/repos/conversations.js'; +import * as messages from '../../lib/db/repos/messages.js'; + +const owner = { kind: 'user', id: null }; +beforeEach(async () => { await resetDb(); await migrateUp(); }); + +describe('conversations + messages', () => { + it('creates a conversation and appends messages', async () => { + const a = await agents.create({ slug: 'mercy', name: 'Mercy', kind: 'claude' }, owner); + const c = await conversations.create({ title: 'hi', agent_id: a.id }, owner); + const m1 = await messages.append(c.id, { role: 'user', body: 'hello' }); + const m2 = await messages.append(c.id, { role: 'assistant', body: 'hi', agent_id: a.id }); + const list = await messages.listByConversation(c.id); + expect(list).toHaveLength(2); + expect(list[0].id).toBe(m1.id); // ordered by created_at + }); +}); +``` + +```javascript +// lib/db/repos/conversations.js +import { pool } from '../pool.js'; +import { recordAudit } from './audit_stub.js'; + +export async function create({ title, agent_id, participants, metadata }, actor) { + const { rows: [r] } = await pool.query( + `INSERT INTO conversations(title, agent_id, participants, metadata) + VALUES($1,$2,$3,$4) RETURNING *`, + [title || null, agent_id || null, participants || [], metadata || {}] + ); + await recordAudit(actor, 'create', 'conversation', r.id, null, r); + return r; +} + +export async function getById(id) { + const { rows: [r] } = await pool.query(`SELECT * FROM conversations WHERE id=$1`, [id]); + return r; +} + +export async function list({ limit = 50, offset = 0 } = {}) { + const { rows } = await pool.query( + `SELECT * FROM conversations ORDER BY started_at DESC LIMIT $1 OFFSET $2`, + [limit, offset] + ); + return rows; +} + +export async function setStatus(id, status, actor) { + const before = await getById(id); + const { rows: [r] } = await pool.query( + `UPDATE conversations SET status=$1, ended_at = CASE WHEN $1='archived' THEN now() ELSE ended_at END + WHERE id=$2 RETURNING *`, + [status, id] + ); + await recordAudit(actor, 'update', 'conversation', id, before, r); + return r; +} + +export async function setSummary(id, summary) { + const { rows: [r] } = await pool.query( + `UPDATE conversations SET summary=$1, status='summarized' WHERE id=$2 RETURNING *`, + [summary, id] + ); + return r; +} +``` + +```javascript +// lib/db/repos/messages.js +import { pool } from '../pool.js'; + +export async function append(conversation_id, { role, body, agent_id, metadata }) { + const { rows: [r] } = await pool.query( + `INSERT INTO messages(conversation_id, role, body, agent_id, metadata) + VALUES($1,$2,$3,$4,$5) RETURNING *`, + [conversation_id, role, body, agent_id || null, metadata || {}] + ); + return r; +} + +export async function listByConversation(conversation_id, { limit = 1000 } = {}) { + const { rows } = await pool.query( + `SELECT * FROM messages WHERE conversation_id=$1 ORDER BY created_at LIMIT $2`, + [conversation_id, limit] + ); + return rows; +} +``` + +- [ ] **Step 4: Run + commit** + +```bash +npm test -- tests/repos/agents.test.js tests/repos/conversations.test.js +git add lib/db/repos/agents.js lib/db/repos/conversations.js lib/db/repos/messages.js tests/repos/agents.test.js tests/repos/conversations.test.js +git commit -m "feat(repos): agents (+ tokens + caps), conversations, messages" +``` + +--- + +## Task 14: Migration 005 — Cross-cutting (tags, entity_tags, entity_links, attachments) + +**Files:** +- Create: `/project/src/void-v2/lib/db/migrations/005_cross.sql` +- Create: `/project/src/void-v2/tests/db/migration_005.test.js` + +- [ ] **Step 1: Test (existence checks)** + +```javascript +// tests/db/migration_005.test.js +import { describe, it, expect, beforeAll } from 'vitest'; +import { resetDb, withClient } from '../helpers/db.js'; +import { migrateUp } from '../../lib/db/migrate.js'; + +describe('migration 005 — cross', () => { + beforeAll(async () => { await resetDb(); await migrateUp(); }); + + it('creates tags, entity_tags, entity_links, attachments', async () => { + await withClient(async (c) => { + for (const t of ['tags','entity_tags','entity_links','attachments']) { + const { rows } = await c.query(`SELECT to_regclass('public.' || $1) AS t;`, [t]); + expect(rows[0].t).toBe(t); + } + }); + }); +}); +``` + +- [ ] **Step 2: Write `005_cross.sql`** + +```sql +CREATE TABLE tags ( + id uuid PRIMARY KEY DEFAULT gen_random_uuid(), + name text NOT NULL UNIQUE, + description text, + color text, + created_at timestamptz NOT NULL DEFAULT now() +); + +CREATE TABLE entity_tags ( + entity_type text NOT NULL, + entity_id uuid NOT NULL, + tag_id uuid NOT NULL REFERENCES tags(id) ON DELETE CASCADE, + created_at timestamptz NOT NULL DEFAULT now(), + PRIMARY KEY (entity_type, entity_id, tag_id) +); + +CREATE TABLE entity_links ( + id uuid PRIMARY KEY DEFAULT gen_random_uuid(), + from_type text NOT NULL, + from_id uuid NOT NULL, + to_type text NOT NULL, + to_id uuid NOT NULL, + relation text NOT NULL DEFAULT 'attached', + created_at timestamptz NOT NULL DEFAULT now(), + UNIQUE (from_type, from_id, to_type, to_id, relation) +); + +CREATE TABLE attachments ( + id uuid PRIMARY KEY DEFAULT gen_random_uuid(), + entity_type text, + entity_id uuid, + filename text NOT NULL, + mime_type text, + size_bytes bigint, + blob_path text NOT NULL, + checksum text, + uploaded_at timestamptz NOT NULL DEFAULT now() +); + +CREATE INDEX idx_entity_tags_entity ON entity_tags(entity_type, entity_id); +CREATE INDEX idx_entity_links_from ON entity_links(from_type, from_id); +CREATE INDEX idx_entity_links_to ON entity_links(to_type, to_id); +CREATE INDEX idx_attachments_entity ON attachments(entity_type, entity_id); +``` + +- [ ] **Step 3: Run + commit** + +```bash +npm test -- tests/db/migration_005.test.js +git add lib/db/migrations/005_cross.sql tests/db/migration_005.test.js +git commit -m "feat(schema): 005 — tags, entity_tags, entity_links, attachments" +``` + +--- + +## Task 15: Repos — tags, links, attachments + +**Files:** +- Create: `/project/src/void-v2/lib/db/repos/tags.js` +- Create: `/project/src/void-v2/lib/db/repos/links.js` +- Create: `/project/src/void-v2/lib/db/repos/attachments.js` +- Create tests for each. + +- [ ] **Step 1: Tags repo + tests** + +```javascript +// tests/repos/tags.test.js +import { describe, it, expect, beforeEach } from 'vitest'; +import { resetDb } from '../helpers/db.js'; +import { migrateUp } from '../../lib/db/migrate.js'; +import * as spaces from '../../lib/db/repos/spaces.js'; +import * as tags from '../../lib/db/repos/tags.js'; + +const owner = { kind: 'user', id: null }; +beforeEach(async () => { await resetDb(); await migrateUp(); }); + +describe('tags repo', () => { + it('upserts a tag by name', async () => { + const t1 = await tags.upsert('homelab'); + const t2 = await tags.upsert('homelab'); + expect(t2.id).toBe(t1.id); + }); + + it('attach + detach + listForEntity', async () => { + const s = await spaces.create({ slug: 'h', name: 'H' }, owner); + const t = await tags.upsert('urgent'); + await tags.attach('space', s.id, t.id); + const list = await tags.listForEntity('space', s.id); + expect(list).toHaveLength(1); + await tags.detach('space', s.id, t.id); + expect(await tags.listForEntity('space', s.id)).toHaveLength(0); + }); +}); +``` + +```javascript +// lib/db/repos/tags.js +import { pool } from '../pool.js'; + +export async function upsert(name, { description, color } = {}) { + const { rows: [r] } = await pool.query( + `INSERT INTO tags(name, description, color) + VALUES($1,$2,$3) + ON CONFLICT(name) DO UPDATE SET description=COALESCE(EXCLUDED.description, tags.description), + color=COALESCE(EXCLUDED.color, tags.color) + RETURNING *`, + [name, description || null, color || null] + ); + return r; +} + +export async function list() { + const { rows } = await pool.query(`SELECT * FROM tags ORDER BY name`); + return rows; +} + +export async function attach(entity_type, entity_id, tag_id) { + await pool.query( + `INSERT INTO entity_tags(entity_type, entity_id, tag_id) + VALUES($1,$2,$3) ON CONFLICT DO NOTHING`, + [entity_type, entity_id, tag_id] + ); +} + +export async function detach(entity_type, entity_id, tag_id) { + await pool.query( + `DELETE FROM entity_tags WHERE entity_type=$1 AND entity_id=$2 AND tag_id=$3`, + [entity_type, entity_id, tag_id] + ); +} + +export async function listForEntity(entity_type, entity_id) { + const { rows } = await pool.query( + `SELECT t.* FROM tags t JOIN entity_tags et ON et.tag_id=t.id + WHERE et.entity_type=$1 AND et.entity_id=$2 ORDER BY t.name`, + [entity_type, entity_id] + ); + return rows; +} +``` + +- [ ] **Step 2: Links repo + tests** + +```javascript +// tests/repos/links.test.js +import { describe, it, expect, beforeEach } from 'vitest'; +import { resetDb } from '../helpers/db.js'; +import { migrateUp } from '../../lib/db/migrate.js'; +import * as links from '../../lib/db/repos/links.js'; + +beforeEach(async () => { await resetDb(); await migrateUp(); }); + +describe('links repo', () => { + it('create + listFrom + listTo + remove', async () => { + const a = '11111111-1111-1111-1111-111111111111'; + const b = '22222222-2222-2222-2222-222222222222'; + const link = await links.create('project', a, 'page', b, 'mentions'); + expect(link.relation).toBe('mentions'); + expect(await links.listFrom('project', a)).toHaveLength(1); + expect(await links.listTo('page', b)).toHaveLength(1); + await links.remove(link.id); + expect(await links.listFrom('project', a)).toHaveLength(0); + }); + + it('idempotent on the unique tuple', async () => { + const a = '11111111-1111-1111-1111-111111111111'; + const b = '22222222-2222-2222-2222-222222222222'; + const l1 = await links.create('project', a, 'page', b, 'mentions'); + const l2 = await links.create('project', a, 'page', b, 'mentions'); + expect(l2.id).toBe(l1.id); + }); +}); +``` + +```javascript +// lib/db/repos/links.js +import { pool } from '../pool.js'; + +export async function create(from_type, from_id, to_type, to_id, relation = 'attached') { + const { rows: [r] } = await pool.query( + `INSERT INTO entity_links(from_type, from_id, to_type, to_id, relation) + VALUES($1,$2,$3,$4,$5) + ON CONFLICT (from_type, from_id, to_type, to_id, relation) + DO UPDATE SET relation=EXCLUDED.relation + RETURNING *`, + [from_type, from_id, to_type, to_id, relation] + ); + return r; +} + +export async function listFrom(from_type, from_id) { + const { rows } = await pool.query( + `SELECT * FROM entity_links WHERE from_type=$1 AND from_id=$2`, + [from_type, from_id] + ); + return rows; +} + +export async function listTo(to_type, to_id) { + const { rows } = await pool.query( + `SELECT * FROM entity_links WHERE to_type=$1 AND to_id=$2`, + [to_type, to_id] + ); + return rows; +} + +export async function remove(id) { + await pool.query(`DELETE FROM entity_links WHERE id=$1`, [id]); +} +``` + +- [ ] **Step 3: Attachments repo + tests** + +```javascript +// tests/repos/attachments.test.js +import { describe, it, expect, beforeEach } from 'vitest'; +import { resetDb } from '../helpers/db.js'; +import { migrateUp } from '../../lib/db/migrate.js'; +import * as attachments from '../../lib/db/repos/attachments.js'; + +beforeEach(async () => { await resetDb(); await migrateUp(); }); + +describe('attachments repo', () => { + it('records an attachment', async () => { + const a = await attachments.create({ + entity_type: 'page', + entity_id: '11111111-1111-1111-1111-111111111111', + filename: 'spec.pdf', mime_type: 'application/pdf', + size_bytes: 1024, blob_path: 'aa/abc', checksum: 'abc' + }); + expect(a.filename).toBe('spec.pdf'); + }); +}); +``` + +```javascript +// lib/db/repos/attachments.js +import { pool } from '../pool.js'; + +export async function create({ entity_type, entity_id, filename, mime_type, size_bytes, blob_path, checksum }) { + const { rows: [r] } = await pool.query( + `INSERT INTO attachments(entity_type, entity_id, filename, mime_type, size_bytes, blob_path, checksum) + VALUES($1,$2,$3,$4,$5,$6,$7) RETURNING *`, + [entity_type, entity_id, filename, mime_type || null, size_bytes || null, blob_path, checksum || null] + ); + return r; +} + +export async function listForEntity(entity_type, entity_id) { + const { rows } = await pool.query( + `SELECT * FROM attachments WHERE entity_type=$1 AND entity_id=$2 ORDER BY uploaded_at DESC`, + [entity_type, entity_id] + ); + return rows; +} + +export async function getById(id) { + const { rows: [r] } = await pool.query(`SELECT * FROM attachments WHERE id=$1`, [id]); + return r; +} + +export async function del(id) { + await pool.query(`DELETE FROM attachments WHERE id=$1`, [id]); +} +``` + +- [ ] **Step 4: Run + commit** + +```bash +npm test -- tests/repos/tags.test.js tests/repos/links.test.js tests/repos/attachments.test.js +git add lib/db/repos/tags.js lib/db/repos/links.js lib/db/repos/attachments.js tests/repos/tags.test.js tests/repos/links.test.js tests/repos/attachments.test.js +git commit -m "feat(repos): tags, polymorphic entity_links, attachments" +``` + +--- + +## Task 16: Migration 006 + Real Audit Log + Pending Changes + +**Files:** +- Create: `/project/src/void-v2/lib/db/migrations/006_audit.sql` +- Create: `/project/src/void-v2/lib/db/repos/audit.js` +- Create: `/project/src/void-v2/lib/db/repos/pending_changes.js` +- Modify: `/project/src/void-v2/lib/db/repos/audit_stub.js` — re-export from real `audit.js` +- Create: `/project/src/void-v2/tests/db/migration_006.test.js` +- Create: `/project/src/void-v2/tests/repos/audit.test.js` +- Create: `/project/src/void-v2/tests/repos/pending_changes.test.js` + +- [ ] **Step 1: Migration test + SQL** + +```javascript +// tests/db/migration_006.test.js +import { describe, it, expect, beforeAll } from 'vitest'; +import { resetDb, withClient } from '../helpers/db.js'; +import { migrateUp } from '../../lib/db/migrate.js'; + +describe('migration 006 — audit', () => { + beforeAll(async () => { await resetDb(); await migrateUp(); }); + + it('creates audit_log and pending_changes', async () => { + await withClient(async (c) => { + for (const t of ['audit_log','pending_changes']) { + const { rows } = await c.query(`SELECT to_regclass('public.' || $1) AS t;`, [t]); + expect(rows[0].t).toBe(t); + } + }); + }); +}); +``` + +```sql +-- lib/db/migrations/006_audit.sql +CREATE TABLE audit_log ( + id bigserial PRIMARY KEY, + actor_kind text NOT NULL CHECK (actor_kind IN ('user','agent','cron','worker','system')), + actor_id uuid, + entity_type text NOT NULL, + entity_id uuid, + action text NOT NULL CHECK (action IN ('create','update','delete','suggest','approve','reject')), + diff jsonb, + occurred_at timestamptz NOT NULL DEFAULT now() +); + +CREATE TABLE pending_changes ( + id uuid PRIMARY KEY DEFAULT gen_random_uuid(), + agent_id uuid NOT NULL REFERENCES agents(id) ON DELETE CASCADE, + entity_type text NOT NULL, + entity_id uuid, + action text NOT NULL CHECK (action IN ('create','update','delete')), + payload jsonb NOT NULL, + reason text, + status text NOT NULL DEFAULT 'pending' + CHECK (status IN ('pending','approved','rejected')), + resolved_at timestamptz, + resolved_by text, + created_at timestamptz NOT NULL DEFAULT now() +); + +CREATE INDEX idx_audit_entity ON audit_log(entity_type, entity_id, occurred_at); +CREATE INDEX idx_audit_actor ON audit_log(actor_kind, actor_id, occurred_at); +CREATE INDEX idx_pending_status ON pending_changes(status, created_at) + WHERE status='pending'; +``` + +- [ ] **Step 2: Real audit repo** + +```javascript +// lib/db/repos/audit.js +import { pool } from '../pool.js'; + +const REDACT_KEYS = new Set([ + 'token','token_hash','password','api_key','secret','authorization' +]); + +function redact(obj) { + if (!obj || typeof obj !== 'object') return obj; + if (Array.isArray(obj)) return obj.map(redact); + const out = {}; + for (const [k, v] of Object.entries(obj)) { + if (REDACT_KEYS.has(k.toLowerCase())) out[k] = '[REDACTED]'; + else out[k] = redact(v); + } + return out; +} + +function diff(before, after) { + if (before === null && after === null) return null; + if (before === null) return { kind: 'create', after: redact(after) }; + if (after === null) return { kind: 'delete', before: redact(before) }; + const changed = {}; + for (const k of new Set([...Object.keys(before), ...Object.keys(after)])) { + if (JSON.stringify(before[k]) !== JSON.stringify(after[k])) { + changed[k] = { before: redact(before[k]), after: redact(after[k]) }; + } + } + return Object.keys(changed).length ? { kind: 'update', changes: changed } : null; +} + +export async function recordAudit(actor, action, entity_type, entity_id, before, after) { + const d = diff(before, after); + await pool.query( + `INSERT INTO audit_log(actor_kind, actor_id, entity_type, entity_id, action, diff) + VALUES($1,$2,$3,$4,$5,$6)`, + [actor?.kind || 'system', actor?.id || null, entity_type, entity_id, action, d] + ); +} + +export async function listForEntity(entity_type, entity_id, { limit = 100 } = {}) { + const { rows } = await pool.query( + `SELECT * FROM audit_log WHERE entity_type=$1 AND entity_id=$2 + ORDER BY occurred_at DESC LIMIT $3`, + [entity_type, entity_id, limit] + ); + return rows; +} + +export async function listByActor({ actor_kind, actor_id, limit = 100 } = {}) { + const where = [], vals = []; + let i = 1; + if (actor_kind) { where.push(`actor_kind=$${i++}`); vals.push(actor_kind); } + if (actor_id) { where.push(`actor_id=$${i++}`); vals.push(actor_id); } + vals.push(limit); + const w = where.length ? `WHERE ${where.join(' AND ')}` : ''; + const { rows } = await pool.query( + `SELECT * FROM audit_log ${w} ORDER BY occurred_at DESC LIMIT $${i}`, + vals + ); + return rows; +} +``` + +- [ ] **Step 3: Replace stub with re-export** + +```javascript +// lib/db/repos/audit_stub.js +export { recordAudit } from './audit.js'; +``` + +- [ ] **Step 4: Audit + Pending tests** + +```javascript +// tests/repos/audit.test.js +import { describe, it, expect, beforeEach } from 'vitest'; +import { resetDb } from '../helpers/db.js'; +import { migrateUp } from '../../lib/db/migrate.js'; +import * as spaces from '../../lib/db/repos/spaces.js'; +import * as audit from '../../lib/db/repos/audit.js'; + +const owner = { kind: 'user', id: null }; +beforeEach(async () => { await resetDb(); await migrateUp(); }); + +describe('audit log', () => { + it('creates an audit row on space create', async () => { + const s = await spaces.create({ slug: 'h', name: 'H' }, owner); + const rows = await audit.listForEntity('space', s.id); + expect(rows).toHaveLength(1); + expect(rows[0].action).toBe('create'); + }); + + it('records the diff of an update', async () => { + const s = await spaces.create({ slug: 'h', name: 'H' }, owner); + await spaces.update(s.id, { name: 'Hh' }, owner); + const rows = await audit.listForEntity('space', s.id); + const upd = rows.find(r => r.action === 'update'); + expect(upd.diff.changes.name.before).toBe('H'); + expect(upd.diff.changes.name.after).toBe('Hh'); + }); + + it('redacts known sensitive keys', async () => { + // Simulate by writing audit directly with a sensitive payload + await audit.recordAudit(owner, 'update', 'test', null, + { token: 'secret-abc' }, { token: 'secret-def' } + ); + const rows = await audit.listByActor({}); + const r = rows.find(r => r.entity_type === 'test'); + expect(r.diff.changes.token.before).toBe('[REDACTED]'); + expect(r.diff.changes.token.after).toBe('[REDACTED]'); + }); +}); +``` + +```javascript +// lib/db/repos/pending_changes.js +import { pool } from '../pool.js'; + +export async function create({ agent_id, entity_type, entity_id, action, payload, reason }) { + const { rows: [r] } = await pool.query( + `INSERT INTO pending_changes(agent_id, entity_type, entity_id, action, payload, reason) + VALUES($1,$2,$3,$4,$5,$6) RETURNING *`, + [agent_id, entity_type, entity_id || null, action, payload, reason || null] + ); + return r; +} + +export async function listPending({ limit = 100 } = {}) { + const { rows } = await pool.query( + `SELECT * FROM pending_changes WHERE status='pending' ORDER BY created_at LIMIT $1`, + [limit] + ); + return rows; +} + +export async function getById(id) { + const { rows: [r] } = await pool.query(`SELECT * FROM pending_changes WHERE id=$1`, [id]); + return r; +} + +export async function resolve(id, status, resolved_by) { + const { rows: [r] } = await pool.query( + `UPDATE pending_changes SET status=$1, resolved_at=now(), resolved_by=$2 + WHERE id=$3 AND status='pending' RETURNING *`, + [status, resolved_by || null, id] + ); + return r; +} +``` + +```javascript +// tests/repos/pending_changes.test.js +import { describe, it, expect, beforeEach } from 'vitest'; +import { resetDb } from '../helpers/db.js'; +import { migrateUp } from '../../lib/db/migrate.js'; +import * as agents from '../../lib/db/repos/agents.js'; +import * as pending from '../../lib/db/repos/pending_changes.js'; + +const owner = { kind: 'user', id: null }; +beforeEach(async () => { await resetDb(); await migrateUp(); }); + +describe('pending changes', () => { + it('creates and resolves a pending change', async () => { + const a = await agents.create({ slug: 'mercy', name: 'Mercy', kind: 'claude' }, owner); + const p = await pending.create({ + agent_id: a.id, entity_type: 'page', action: 'create', + payload: { title: 'draft', body_md: 'hi' }, reason: 'inferred from chat' + }); + expect(p.status).toBe('pending'); + const resolved = await pending.resolve(p.id, 'approved', 'owner'); + expect(resolved.status).toBe('approved'); + }); +}); +``` + +- [ ] **Step 5: Run + commit** + +```bash +npm test -- tests/repos/audit.test.js tests/repos/pending_changes.test.js +git add lib/db/migrations/006_audit.sql lib/db/repos/audit.js lib/db/repos/audit_stub.js lib/db/repos/pending_changes.js tests/db/migration_006.test.js tests/repos/audit.test.js tests/repos/pending_changes.test.js +git commit -m "feat: real audit_log with redaction + pending_changes; replace stub" +``` + +--- + +## Task 17: Capability Check Module + +**Files:** +- Create: `/project/src/void-v2/lib/auth/capability.js` +- Create: `/project/src/void-v2/tests/auth/capability.test.js` + +The capability module is the single decision point for "can this actor perform this action on this entity type?" It's called from REST handlers and (later) MCP tools. + +- [ ] **Step 1: Write tests** + +```javascript +// tests/auth/capability.test.js +import { describe, it, expect } from 'vitest'; +import { canAct } from '../../lib/auth/capability.js'; + +const ownerActor = { kind: 'user', id: null }; +const readAgent = { + kind: 'agent', id: 'a1', + capabilities: { read: true, suggest: true, write: false }, + scopes: {} +}; +const writeAgent = { + kind: 'agent', id: 'a2', + capabilities: { read: true, suggest: true, write: true }, + scopes: { pages: true } +}; + +describe('canAct', () => { + it('owner can do anything', () => { + expect(canAct(ownerActor, 'create', 'page')).toBe('allow'); + expect(canAct(ownerActor, 'delete', 'resource')).toBe('allow'); + }); + + it('read-only agent can read', () => { + expect(canAct(readAgent, 'read', 'page')).toBe('allow'); + }); + + it('default agent on create returns "suggest"', () => { + expect(canAct(readAgent, 'create', 'page')).toBe('suggest'); + }); + + it('write-scoped agent can write to its scope', () => { + expect(canAct(writeAgent, 'create', 'page')).toBe('allow'); + }); + + it('write-capable agent without scope still suggests outside it', () => { + expect(canAct(writeAgent, 'create', 'resource')).toBe('suggest'); + }); + + it('agent with no capabilities is deny', () => { + expect(canAct({ kind: 'agent', id: 'x', capabilities: {} }, 'read', 'page')).toBe('deny'); + }); +}); +``` + +- [ ] **Step 2: Implement** + +```javascript +// lib/auth/capability.js +/** + * canAct(actor, action, entity_type) -> 'allow' | 'suggest' | 'deny' + * + * - User (owner): always 'allow'. + * - Agent with capabilities.write=true AND scope present for entity_type: 'allow'. + * - Agent with capabilities.suggest=true: 'suggest' (caller writes to pending_changes). + * - Agent with capabilities.read=true on read actions: 'allow'. + * - Otherwise: 'deny'. + */ +export function canAct(actor, action, entity_type) { + if (!actor) return 'deny'; + if (actor.kind === 'user') return 'allow'; + if (actor.kind === 'cron' || actor.kind === 'worker' || actor.kind === 'system') return 'allow'; + + if (actor.kind !== 'agent') return 'deny'; + + const caps = actor.capabilities || {}; + const scopes = actor.scopes || {}; + + if (action === 'read') return caps.read ? 'allow' : 'deny'; + + const isMutation = ['create','update','delete'].includes(action); + if (!isMutation) return 'deny'; + + if (caps.write && scopes[entity_type]) return 'allow'; + if (caps.suggest) return 'suggest'; + return 'deny'; +} +``` + +- [ ] **Step 3: Run + commit** + +```bash +mkdir -p tests/auth +npm test -- tests/auth/capability.test.js +git add lib/auth/capability.js tests/auth/capability.test.js +git commit -m "feat(auth): capability check — user/cron/worker allow; agents tiered allow/suggest/deny" +``` + +--- + +## Task 18: Owner Token Middleware + +**Files:** +- Create: `/project/src/void-v2/lib/auth/owner.js` +- Create: `/project/src/void-v2/tests/auth/owner.test.js` + +The owner middleware checks `Authorization: Bearer ` against the env var. Anything else fails with 401. Agent-token verification (via the `agents` repo) is a separate middleware added later. + +- [ ] **Step 1: Write tests** + +```javascript +// tests/auth/owner.test.js +import { describe, it, expect, vi } from 'vitest'; +import { ownerOnly } from '../../lib/auth/owner.js'; + +function mockReq(token) { + return { headers: { authorization: token ? `Bearer ${token}` : undefined } }; +} +function mockRes() { + const r = { status: vi.fn().mockReturnThis(), json: vi.fn().mockReturnThis(), end: vi.fn() }; + return r; +} + +describe('ownerOnly middleware', () => { + beforeEach(() => { process.env.OWNER_TOKEN = 'test-token'; }); + + it('rejects missing token', () => { + const res = mockRes(); + const next = vi.fn(); + ownerOnly(mockReq(null), res, next); + expect(res.status).toHaveBeenCalledWith(401); + expect(next).not.toHaveBeenCalled(); + }); + + it('rejects wrong token', () => { + const res = mockRes(); + const next = vi.fn(); + ownerOnly(mockReq('wrong'), res, next); + expect(res.status).toHaveBeenCalledWith(401); + }); + + it('accepts correct token + attaches actor', () => { + const res = mockRes(); + const next = vi.fn(); + const req = mockReq('test-token'); + ownerOnly(req, res, next); + expect(next).toHaveBeenCalled(); + expect(req.actor).toEqual({ kind: 'user', id: null }); + }); +}); +``` + +- [ ] **Step 2: Implement** + +```javascript +// lib/auth/owner.js +export function ownerOnly(req, res, next) { + const expected = process.env.OWNER_TOKEN; + if (!expected) return res.status(500).json({ error: { code: 'no_owner_token', message: 'OWNER_TOKEN not configured' } }); + const auth = req.headers.authorization || ''; + const [scheme, token] = auth.split(' '); + if (scheme !== 'Bearer' || token !== expected) { + return res.status(401).json({ error: { code: 'unauthorized', message: 'invalid token' } }); + } + req.actor = { kind: 'user', id: null }; + next(); +} +``` + +- [ ] **Step 3: Run + commit** + +```bash +npm test -- tests/auth/owner.test.js +git add lib/auth/owner.js tests/auth/owner.test.js +git commit -m "feat(auth): owner-only middleware for single-user bearer auth" +``` + +--- + +## Task 19: Express Bootstrap + /health Endpoint + +**Files:** +- Create: `/project/src/void-v2/server.js` +- Create: `/project/src/void-v2/tests/server.test.js` + +The health endpoint pings the DB pool (`SELECT 1`) and returns version info. It does NOT require auth (probes need to be unauthenticated). + +- [ ] **Step 1: Install supertest** + +```bash +npm install --save-dev supertest +``` + +- [ ] **Step 2: Write test using supertest** + +```javascript +// tests/server.test.js +import { describe, it, expect, beforeAll, afterAll } from 'vitest'; +import request from 'supertest'; +import { createApp } from '../server.js'; +import { resetDb } from './helpers/db.js'; +import { migrateUp } from '../lib/db/migrate.js'; +import { pool } from '../lib/db/pool.js'; + +let app; +beforeAll(async () => { + await resetDb(); await migrateUp(); + process.env.OWNER_TOKEN = 'test-token'; + app = createApp(); +}); +afterAll(async () => { await pool.end(); }); + +describe('server', () => { + it('GET /health returns 200 with db_ok=true', async () => { + const res = await request(app).get('/health'); + expect(res.status).toBe(200); + expect(res.body.db_ok).toBe(true); + expect(res.body.version).toBeDefined(); + }); + + it('GET /api/spaces without token returns 401', async () => { + const res = await request(app).get('/api/spaces'); + expect(res.status).toBe(401); + }); + + it('GET /api/spaces with token returns 200 and empty array', async () => { + const res = await request(app) + .get('/api/spaces') + .set('Authorization', 'Bearer test-token'); + expect(res.status).toBe(200); + expect(res.body).toEqual([]); + }); +}); +``` + +- [ ] **Step 3: Implement `server.js`** + +```javascript +// server.js +import 'dotenv/config'; +import express from 'express'; +import { pool } from './lib/db/pool.js'; +import { ownerOnly } from './lib/auth/owner.js'; +import { log } from './lib/log.js'; +import * as spaces from './lib/db/repos/spaces.js'; + +const VERSION = '2.0.0-alpha.1'; + +export function createApp() { + const app = express(); + app.use(express.json({ limit: '10mb' })); + + // Healthcheck — no auth, used by probes + app.get('/health', async (_req, res) => { + let db_ok = false; + try { + await pool.query('SELECT 1'); + db_ok = true; + } catch (e) { + log.error({ err: e }, 'healthcheck db ping failed'); + } + res.json({ ok: true, db_ok, version: VERSION }); + }); + + // All /api/* requires owner token (for now; agent tokens added later) + app.use('/api', ownerOnly); + + // Smoke endpoint — list spaces. Full CRUD lands in Plan 2. + app.get('/api/spaces', async (_req, res) => { + res.json(await spaces.list()); + }); + + // Generic 404 + app.use((_req, res) => res.status(404).json({ error: { code: 'not_found' } })); + + // Generic error handler + app.use((err, _req, res, _next) => { + log.error({ err }, 'unhandled'); + res.status(500).json({ error: { code: 'internal', message: err.message } }); + }); + + return app; +} + +if (import.meta.url === `file://${process.argv[1]}`) { + const port = process.env.PORT || 3000; + createApp().listen(port, () => log.info({ port }, 'void-server listening')); +} +``` + +- [ ] **Step 4: Run tests + commit** + +```bash +npm test -- tests/server.test.js +git add server.js package.json package-lock.json tests/server.test.js +git commit -m "feat(server): Express bootstrap, /health, ownerOnly on /api, smoke /api/spaces" +``` + +--- + +## Task 20: Smoke Test Against Real DB + +**Files:** +- None (manual verification) + +- [ ] **Step 1: Confirm `.env` on dev box points at `void2-db`** + +```bash +cat /project/src/void-v2/.env +# Expect DATABASE_URL=postgres://void:...@192.168.1.X:5432/void +``` + +- [ ] **Step 2: Run all tests against real DB** + +```bash +cd /project/src/void-v2 +npm test +``` + +Expect ALL tests pass. + +- [ ] **Step 3: Run migrations against real DB** + +```bash +npm run migrate +``` + +Expect: log lines applying 001 → 006, exit code 0. + +- [ ] **Step 4: Start server** + +```bash +OWNER_TOKEN=$(openssl rand -base64 24) npm start & +echo $! > /tmp/void-server.pid +sleep 1 +``` + +- [ ] **Step 5: Curl smoke** + +```bash +curl -s http://localhost:3000/health | jq +# Expect: {"ok":true,"db_ok":true,"version":"2.0.0-alpha.1"} + +curl -s -H "Authorization: Bearer $OWNER_TOKEN" http://localhost:3000/api/spaces | jq +# Expect: [] +``` + +- [ ] **Step 6: Stop server** + +```bash +kill $(cat /tmp/void-server.pid); rm /tmp/void-server.pid +``` + +- [ ] **Step 7: Commit nothing — verification only. Note completion in CHANGELOG.** + +Append to `CHANGELOG.md` under `## [Unreleased]`: + +```markdown +### Added (Plan 1: Foundation) +- LXC provisioning for `void2-db` (Postgres 16 + pgvector) and `void2-app` +- Schema migrations 001-006 covering core, knowledge, resources, agents, cross-cutting, audit +- Repos with capability-checked `actor` parameter and audit trail +- Owner-token bearer auth +- `/health` endpoint +- Test coverage on all repos + migrations + capability + owner middleware +``` + +```bash +git add CHANGELOG.md +git commit -m "docs: changelog entry for Plan 1 completion" +``` + +--- + +## Task 21: systemd Service Unit + Deploy Script + +**Files:** +- Create: `/project/src/void-v2/deploy/void-server.service` +- Create: `/project/src/void-v2/deploy/push.sh` +- Modify: `/project/src/void-v2/deploy/README.md` (append deploy section) + +- [ ] **Step 1: Write systemd unit** + +```ini +# deploy/void-server.service +[Unit] +Description=Void 2.0 server +After=network-online.target +Wants=network-online.target + +[Service] +Type=simple +User=void +WorkingDirectory=/opt/void-server +EnvironmentFile=/opt/void-server/.env +ExecStart=/usr/bin/node /opt/void-server/server.js +Restart=on-failure +RestartSec=5 +StandardOutput=journal +StandardError=journal + +[Install] +WantedBy=multi-user.target +``` + +- [ ] **Step 2: Write `deploy/push.sh`** + +```bash +#!/usr/bin/env bash +set -euo pipefail + +# Push dev source to void2-app (CT 311) and restart service. +# Run from /project/src/void-v2. + +TARGET=${TARGET:-root@192.168.1.Y} # void2-app +REMOTE_DIR=${REMOTE_DIR:-/opt/void-server} + +rsync -avz --delete \ + --exclude node_modules \ + --exclude .git \ + --exclude tests \ + --exclude coverage \ + ./ "$TARGET:$REMOTE_DIR/" + +ssh "$TARGET" "cd $REMOTE_DIR && npm install --omit=dev && systemctl restart void-server" +echo "Deployed." +``` + +```bash +chmod +x deploy/push.sh +``` + +- [ ] **Step 3: Append deploy notes** + +```markdown +## App deploy (CT 311) + +One-time setup on `void2-app`: +- `apt install -y nodejs npm` (or install Node 22 from nodesource) +- `useradd -r -m -d /opt/void-server void` +- `mkdir -p /opt/void-server && chown void: /opt/void-server` +- Copy `void-server.service` to `/etc/systemd/system/void-server.service` +- `systemctl daemon-reload && systemctl enable void-server` +- Create `/opt/void-server/.env` with `DATABASE_URL` + `OWNER_TOKEN` +- Run `deploy/push.sh` from the dev box + +Maintenance: +- Logs: `journalctl -u void-server -f` +- Status: `systemctl status void-server` +- Migrate: `cd /opt/void-server && npm run migrate` +``` + +- [ ] **Step 4: Commit** + +```bash +git add deploy/ +git commit -m "chore(deploy): systemd unit, push.sh, one-time setup notes" +``` + +--- + +## Task 22: Plan 1 Completion Doc + +**Files:** +- Create: `/project/src/void-v2/docs/plan-1-complete.md` + +- [ ] **Step 1: Write completion doc** + +```markdown +# Plan 1 Complete — Foundation + +Done: 2026-XX-XX + +## What landed +- Two LXCs provisioned: `void2-db` (Postgres 16 + pgvector), `void2-app` + (Node 22 + Express) +- Schema migrations 001-006: all entity tables + cross-cutting + audit +- All repos with capability-ready `actor` parameter, audit trail enabled +- Owner bearer-token auth on `/api/*`; `/health` is open +- ~30+ test cases across migrations, repos, capability, owner middleware +- systemd unit + push.sh deploy script + +## What's NOT here +- Routes for every entity (Plan 2) +- MCP server (Plan 5) +- The Void UI (Plan 2) +- Capture workers (Plan 3 + 4) +- Migrations from Void 1.x / BookStack / Karakeep (Plan 7) + +## How to verify +``` +cd /project/src/void-v2 +npm test # all green +npm run migrate # applies 001-006 +OWNER_TOKEN=test npm start & # server up +curl localhost:3000/health # db_ok=true +curl -H "Authorization: Bearer test" localhost:3000/api/spaces # [] +``` + +## Next: Plan 2 — Core REST API + Void UI shell +``` + +- [ ] **Step 2: Commit** + +```bash +git add docs/plan-1-complete.md +git commit -m "docs: Plan 1 completion summary" +``` + +--- + +## Self-Review Checklist + +Run through this once after implementing: + +- [ ] All tests pass (`npm test`) +- [ ] Migrations apply cleanly on a fresh DB (`resetDb` then `migrateUp`) +- [ ] `/health` returns `db_ok: true` against the real DB +- [ ] `/api/spaces` returns 401 without token, 200 with token +- [ ] Audit log records create + update + delete on a space (manual `curl` + `psql` check) +- [ ] `git log --oneline` shows ~20 commits, each scoped and meaningful +- [ ] No `console.log`s left in `lib/` or `server.js` +- [ ] `.env` is gitignored +- [ ] CHANGELOG.md has the Plan 1 entry +- [ ] `docs/plan-1-complete.md` describes the state + +--- + +**Plan 1 produces:** a running Void 2.0 backend with the full schema in place, repos that record audit, owner auth, and a smoke endpoint — but no business endpoints, no UI, no capture, no MCP. That's all Plan 2+. diff --git a/docs/superpowers/plans/2026-05-31-void-v2-plan1-progress.md b/docs/superpowers/plans/2026-05-31-void-v2-plan1-progress.md new file mode 100644 index 0000000..c7c81ff --- /dev/null +++ b/docs/superpowers/plans/2026-05-31-void-v2-plan1-progress.md @@ -0,0 +1,150 @@ +# Void 2.0 Plan 1 — Execution Progress + +**Updated:** 2026-05-31 (session paused at 92% context) +**Plan:** `/project/docs/superpowers/plans/2026-05-31-void-v2-plan1-foundation.md` +**Spec:** `/project/docs/superpowers/specs/2026-05-31-void-v2-design.md` +**Repo:** `/project/src/void-v2/` (git init done, on branch `main`) +**Execution mode:** subagent-driven-development + +## Status by Phase + +| Phase | Plan Tasks | Status | Notes | +|---|---|---|---| +| A — Scaffolding | 1, 4 | **DONE** | Repo init + Node project init. 2 commits. | +| B — Infrastructure | 2, 3 | **DONE** | void2-db (CT 310 @ 192.168.1.12) + void2-app (CT 311 @ 192.168.1.13) running on Z. Postgres 16.14 + pgvector 0.8.2 + pgcrypto 1.3. DB user `void` + database `void` created. Verified reachable from this LXC. `.env` populated. | +| C — Migrations | 5, 6, 8, 10, 12, 14, 16 | **READY** | DB is reachable — can dispatch Task 5 implementer. | +| D — Repos | 7, 9, 11, 13, 15, 16-real | Blocked by B | All entity repos. | +| E — Auth + Server | 17, 18, 19, 20 | Blocked by B | Capability check, owner middleware, Express, /health smoke. | +| F — Deploy + docs | 21, 22 | Blocked by E | systemd, push.sh, completion doc. | + +## Completed Tasks + +### Task 1 — Repo Scaffolding [commit 0ede9fe] +- 5 files: `.gitignore`, `README.md`, `CHANGELOG.md`, `docs/VERSION_HISTORY.md`, `.env.example` +- Spec reviewed ✅, code quality reviewed ✅ + +### Task 4 — Node Project Init [commit 45186f7] +- `package.json` (name=void-server, version=2.0.0-alpha.1, type=module) +- 7 runtime deps + 2 dev deps installed +- `vitest.config.js`, `lib/log.js` (pino logger) +- **NOT YET REVIEWED** — spec + code quality review skipped due to token pressure. Re-do on resume if desired, OR proceed (Task 4 is mechanical setup with no functional code). + +## Deviations from Plan + +1. **Express 5.2.1 installed** (plan said "Express 4"). Express 5 is the current `npm install express` default. **Likely fine** — Express 5 changed middleware error handling (promises auto-catch) and removed deprecated APIs, but our usage (json body, simple middlewares) works on both. Flag if any test fails with Express 5-specific behaviour. +2. **Debian 13** used for LXCs (plan said 12). Only `debian-13-standard_13.1-2` template was on Z. No functional impact. +3. **Storage: `localzfs` on Z** (plan said `donatello-zfs`). Donatello + Leonardo ZFS pools are OFFLINE — leftover from your 2026-05-22 SATA bus incident. **HA migrate is NOT blocked** — `localzfs` is the standard pattern here (CT 104, 105, 106, 108-112 all run on it) and PVE storage replication to Z3 every 15 min is configured. Replication jobs `310-0` and `311-0` added with `*/15` schedule, matching the rest of the fleet. `pct migrate 310 z3` will work like every other CT. Donatello/Leonardo restoration is a separate issue, not Void-blocking. +4. **`su - postgres` instead of `sudo -u postgres`** — Debian 13 minimal doesn't ship sudo. Not a deviation in outcome, just adjusted command form. +5. **DB password stored at `/root/void2-db-pass.txt` on Z** (chmod 600). Also baked into `/project/src/void-v2/.env` on this LXC. + +## Awaiting User — Phase B (Tasks 2 + 3) + +These need PVE host access. Agent inside /project cannot run them. + +### Task 2 — Provision LXCs on PVE host `z` + +Pick two free CT IDs (suggestion: 310 = `void2-db`, 311 = `void2-app`) and two free IPs on 192.168.1.0/24. + +```bash +# On PVE host as root +pct create 310 local:vztmpl/debian-12-standard_12.7-1_amd64.tar.zst \ + --hostname void2-db \ + --cores 2 --memory 4096 --swap 1024 \ + --net0 name=eth0,bridge=vmbr0,ip=192.168.1.X/24,gw=192.168.1.1 \ + --storage donatello-zfs --rootfs donatello-zfs:32 \ + --unprivileged 1 --features nesting=1 --onboot 1 --start 0 +cat >> /etc/pve/lxc/310.conf <<'EOF' +lxc.apparmor.profile: unconfined +EOF +pct start 310 + +pct create 311 local:vztmpl/debian-12-standard_12.7-1_amd64.tar.zst \ + --hostname void2-app \ + --cores 4 --memory 4096 --swap 1024 \ + --net0 name=eth0,bridge=vmbr0,ip=192.168.1.Y/24,gw=192.168.1.1 \ + --storage donatello-zfs --rootfs donatello-zfs:16 \ + --unprivileged 1 --features nesting=1 --onboot 1 --start 0 +cat >> /etc/pve/lxc/311.conf <<'EOF' +lxc.apparmor.profile: unconfined +EOF +pct start 311 +``` + +### Task 3 — Install Postgres 16 + pgvector on void2-db (CT 310) + +```bash +pct enter 310 + +apt update +apt install -y curl ca-certificates gnupg +install -d /usr/share/postgresql-common/pgdg +curl -fsSL https://www.postgresql.org/media/keys/ACCC4CF8.asc \ + -o /usr/share/postgresql-common/pgdg/apt.postgresql.org.asc +echo "deb [signed-by=/usr/share/postgresql-common/pgdg/apt.postgresql.org.asc] \ + https://apt.postgresql.org/pub/repos/apt $(. /etc/os-release; echo $VERSION_CODENAME)-pgdg main" \ + > /etc/apt/sources.list.d/pgdg.list +apt update +apt install -y postgresql-16 postgresql-16-pgvector + +sed -i "s/^#listen_addresses.*/listen_addresses = '*'/" /etc/postgresql/16/main/postgresql.conf + +cat >> /etc/postgresql/16/main/pg_hba.conf <<'EOF' +host void void 192.168.1.0/24 scram-sha-256 +EOF + +systemctl restart postgresql + +DB_PASS=$(openssl rand -base64 24) +echo "Generated DB password (SAVE THIS): $DB_PASS" + +sudo -u postgres psql < -U void -d void -c 'SELECT version();'` succeeds from any LAN host + +Then update `/project/src/void-v2/.env`: +``` +DATABASE_URL=postgres://void:@:5432/void +OWNER_TOKEN= +PORT=3000 +LOG_LEVEL=info +NODE_ENV=development +``` + +## How to Resume + +In the next session, say something like: +> "Resume Plan 1 Void 2.0 execution. Read `/project/docs/superpowers/plans/2026-05-31-void-v2-plan1-progress.md` — Phases A+B done, start dispatching from Task 5." + +Resume agent should: +1. Read this progress file +2. Verify DB still reachable: `PGPASSWORD=$(ssh root@192.168.1.124 'grep DB_PASS /root/void2-db-pass.txt | cut -d= -f2') psql -h 192.168.1.12 -U void -d void -c 'SELECT 1;'` +3. Continue dispatching implementer subagents for Task 5 onward +4. Token-saving advice: skip code-quality review for trivial scaffolding tasks (the spec compliance review is sufficient there); do full two-stage review for repo + auth + server code + +## Next subagent dispatch (when ready) + +**Task 5: Postgres Pool + Migration Runner** — full task text in plan file lines covering DB pool, migration runner with idempotency, test helpers. + +## Quick environment cheatsheet for resume + +- Repo: `/project/src/void-v2/` (on `main`, 2 commits) +- DB: `192.168.1.12:5432`, user `void`, db `void`, password in `.env` and at `/root/void2-db-pass.txt` on Z (192.168.1.124) +- App LXC ready (CT 311 @ 192.168.1.13) but Node not installed there yet (Task 21 handles that) +- `cd /project/src/void-v2 && npm test` should work once tests exist; `.env` will be picked up by `dotenv` +- SSH to Z: `ssh root@192.168.1.124` (key auth works) +- SSH to void2-db/app: `pct exec 310 -- bash` / `pct exec 311 -- bash` from Z (via the SSH chain) diff --git a/docs/superpowers/plans/2026-05-31-void-v2-plan2-api-and-shell.md b/docs/superpowers/plans/2026-05-31-void-v2-plan2-api-and-shell.md new file mode 100644 index 0000000..471571c --- /dev/null +++ b/docs/superpowers/plans/2026-05-31-void-v2-plan2-api-and-shell.md @@ -0,0 +1,536 @@ +# Void 2.0 — Plan 2: Core REST API + Void UI Shell + +**Goal:** Expose all Plan 1 repos as a REST API and ship the Cradle-themed Void UI shell on top. + +**Architecture:** Thin Express routes in `lib/api/routes/` call the existing `lib/db/repos/` (no raw SQL in routes). Shared zod-validate + error middleware. Static SPA in `public/` consumed by the bearer-protected `/api/*`. Agent bearer auth composes with owner. FTS-only search; vector search is Plan 3. Capture endpoints + jobs surface are Plan 3+. + +**Tech stack:** Express 5, zod 4 (already in deps), supertest 7, vanilla ES modules in browser, marked.js (CDN) for markdown render. No new server deps beyond `marked` if we choose to vendor it. + +**Out of scope (deferred):** +- Vector/RRF search (needs embeddings — Plan 3) +- Capture endpoints (`/api/capture/*`) and `/api/jobs` (needs pg-boss — Plan 3) +- MCP server (Plan 5) +- Sacred Valley gridstack widgets (Plan 6) — ship a placeholder card +- E2E Playwright tests (Plan 8 sweep) + +--- + +## File structure + +``` +lib/api/ + index.js # registers routers onto app + errors.js # NotFoundError, ValidationError, asyncWrap, errorMiddleware + validate.js # validate({ body?, params?, query? }) using zod + pagination.js # parsePagination(req) → { limit, offset } + middleware/ + agent_auth.js # bearer → agent actor; composes with ownerOnly + routes/ + spaces.js + projects.js + tasks.js + pages.js + refs.js + resources.js + source_docs.js + conversations.js + messages.js + agents.js + tags.js + links.js + pending_changes.js + audit.js + search.js + +public/ + index.html # SPA shell + style.css # blackflame palette + three-column layout + app.js # bootstrap, router, fetch wrapper (auth header) + router.js # hash router + api.js # typed-ish wrappers over fetch + components/ + sidebar.js + topbar.js + rightrail.js + markdown_editor.js + views/ + space.js + project.js + page.js + reference.js + resource.js + search.js + inbox.js + sacred_valley.js # placeholder + home.js # landing fallback + vendor/ + marked.min.js # vendored, no CDN at runtime + +tests/api/ + helpers.js # createApp + auth headers + reset/migrate + spaces.test.js + projects.test.js + tasks.test.js + pages.test.js + refs.test.js + resources.test.js + source_docs.test.js + conversations.test.js + messages.test.js + agents.test.js + tags.test.js + links.test.js + pending_changes.test.js + audit.test.js + search.test.js + agent_auth.test.js + validate.test.js + errors.test.js +``` + +`server.js` shrinks: build `app`, mount static, mount `/api` router from `lib/api/index.js`. Drop the inline `/api/spaces` smoke route. + +--- + +## Conventions (apply to every route task) + +1. **TDD always.** Write the failing supertest test first. Run it red. Then write the route. Run it green. Commit. +2. **Route file shape:** + ```js + import { Router } from 'express'; + import * as repo from '../../db/repos/.js'; + import { validate } from '../validate.js'; + import { asyncWrap } from '../errors.js'; + import { z } from 'zod'; + export const router = Router(); + ``` +3. **No raw SQL in routes** — every data access is `repo.fn(...)`. +4. **Mutations pass `req.actor`** to the repo. +5. **Errors:** throw `new NotFoundError(...)` / `new ValidationError(...)`. The shared error middleware shapes them as `{error:{code,message,details?}}`. Use `asyncWrap` or rely on Express 5's native promise handling (already default in 5.2). +6. **Pagination:** all `GET` list endpoints accept `?limit=&offset=` via `parsePagination`. Default `limit=50`, max `200`. +7. **Status codes:** `201` for create, `200` for read/update, `204` for delete, `400` for validation errors, `401` for unauthenticated, `403` for capability deny, `404` not found, `409` for conflicts. +8. **Test file shape:** import `tests/api/helpers.js`'s `setup()` which calls `resetDb` + `migrateUp` + returns `{ app, ownerHeaders }`. Each test seeds the minimum it needs (e.g. one space) via repos, then hits the route. +9. **Commit per task** with message `feat(api): routes` or `feat(ui): ` etc. + +--- + +## Task list + +### Phase A — Plumbing + +#### Task 1: Error + validation + pagination plumbing + +**Files:** create `lib/api/errors.js`, `lib/api/validate.js`, `lib/api/pagination.js`, `lib/api/index.js`; create `tests/api/helpers.js`, `tests/api/errors.test.js`, `tests/api/validate.test.js`. + +- `errors.js`: export classes `NotFoundError`, `ValidationError`, `ConflictError`, `ForbiddenError` each with `.code` and `.status`; export `errorMiddleware(err, req, res, next)` that maps known errors to `{error:{code,message,details?}}` with correct status, logs unknowns at 500. +- `validate.js`: `validate({ body, params, query })` returns middleware that runs the relevant zod schemas, on parse failure throws `ValidationError` with zod's `error.issues` as `details`. +- `pagination.js`: `parsePagination(req, { defaultLimit=50, max=200 })` → `{ limit, offset }`, throws `ValidationError` if out of range. +- `lib/api/index.js`: exports `mountApi(app)` that mounts an `/api` router (initially empty) under `ownerOnly`. We'll register each route module here in later tasks. +- Update `server.js` to call `mountApi(app)` and remove the inline `/api/spaces` route. The existing server smoke test must keep passing — it should now route through the new `spaces` router (added in Task 2). Until then, expect that test to break — fix it in Task 2. + +Tests: validate.test exercises happy + zod failure; errors.test exercises the middleware status mapping and JSON shape. + +Commit: `feat(api): error + validate + pagination plumbing`. + +#### Task 2: Spaces routes + +**Files:** create `lib/api/routes/spaces.js`, `tests/api/spaces.test.js`. Register router in `lib/api/index.js`. + +Endpoints: +- `GET /api/spaces` → `repo.list()` +- `POST /api/spaces` body `{slug,name,description?,theme?}` → `repo.create(body, req.actor)`; `201` +- `GET /api/spaces/:id` → `repo.getById`; 404 if missing +- `GET /api/spaces/by-slug/:slug` → `repo.getBySlug`; 404 if missing +- `PATCH /api/spaces/:id` partial body → `repo.update` +- `DELETE /api/spaces/:id` → `repo.del`; `204` + +Tests: list empty → `[]`; create → 201 + record exists in DB; create with bad slug → 400 + zod details; get unknown id → 404; patch updates; delete then get → 404. Re-enable existing `tests/server.test.js` expectations (the `[]` smoke should now serve from this router). + +Commit: `feat(api): spaces routes`. + +#### Task 3: Projects routes + +**Files:** `lib/api/routes/projects.js`, `tests/api/projects.test.js`. + +Endpoints: +- `GET /api/spaces/:space_id/projects?status=` → `repo.listBySpace` +- `POST /api/spaces/:space_id/projects` body `{slug,name,description?,status?,started_at?}` +- `GET /api/projects/:id` +- `PATCH /api/projects/:id` +- `DELETE /api/projects/:id` + +Tests: list filter by status; create rejects unknown space (FK error → map to 400 with code `invalid_space`); patch flips status to `done` (does not auto-set `completed_at` — that's a client/UI concern, mirroring repo behavior). Verify `completed_at` only changes when caller passes it. + +Commit: `feat(api): projects routes`. + +#### Task 4: Tasks routes + +**Files:** `lib/api/routes/tasks.js`, `tests/api/tasks.test.js`. + +Endpoints: +- `GET /api/spaces/:space_id/tasks?status=` → `repo.listBySpace` +- `GET /api/projects/:project_id/tasks` → `repo.listByProject` +- `POST /api/spaces/:space_id/tasks` body `{project_id?,title,body?,priority?,due_at?,position?}` +- `GET /api/tasks/:id` +- `PATCH /api/tasks/:id` +- `DELETE /api/tasks/:id` + +Tests: create sibling task (no project_id); create child task (project_id present); listByProject ordered by `position NULLS LAST, created_at`; patch with `status:'done'` sets `completed_at`. + +Commit: `feat(api): tasks routes`. + +#### Task 5: Pages routes (+ revisions, backlinks) + +**Files:** `lib/api/routes/pages.js`, `tests/api/pages.test.js`. + +Endpoints: +- `GET /api/spaces/:space_id/pages` → `repo.listBySpace` +- `POST /api/spaces/:space_id/pages` body `{slug,title,body_md?,parent_id?}` +- `GET /api/pages/:id` +- `GET /api/spaces/:space_id/pages/by-slug/:slug` +- `PATCH /api/pages/:id` +- `DELETE /api/pages/:id` +- `GET /api/pages/:id/revisions` → `repo.listRevisions` +- `GET /api/pages/:id/backlinks` → `links.listTo('page', id)` joined with the source entity's title for display (route does the join via repos: read each from_type/from_id and resolve title) + +Tests: create with body_md writes a revision; update body_md adds a revision; revisions ordered desc; backlinks returns rows when a `entity_links` row points at the page. + +Commit: `feat(api): pages routes`. + +#### Task 6: Refs routes + +**Files:** `lib/api/routes/refs.js`, `tests/api/refs.test.js`. + +Endpoints: +- `GET /api/refs?space_id=&kind=&limit=&offset=` → `repo.list` +- `POST /api/refs` body matches `repo.create` (all `FIELDS` whitelist) +- `GET /api/refs/:id` +- `PATCH /api/refs/:id` +- `DELETE /api/refs/:id` +- `POST /api/refs/upsert` body must include `space_id+source_kind+external_id` → `repo.upsertByExternal` + +Tests: list with `kind=url` filter; upsert twice with same external_id returns the same row id with updated fields; pagination caps at 200. + +Commit: `feat(api): refs routes`. + +#### Task 7: Resources routes (+ deps) + +**Files:** `lib/api/routes/resources.js`, `tests/api/resources.test.js`. + +Endpoints: +- `GET /api/spaces/:space_id/resources` → `repo.listBySpace` +- `POST /api/spaces/:space_id/resources` body matches resource FIELDS +- `GET /api/resources/:id` +- `PATCH /api/resources/:id` +- `DELETE /api/resources/:id` +- `POST /api/resources/:id/dependencies` body `{depends_on, kind?}` → `repo.addDependency`; 400 on self-dep +- `GET /api/resources/:id/dependencies` → `repo.listDependencies` +- `DELETE /api/resources/:id/dependencies/:dep_id` → `repo.removeDependency` +- `GET /api/resources/:id/source-docs` → `source_docs.listByResource` +- `GET /api/resources/:id/changes` → `audit.listForEntity('resource', id)` — the resource change history is the audit log filtered to that resource + +Tests: dependency create rejects self; cross-space dependency rejected by composite FK → mapped to 400 with `cross_space` code; listing dependencies returns the right rows; changes endpoint returns audit entries (create + each patch). + +Commit: `feat(api): resources routes`. + +#### Task 8: Source docs routes + +**Files:** `lib/api/routes/source_docs.js`, `tests/api/source_docs.test.js`. + +Endpoints: +- `POST /api/resources/:resource_id/source-docs` body matches source_docs FIELDS (minus resource_id, taken from URL) +- `GET /api/source-docs/:id` +- `PATCH /api/source-docs/:id` +- `DELETE /api/source-docs/:id` +- `POST /api/source-docs/:id/resync` — stub for now: returns `202 {queued:true, note:"workers land in Plan 3"}`. Behind a feature flag check `if (process.env.ENABLE_RESYNC === 'true')` → 503 otherwise. Document in the route comment that this hooks into `pg-boss` in Plan 3. + +Tests: create requires resource_id from URL; resync returns 202/503 based on env. + +Commit: `feat(api): source-docs routes`. + +### Phase B — Agents + auth + +#### Task 9: Agent bearer auth middleware + +**Files:** create `lib/api/middleware/agent_auth.js`, `tests/api/agent_auth.test.js`. Modify `lib/api/index.js`. + +`agent_auth.js` exports `agentOrOwner(req, res, next)`: +1. Read `Authorization: Bearer ` (401 if absent). +2. If token equals `OWNER_TOKEN` → `req.actor = { kind:'user', id:null }`; next(). +3. Else `agents.verifyToken(token)`: + - null → 401. + - row → `req.actor = { kind:'agent', id:row.id, capabilities:row.capabilities, scopes:row.scopes }`; next(). + +`lib/api/index.js`: swap `ownerOnly` for `agentOrOwner` on the `/api` router. Owner tests continue to pass (same token path). New agent token tests pass. + +Tests: missing header → 401; wrong token → 401; owner token → 200 + actor.kind='user'; valid agent token → 200 + actor.kind='agent'; revoked agent token → 401. + +Commit: `feat(api): agent bearer auth`. + +#### Task 10: Capability enforcement on mutating routes + +**Files:** modify `lib/auth/capability.js` if needed; add helper `lib/api/cap.js`; add tests `tests/api/capability_routes.test.js`. + +Add `requireWrite(entity_type)` middleware that calls `canAct(req.actor, 'write', entity_type)`: +- `allow` → next(). +- `suggest` → divert: write the operation into `pending_changes` instead of running it, return `202 {pending:true, change_id}`. The handler still needs to know what payload to record. Strategy: middleware attaches a `req.divertToPending(payloadFactory)` helper the route calls just before invoking the repo. If `req.actor.kind === 'agent'` and tier is `suggest`, route does `await pending_changes.create({agent_id, entity_type, entity_id, action, payload, reason})` and returns 202. +- `deny` → 403. + +For Plan 2, apply to: `POST/PATCH/DELETE` on `pages`, `refs`, `resources`, `source_docs`, `projects`, `tasks`, `tags`, `links`. (Read endpoints stay open to any authed agent.) `agents` writes are **owner-only** regardless (a hard `req.actor.kind === 'user'` check, 403 otherwise). + +Tests: agent at `suggest` tier POSTing a page → 202 + pending row exists; agent at `allow` tier POSTing a page → 201 + page row exists; agent at `deny` tier → 403; agent attempting `POST /api/agents` → 403. + +Commit: `feat(api): capability enforcement on writes`. + +#### Task 11: Agents routes + +**Files:** `lib/api/routes/agents.js`, `tests/api/agents.test.js`. + +All endpoints owner-only (see Task 10 rule). Endpoints: +- `GET /api/agents` → `repo.list` +- `POST /api/agents` body matches FIELDS → `repo.create` +- `GET /api/agents/:id` +- `PATCH /api/agents/:id/capabilities` body `{capabilities, scopes}` → `repo.setCapabilities` +- `POST /api/agents/:id/tokens` body `{label?}` → `repo.createToken`; response `{id, token}` (plaintext shown **once** — document in comment) +- `DELETE /api/agent-tokens/:token_id` → `repo.revokeToken` + +Tests: token mint returns plaintext; mint then auth-as-agent works; revoke then auth fails. + +Commit: `feat(api): agents routes + token mgmt`. + +### Phase C — Cross-cutting + +#### Task 12: Conversations + messages routes + +**Files:** `lib/api/routes/conversations.js`, `lib/api/routes/messages.js`, `tests/api/conversations.test.js`, `tests/api/messages.test.js`. + +Conversations: +- `GET /api/conversations?limit=&offset=` → `repo.list` +- `POST /api/conversations` → `repo.create` +- `GET /api/conversations/:id` +- `PATCH /api/conversations/:id/status` body `{status}` → `repo.setStatus` +- `PATCH /api/conversations/:id/summary` body `{summary}` → `repo.setSummary` + +Messages: +- `GET /api/conversations/:conversation_id/messages?limit=` → `messages.listByConversation` +- `POST /api/conversations/:conversation_id/messages` body `{role,body,agent_id?,metadata?}` → `messages.append` + +Tests: append message, list returns it ordered; setSummary flips status to `summarized`. + +Commit: `feat(api): conversations + messages routes`. + +#### Task 13: Tags routes + +**Files:** `lib/api/routes/tags.js`, `tests/api/tags.test.js`. + +Endpoints: +- `GET /api/tags` → `tags.list` +- `POST /api/tags` body `{name, description?, color?}` → `tags.upsert` +- `POST /api//:entity_id/tags` body `{tag_id}` → `tags.attach`; 204 +- `DELETE /api//:entity_id/tags/:tag_id` → `tags.detach`; 204 +- `GET /api//:entity_id/tags` → `tags.listForEntity` + +Allow `entity_type` values: `space|project|task|page|ref|resource|source_doc|conversation`. Validate via zod enum. + +Tests: upsert idempotent; attach idempotent on conflict; listForEntity returns tags sorted by name. + +Commit: `feat(api): tags routes`. + +#### Task 14: Entity links routes + +**Files:** `lib/api/routes/links.js`, `tests/api/links.test.js`. + +Endpoints: +- `POST /api/links` body `{from_type,from_id,to_type,to_id,relation?}` → `links.create` +- `GET /api/links/from/:type/:id` → `links.listFrom` +- `GET /api/links/to/:type/:id` → `links.listTo` +- `DELETE /api/links/:id` → `links.remove` + +Tests: create twice with same tuple returns same row (ON CONFLICT path); list from/to. + +Commit: `feat(api): links routes`. + +#### Task 15: Pending-changes + audit routes + +**Files:** `lib/api/routes/pending_changes.js`, `lib/api/routes/audit.js`, `tests/api/pending_changes.test.js`, `tests/api/audit.test.js`. + +Pending changes (owner-only): +- `GET /api/pending-changes?limit=` → `pending_changes.listPending` +- `POST /api/pending-changes/:id/approve` → load row; dispatch by `entity_type+action` through the same repo (`pages.create`, `refs.update`, etc.) using `req.actor` (the approving user); mark row `approved`. Single dispatch helper `applyPendingChange(row, actor)` lives in `lib/api/routes/pending_changes.js` and uses a small switch table mapping `entity_type` → repo module. +- `POST /api/pending-changes/:id/reject` → mark `rejected`. + +Audit (owner-only): +- `GET /api/audit/entity/:type/:id?limit=` → `audit.listForEntity` +- `GET /api/audit/actor?actor_kind=&actor_id=&limit=` → `audit.listByActor` + +Tests: agent at `suggest` POSTs a page (from Task 10) → owner approves → page now exists, pending row is `approved`, audit log shows the create with `actor_kind='user'` (the approver). Reject test marks row `rejected`, no entity created. + +Commit: `feat(api): pending-changes + audit routes`. + +### Phase D — Search + +#### Task 16: FTS search endpoint + +**Files:** create `lib/db/repos/search.js`, `lib/api/routes/search.js`, `tests/repos/search.test.js`, `tests/api/search.test.js`. + +`search.fts({q, space_id?, kinds?, limit, offset})` runs `tsvector @@ plainto_tsquery` against four sources unioned with a `kind` discriminator: +- `pages` — fts column already exists in migration 002 (`fts_tsv`); fall back to `to_tsvector('english', title || ' ' || coalesce(body_md,''))` if not. +- `refs` — uses `refs.fts_tsv` from migration 002. +- `source_docs` — `to_tsvector('english', name || ' ' || coalesce(body_text,''))`. +- `messages` — uses `messages.fts_tsv` from migration 004. + +Each branch returns `{kind, id, space_id, title_or_snippet, rank}`. Final SELECT orders by `ts_rank` desc and applies `limit/offset`. `kinds` filter restricts which branches run. + +Endpoint: +- `GET /api/search?q=&space_id=&kinds=page,ref&limit=&offset=` → results grouped client-side; server returns flat array with `kind` discriminator. + +Tests (repo): seed 1 page + 1 ref + 1 source_doc + 1 message containing the word "blackflame", search for "blackflame" returns 4 hits, kinds filter narrows correctly. Tests (api): 401 without auth, 200 with results. + +**Vector search and RRF are explicitly deferred to Plan 3** — add a TODO comment in `search.js` linking to the spec section. + +Commit: `feat(api): unified FTS search`. + +### Phase E — Void UI shell + +#### Task 17: Static serving + shell HTML/CSS + SPA bootstrap + +**Files:** create `public/index.html`, `public/style.css`, `public/app.js`, `public/router.js`, `public/api.js`. Modify `server.js` to `app.use(express.static('public'))` BEFORE the `/api` mount and ABOVE the 404 catch-all. + +`index.html`: three-column flex layout: `