# Void 2.0 — Plan 2: Core REST API + Void UI Shell **Goal:** Expose all Plan 1 repos as a REST API and ship the Cradle-themed Void UI shell on top. **Architecture:** Thin Express routes in `lib/api/routes/` call the existing `lib/db/repos/` (no raw SQL in routes). Shared zod-validate + error middleware. Static SPA in `public/` consumed by the bearer-protected `/api/*`. Agent bearer auth composes with owner. FTS-only search; vector search is Plan 3. Capture endpoints + jobs surface are Plan 3+. **Tech stack:** Express 5, zod 4 (already in deps), supertest 7, vanilla ES modules in browser, marked.js (CDN) for markdown render. No new server deps beyond `marked` if we choose to vendor it. **Out of scope (deferred):** - Vector/RRF search (needs embeddings — Plan 3) - Capture endpoints (`/api/capture/*`) and `/api/jobs` (needs pg-boss — Plan 3) - MCP server (Plan 5) - Sacred Valley gridstack widgets (Plan 6) — ship a placeholder card - E2E Playwright tests (Plan 8 sweep) --- ## File structure ``` lib/api/ index.js # registers routers onto app errors.js # NotFoundError, ValidationError, asyncWrap, errorMiddleware validate.js # validate({ body?, params?, query? }) using zod pagination.js # parsePagination(req) → { limit, offset } middleware/ agent_auth.js # bearer → agent actor; composes with ownerOnly routes/ spaces.js projects.js tasks.js pages.js refs.js resources.js source_docs.js conversations.js messages.js agents.js tags.js links.js pending_changes.js audit.js search.js public/ index.html # SPA shell style.css # blackflame palette + three-column layout app.js # bootstrap, router, fetch wrapper (auth header) router.js # hash router api.js # typed-ish wrappers over fetch components/ sidebar.js topbar.js rightrail.js markdown_editor.js views/ space.js project.js page.js reference.js resource.js search.js inbox.js sacred_valley.js # placeholder home.js # landing fallback vendor/ marked.min.js # vendored, no CDN at runtime tests/api/ helpers.js # createApp + auth headers + reset/migrate spaces.test.js projects.test.js tasks.test.js pages.test.js refs.test.js resources.test.js source_docs.test.js conversations.test.js messages.test.js agents.test.js tags.test.js links.test.js pending_changes.test.js audit.test.js search.test.js agent_auth.test.js validate.test.js errors.test.js ``` `server.js` shrinks: build `app`, mount static, mount `/api` router from `lib/api/index.js`. Drop the inline `/api/spaces` smoke route. --- ## Conventions (apply to every route task) 1. **TDD always.** Write the failing supertest test first. Run it red. Then write the route. Run it green. Commit. 2. **Route file shape:** ```js import { Router } from 'express'; import * as repo from '../../db/repos/.js'; import { validate } from '../validate.js'; import { asyncWrap } from '../errors.js'; import { z } from 'zod'; export const router = Router(); ``` 3. **No raw SQL in routes** — every data access is `repo.fn(...)`. 4. **Mutations pass `req.actor`** to the repo. 5. **Errors:** throw `new NotFoundError(...)` / `new ValidationError(...)`. The shared error middleware shapes them as `{error:{code,message,details?}}`. Use `asyncWrap` or rely on Express 5's native promise handling (already default in 5.2). 6. **Pagination:** all `GET` list endpoints accept `?limit=&offset=` via `parsePagination`. Default `limit=50`, max `200`. 7. **Status codes:** `201` for create, `200` for read/update, `204` for delete, `400` for validation errors, `401` for unauthenticated, `403` for capability deny, `404` not found, `409` for conflicts. 8. **Test file shape:** import `tests/api/helpers.js`'s `setup()` which calls `resetDb` + `migrateUp` + returns `{ app, ownerHeaders }`. Each test seeds the minimum it needs (e.g. one space) via repos, then hits the route. 9. **Commit per task** with message `feat(api): routes` or `feat(ui): ` etc. --- ## Task list ### Phase A — Plumbing #### Task 1: Error + validation + pagination plumbing **Files:** create `lib/api/errors.js`, `lib/api/validate.js`, `lib/api/pagination.js`, `lib/api/index.js`; create `tests/api/helpers.js`, `tests/api/errors.test.js`, `tests/api/validate.test.js`. - `errors.js`: export classes `NotFoundError`, `ValidationError`, `ConflictError`, `ForbiddenError` each with `.code` and `.status`; export `errorMiddleware(err, req, res, next)` that maps known errors to `{error:{code,message,details?}}` with correct status, logs unknowns at 500. - `validate.js`: `validate({ body, params, query })` returns middleware that runs the relevant zod schemas, on parse failure throws `ValidationError` with zod's `error.issues` as `details`. - `pagination.js`: `parsePagination(req, { defaultLimit=50, max=200 })` → `{ limit, offset }`, throws `ValidationError` if out of range. - `lib/api/index.js`: exports `mountApi(app)` that mounts an `/api` router (initially empty) under `ownerOnly`. We'll register each route module here in later tasks. - Update `server.js` to call `mountApi(app)` and remove the inline `/api/spaces` route. The existing server smoke test must keep passing — it should now route through the new `spaces` router (added in Task 2). Until then, expect that test to break — fix it in Task 2. Tests: validate.test exercises happy + zod failure; errors.test exercises the middleware status mapping and JSON shape. Commit: `feat(api): error + validate + pagination plumbing`. #### Task 2: Spaces routes **Files:** create `lib/api/routes/spaces.js`, `tests/api/spaces.test.js`. Register router in `lib/api/index.js`. Endpoints: - `GET /api/spaces` → `repo.list()` - `POST /api/spaces` body `{slug,name,description?,theme?}` → `repo.create(body, req.actor)`; `201` - `GET /api/spaces/:id` → `repo.getById`; 404 if missing - `GET /api/spaces/by-slug/:slug` → `repo.getBySlug`; 404 if missing - `PATCH /api/spaces/:id` partial body → `repo.update` - `DELETE /api/spaces/:id` → `repo.del`; `204` Tests: list empty → `[]`; create → 201 + record exists in DB; create with bad slug → 400 + zod details; get unknown id → 404; patch updates; delete then get → 404. Re-enable existing `tests/server.test.js` expectations (the `[]` smoke should now serve from this router). Commit: `feat(api): spaces routes`. #### Task 3: Projects routes **Files:** `lib/api/routes/projects.js`, `tests/api/projects.test.js`. Endpoints: - `GET /api/spaces/:space_id/projects?status=` → `repo.listBySpace` - `POST /api/spaces/:space_id/projects` body `{slug,name,description?,status?,started_at?}` - `GET /api/projects/:id` - `PATCH /api/projects/:id` - `DELETE /api/projects/:id` Tests: list filter by status; create rejects unknown space (FK error → map to 400 with code `invalid_space`); patch flips status to `done` (does not auto-set `completed_at` — that's a client/UI concern, mirroring repo behavior). Verify `completed_at` only changes when caller passes it. Commit: `feat(api): projects routes`. #### Task 4: Tasks routes **Files:** `lib/api/routes/tasks.js`, `tests/api/tasks.test.js`. Endpoints: - `GET /api/spaces/:space_id/tasks?status=` → `repo.listBySpace` - `GET /api/projects/:project_id/tasks` → `repo.listByProject` - `POST /api/spaces/:space_id/tasks` body `{project_id?,title,body?,priority?,due_at?,position?}` - `GET /api/tasks/:id` - `PATCH /api/tasks/:id` - `DELETE /api/tasks/:id` Tests: create sibling task (no project_id); create child task (project_id present); listByProject ordered by `position NULLS LAST, created_at`; patch with `status:'done'` sets `completed_at`. Commit: `feat(api): tasks routes`. #### Task 5: Pages routes (+ revisions, backlinks) **Files:** `lib/api/routes/pages.js`, `tests/api/pages.test.js`. Endpoints: - `GET /api/spaces/:space_id/pages` → `repo.listBySpace` - `POST /api/spaces/:space_id/pages` body `{slug,title,body_md?,parent_id?}` - `GET /api/pages/:id` - `GET /api/spaces/:space_id/pages/by-slug/:slug` - `PATCH /api/pages/:id` - `DELETE /api/pages/:id` - `GET /api/pages/:id/revisions` → `repo.listRevisions` - `GET /api/pages/:id/backlinks` → `links.listTo('page', id)` joined with the source entity's title for display (route does the join via repos: read each from_type/from_id and resolve title) Tests: create with body_md writes a revision; update body_md adds a revision; revisions ordered desc; backlinks returns rows when a `entity_links` row points at the page. Commit: `feat(api): pages routes`. #### Task 6: Refs routes **Files:** `lib/api/routes/refs.js`, `tests/api/refs.test.js`. Endpoints: - `GET /api/refs?space_id=&kind=&limit=&offset=` → `repo.list` - `POST /api/refs` body matches `repo.create` (all `FIELDS` whitelist) - `GET /api/refs/:id` - `PATCH /api/refs/:id` - `DELETE /api/refs/:id` - `POST /api/refs/upsert` body must include `space_id+source_kind+external_id` → `repo.upsertByExternal` Tests: list with `kind=url` filter; upsert twice with same external_id returns the same row id with updated fields; pagination caps at 200. Commit: `feat(api): refs routes`. #### Task 7: Resources routes (+ deps) **Files:** `lib/api/routes/resources.js`, `tests/api/resources.test.js`. Endpoints: - `GET /api/spaces/:space_id/resources` → `repo.listBySpace` - `POST /api/spaces/:space_id/resources` body matches resource FIELDS - `GET /api/resources/:id` - `PATCH /api/resources/:id` - `DELETE /api/resources/:id` - `POST /api/resources/:id/dependencies` body `{depends_on, kind?}` → `repo.addDependency`; 400 on self-dep - `GET /api/resources/:id/dependencies` → `repo.listDependencies` - `DELETE /api/resources/:id/dependencies/:dep_id` → `repo.removeDependency` - `GET /api/resources/:id/source-docs` → `source_docs.listByResource` - `GET /api/resources/:id/changes` → `audit.listForEntity('resource', id)` — the resource change history is the audit log filtered to that resource Tests: dependency create rejects self; cross-space dependency rejected by composite FK → mapped to 400 with `cross_space` code; listing dependencies returns the right rows; changes endpoint returns audit entries (create + each patch). Commit: `feat(api): resources routes`. #### Task 8: Source docs routes **Files:** `lib/api/routes/source_docs.js`, `tests/api/source_docs.test.js`. Endpoints: - `POST /api/resources/:resource_id/source-docs` body matches source_docs FIELDS (minus resource_id, taken from URL) - `GET /api/source-docs/:id` - `PATCH /api/source-docs/:id` - `DELETE /api/source-docs/:id` - `POST /api/source-docs/:id/resync` — stub for now: returns `202 {queued:true, note:"workers land in Plan 3"}`. Behind a feature flag check `if (process.env.ENABLE_RESYNC === 'true')` → 503 otherwise. Document in the route comment that this hooks into `pg-boss` in Plan 3. Tests: create requires resource_id from URL; resync returns 202/503 based on env. Commit: `feat(api): source-docs routes`. ### Phase B — Agents + auth #### Task 9: Agent bearer auth middleware **Files:** create `lib/api/middleware/agent_auth.js`, `tests/api/agent_auth.test.js`. Modify `lib/api/index.js`. `agent_auth.js` exports `agentOrOwner(req, res, next)`: 1. Read `Authorization: Bearer ` (401 if absent). 2. If token equals `OWNER_TOKEN` → `req.actor = { kind:'user', id:null }`; next(). 3. Else `agents.verifyToken(token)`: - null → 401. - row → `req.actor = { kind:'agent', id:row.id, capabilities:row.capabilities, scopes:row.scopes }`; next(). `lib/api/index.js`: swap `ownerOnly` for `agentOrOwner` on the `/api` router. Owner tests continue to pass (same token path). New agent token tests pass. Tests: missing header → 401; wrong token → 401; owner token → 200 + actor.kind='user'; valid agent token → 200 + actor.kind='agent'; revoked agent token → 401. Commit: `feat(api): agent bearer auth`. #### Task 10: Capability enforcement on mutating routes **Files:** modify `lib/auth/capability.js` if needed; add helper `lib/api/cap.js`; add tests `tests/api/capability_routes.test.js`. Add `requireWrite(entity_type)` middleware that calls `canAct(req.actor, 'write', entity_type)`: - `allow` → next(). - `suggest` → divert: write the operation into `pending_changes` instead of running it, return `202 {pending:true, change_id}`. The handler still needs to know what payload to record. Strategy: middleware attaches a `req.divertToPending(payloadFactory)` helper the route calls just before invoking the repo. If `req.actor.kind === 'agent'` and tier is `suggest`, route does `await pending_changes.create({agent_id, entity_type, entity_id, action, payload, reason})` and returns 202. - `deny` → 403. For Plan 2, apply to: `POST/PATCH/DELETE` on `pages`, `refs`, `resources`, `source_docs`, `projects`, `tasks`, `tags`, `links`. (Read endpoints stay open to any authed agent.) `agents` writes are **owner-only** regardless (a hard `req.actor.kind === 'user'` check, 403 otherwise). Tests: agent at `suggest` tier POSTing a page → 202 + pending row exists; agent at `allow` tier POSTing a page → 201 + page row exists; agent at `deny` tier → 403; agent attempting `POST /api/agents` → 403. Commit: `feat(api): capability enforcement on writes`. #### Task 11: Agents routes **Files:** `lib/api/routes/agents.js`, `tests/api/agents.test.js`. All endpoints owner-only (see Task 10 rule). Endpoints: - `GET /api/agents` → `repo.list` - `POST /api/agents` body matches FIELDS → `repo.create` - `GET /api/agents/:id` - `PATCH /api/agents/:id/capabilities` body `{capabilities, scopes}` → `repo.setCapabilities` - `POST /api/agents/:id/tokens` body `{label?}` → `repo.createToken`; response `{id, token}` (plaintext shown **once** — document in comment) - `DELETE /api/agent-tokens/:token_id` → `repo.revokeToken` Tests: token mint returns plaintext; mint then auth-as-agent works; revoke then auth fails. Commit: `feat(api): agents routes + token mgmt`. ### Phase C — Cross-cutting #### Task 12: Conversations + messages routes **Files:** `lib/api/routes/conversations.js`, `lib/api/routes/messages.js`, `tests/api/conversations.test.js`, `tests/api/messages.test.js`. Conversations: - `GET /api/conversations?limit=&offset=` → `repo.list` - `POST /api/conversations` → `repo.create` - `GET /api/conversations/:id` - `PATCH /api/conversations/:id/status` body `{status}` → `repo.setStatus` - `PATCH /api/conversations/:id/summary` body `{summary}` → `repo.setSummary` Messages: - `GET /api/conversations/:conversation_id/messages?limit=` → `messages.listByConversation` - `POST /api/conversations/:conversation_id/messages` body `{role,body,agent_id?,metadata?}` → `messages.append` Tests: append message, list returns it ordered; setSummary flips status to `summarized`. Commit: `feat(api): conversations + messages routes`. #### Task 13: Tags routes **Files:** `lib/api/routes/tags.js`, `tests/api/tags.test.js`. Endpoints: - `GET /api/tags` → `tags.list` - `POST /api/tags` body `{name, description?, color?}` → `tags.upsert` - `POST /api//:entity_id/tags` body `{tag_id}` → `tags.attach`; 204 - `DELETE /api//:entity_id/tags/:tag_id` → `tags.detach`; 204 - `GET /api//:entity_id/tags` → `tags.listForEntity` Allow `entity_type` values: `space|project|task|page|ref|resource|source_doc|conversation`. Validate via zod enum. Tests: upsert idempotent; attach idempotent on conflict; listForEntity returns tags sorted by name. Commit: `feat(api): tags routes`. #### Task 14: Entity links routes **Files:** `lib/api/routes/links.js`, `tests/api/links.test.js`. Endpoints: - `POST /api/links` body `{from_type,from_id,to_type,to_id,relation?}` → `links.create` - `GET /api/links/from/:type/:id` → `links.listFrom` - `GET /api/links/to/:type/:id` → `links.listTo` - `DELETE /api/links/:id` → `links.remove` Tests: create twice with same tuple returns same row (ON CONFLICT path); list from/to. Commit: `feat(api): links routes`. #### Task 15: Pending-changes + audit routes **Files:** `lib/api/routes/pending_changes.js`, `lib/api/routes/audit.js`, `tests/api/pending_changes.test.js`, `tests/api/audit.test.js`. Pending changes (owner-only): - `GET /api/pending-changes?limit=` → `pending_changes.listPending` - `POST /api/pending-changes/:id/approve` → load row; dispatch by `entity_type+action` through the same repo (`pages.create`, `refs.update`, etc.) using `req.actor` (the approving user); mark row `approved`. Single dispatch helper `applyPendingChange(row, actor)` lives in `lib/api/routes/pending_changes.js` and uses a small switch table mapping `entity_type` → repo module. - `POST /api/pending-changes/:id/reject` → mark `rejected`. Audit (owner-only): - `GET /api/audit/entity/:type/:id?limit=` → `audit.listForEntity` - `GET /api/audit/actor?actor_kind=&actor_id=&limit=` → `audit.listByActor` Tests: agent at `suggest` POSTs a page (from Task 10) → owner approves → page now exists, pending row is `approved`, audit log shows the create with `actor_kind='user'` (the approver). Reject test marks row `rejected`, no entity created. Commit: `feat(api): pending-changes + audit routes`. ### Phase D — Search #### Task 16: FTS search endpoint **Files:** create `lib/db/repos/search.js`, `lib/api/routes/search.js`, `tests/repos/search.test.js`, `tests/api/search.test.js`. `search.fts({q, space_id?, kinds?, limit, offset})` runs `tsvector @@ plainto_tsquery` against four sources unioned with a `kind` discriminator: - `pages` — fts column already exists in migration 002 (`fts_tsv`); fall back to `to_tsvector('english', title || ' ' || coalesce(body_md,''))` if not. - `refs` — uses `refs.fts_tsv` from migration 002. - `source_docs` — `to_tsvector('english', name || ' ' || coalesce(body_text,''))`. - `messages` — uses `messages.fts_tsv` from migration 004. Each branch returns `{kind, id, space_id, title_or_snippet, rank}`. Final SELECT orders by `ts_rank` desc and applies `limit/offset`. `kinds` filter restricts which branches run. Endpoint: - `GET /api/search?q=&space_id=&kinds=page,ref&limit=&offset=` → results grouped client-side; server returns flat array with `kind` discriminator. Tests (repo): seed 1 page + 1 ref + 1 source_doc + 1 message containing the word "blackflame", search for "blackflame" returns 4 hits, kinds filter narrows correctly. Tests (api): 401 without auth, 200 with results. **Vector search and RRF are explicitly deferred to Plan 3** — add a TODO comment in `search.js` linking to the spec section. Commit: `feat(api): unified FTS search`. ### Phase E — Void UI shell #### Task 17: Static serving + shell HTML/CSS + SPA bootstrap **Files:** create `public/index.html`, `public/style.css`, `public/app.js`, `public/router.js`, `public/api.js`. Modify `server.js` to `app.use(express.static('public'))` BEFORE the `/api` mount and ABOVE the 404 catch-all. `index.html`: three-column flex layout: `