Hynes/Void-Homelab
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
59aba14ef7696d9ebda9d77f86d048751669dad0
Void-Homelab/docs/superpowers/plans/2026-06-04-void-migrate.md
root bbb90c12c6 docs: void-migrate implementation plan 
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-04 22:17:32 +10:00

26 KiB
Raw Blame History

void-migrate Implementation Plan (Plan 8a)

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: A re-runnable Node CLI that imports Void 1 (SQLite), BookStack, Karakeep, and plans .md into Void 2 — idempotently, one Space per source.

Architecture: A migrate/ CLI reusing Void 2's repos. Idempotency via a migration_map table (refs use upsertByExternal). Importers are plain async functions (testable with fixtures); the CLI wires them and supports --dry-run + verify.

Tech Stack: Node 22 ESM (node:sqlite built-in), Postgres, vitest (serial).

Spec: docs/superpowers/specs/2026-06-04-void-migrate-design.md

Task 1: migration_map table + repo

Files: Create lib/db/migrations/018_migration_map.sql, lib/db/repos/migration_map.js; Test tests/db/migration_map.test.js

  • Step 1: Migration
-- 018_migration_map.sql — idempotency ledger for void-migrate. Maps a source row
-- to the Void 2 entity it created, so re-running an import never duplicates.
CREATE TABLE migration_map (
  id          uuid PRIMARY KEY DEFAULT gen_random_uuid(),
  source      text NOT NULL,                 -- 'void1' | 'bookstack' | 'karakeep' | 'plans'
  source_id   text NOT NULL,                 -- e.g. 'wiki_pages:42'
  entity_type text NOT NULL,                 -- 'page' | 'project' | 'task' | 'conversation' | 'message'
  entity_id   uuid NOT NULL,
  created_at  timestamptz NOT NULL DEFAULT now(),
  UNIQUE (source, source_id, entity_type)
);
  • Step 2: Failing test tests/db/migration_map.test.js
import { describe, it, expect, beforeAll } from 'vitest';
import { resetDb } from '../helpers/db.js';
import { migrateUp } from '../../lib/db/migrate.js';
import * as map from '../../lib/db/repos/migration_map.js';
import { randomUUID } from 'crypto';

beforeAll(async () => { await resetDb(); await migrateUp(); });

describe('migration_map', () => {
  it('seen() is null until record(), then returns the entity id', async () => {
    const eid = randomUUID();
    expect(await map.seen('void1', 'wiki_pages:1', 'page')).toBeNull();
    await map.record('void1', 'wiki_pages:1', 'page', eid);
    expect(await map.seen('void1', 'wiki_pages:1', 'page')).toBe(eid);
  });
  it('record() is idempotent on the unique key', async () => {
    const eid = randomUUID();
    await map.record('plans', 'a.md', 'page', eid);
    await map.record('plans', 'a.md', 'page', randomUUID()); // ignored
    expect(await map.seen('plans', 'a.md', 'page')).toBe(eid);
  });
});
  • Step 3: Run → FAIL.
  • Step 4: Implement lib/db/repos/migration_map.js
import { pool } from '../pool.js';

export async function seen(source, source_id, entity_type) {
  const { rows: [r] } = await pool.query(
    `SELECT entity_id FROM migration_map WHERE source=$1 AND source_id=$2 AND entity_type=$3`,
    [source, source_id, entity_type]);
  return r ? r.entity_id : null;
}
export async function record(source, source_id, entity_type, entity_id) {
  await pool.query(
    `INSERT INTO migration_map(source, source_id, entity_type, entity_id) VALUES($1,$2,$3,$4)
     ON CONFLICT (source, source_id, entity_type) DO NOTHING`,
    [source, source_id, entity_type, entity_id]);
}
  • Step 5: Run → PASS. Commit feat(migrate): migration_map idempotency ledger

Task 2: ensureSpace helper

Files: Create migrate/spaces.js; Test tests/migrate/spaces.test.js

  • Step 1: Failing test
import { describe, it, expect, beforeAll } from 'vitest';
import { resetDb } from '../helpers/db.js';
import { migrateUp } from '../../lib/db/migrate.js';
import { ensureSpace } from '../../migrate/spaces.js';

beforeAll(async () => { await resetDb(); await migrateUp(); });

describe('ensureSpace', () => {
  it('creates a space once and reuses it by slug', async () => {
    const a = await ensureSpace('void1', 'Void 1');
    const b = await ensureSpace('void1', 'Void 1');
    expect(a).toBe(b);
  });
});
  • Step 2: Run → FAIL.
  • Step 3: Implement migrate/spaces.js
import { pool } from '../lib/db/pool.js';
import * as spaces from '../lib/db/repos/spaces.js';

const SYS = { kind: 'system', id: null };

// Returns the id of the space with `slug`, creating it if absent. Idempotent.
export async function ensureSpace(slug, name) {
  const { rows: [r] } = await pool.query(`SELECT id FROM spaces WHERE slug=$1`, [slug]);
  if (r) return r.id;
  const created = await spaces.create({ slug, name }, SYS);
  return created.id;
}
  • Step 4: Run → PASS. Commit feat(migrate): ensureSpace helper

Task 3: plans importer (simplest — establishes the pattern)

Files: Create migrate/sources/plans.js; Test tests/migrate/plans.test.js + tests/fixtures/plans/*

  • Step 1: Fixtures — create tests/fixtures/plans/alpha.md (# Alpha Plan\nbody one) and tests/fixtures/plans/beta.md (# Beta\nbody two).

  • Step 2: Failing test tests/migrate/plans.test.js

import { describe, it, expect, beforeAll } from 'vitest';
import { fileURLToPath } from 'url';
import { pool } from '../../lib/db/pool.js';
import { resetDb } from '../helpers/db.js';
import { migrateUp } from '../../lib/db/migrate.js';
import { ensureSpace } from '../../migrate/spaces.js';
import { importPlans } from '../../migrate/sources/plans.js';

const DIR = fileURLToPath(new URL('../fixtures/plans', import.meta.url));
let spaceId;
beforeAll(async () => { await resetDb(); await migrateUp(); spaceId = await ensureSpace('plans', 'Plans'); });

describe('plans importer', () => {
  it('imports each .md as a page, idempotently', async () => {
    const r1 = await importPlans({ dir: DIR, spaceId });
    expect(r1.created).toBe(2);
    const { rows } = await pool.query(`SELECT title FROM pages WHERE space_id=$1 ORDER BY title`, [spaceId]);
    expect(rows.map(r => r.title)).toEqual(['Alpha Plan', 'Beta']);
    const r2 = await importPlans({ dir: DIR, spaceId });   // re-run
    expect(r2.created).toBe(0);
  });
  it('dry-run writes nothing', async () => {
    await resetDb(); await migrateUp(); const s = await ensureSpace('plans', 'Plans');
    const r = await importPlans({ dir: DIR, spaceId: s, dryRun: true });
    expect(r.created).toBe(2);
    const { rows } = await pool.query(`SELECT count(*)::int n FROM pages WHERE space_id=$1`, [s]);
    expect(rows[0].n).toBe(0);
  });
});
  • Step 3: Run → FAIL.
  • Step 4: Implement migrate/sources/plans.js
import { readdirSync, readFileSync } from 'node:fs';
import { join } from 'node:path';
import * as pages from '../../lib/db/repos/pages.js';
import * as map from '../../lib/db/repos/migration_map.js';

const SYS = { kind: 'system', id: null };
const slugify = (s) => s.toLowerCase().replace(/[^a-z0-9]+/g, '-').replace(/(^-|-$)/g, '').slice(0, 60) || 'page';
function titleOf(body, file) {
  const m = body.match(/^#\s+(.+)$/m);
  return m ? m[1].trim() : file.replace(/\.md$/, '');
}

export async function importPlans({ dir, spaceId, dryRun = false }) {
  let created = 0;
  for (const file of readdirSync(dir).filter(f => f.endsWith('.md')).sort()) {
    if (await map.seen('plans', file, 'page')) continue;
    const body = readFileSync(join(dir, file), 'utf8');
    const title = titleOf(body, file);
    created++;
    if (dryRun) continue;
    const page = await pages.create({ space_id: spaceId, slug: slugify(title), title, body_md: body }, SYS);
    await map.record('plans', file, 'page', page.id);
  }
  return { created };
}
  • Step 5: Run → PASS. Commit feat(migrate): plans importer

Task 4: Void 1 SQLite importer

Files: Create migrate/sources/void1.js; Test tests/migrate/void1.test.js

Void 1 columns (verified): wiki_pages(id,slug,title,content,tags,parent_id,created_at,updated_at), projects(id,name,description,status,...), project_tasks(id,project_id,text,done,position,...), project_journal(id,project_id,content,created_at), conversations(id,project_id,title,created_at), messages(id,conversation_id,role,content,created_at).

  • Step 1: Failing test tests/migrate/void1.test.js (builds a fixture sqlite with node:sqlite)
import { describe, it, expect, beforeAll } from 'vitest';
import { DatabaseSync } from 'node:sqlite';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { randomUUID } from 'node:crypto';
import { pool } from '../../lib/db/pool.js';
import { resetDb } from '../helpers/db.js';
import { migrateUp } from '../../lib/db/migrate.js';
import { ensureSpace } from '../../migrate/spaces.js';
import { importVoid1 } from '../../migrate/sources/void1.js';

let dbPath, spaceId;
beforeAll(async () => {
  await resetDb(); await migrateUp(); spaceId = await ensureSpace('void1', 'Void 1');
  dbPath = join(tmpdir(), `void1-${randomUUID()}.db`);
  const db = new DatabaseSync(dbPath);
  db.exec(`CREATE TABLE wiki_pages(id TEXT,slug TEXT,title TEXT,content TEXT,tags TEXT,parent_id TEXT,created_at INT,updated_at INT);
           CREATE TABLE projects(id TEXT,name TEXT,description TEXT,status TEXT);
           CREATE TABLE project_tasks(id TEXT,project_id TEXT,text TEXT,done INT,position INT);
           CREATE TABLE project_journal(id TEXT,project_id TEXT,content TEXT,created_at INT);
           CREATE TABLE conversations(id TEXT,project_id TEXT,title TEXT,created_at INT);
           CREATE TABLE messages(id TEXT,conversation_id TEXT,role TEXT,content TEXT,created_at INT);`);
  db.prepare(`INSERT INTO wiki_pages VALUES('w1','home','Home','welcome','',NULL,0,0)`).run();
  db.prepare(`INSERT INTO projects VALUES('p1','Homelab','my lab','active')`).run();
  db.prepare(`INSERT INTO project_tasks VALUES('t1','p1','do a thing',0,1)`).run();
  db.prepare(`INSERT INTO conversations VALUES('c1','p1','Chat 1',0)`).run();
  db.prepare(`INSERT INTO messages VALUES('m1','c1','user','hi',0)`).run();
  db.prepare(`INSERT INTO messages VALUES('m2','c1','assistant','hello',1)`).run();
  db.close();
});

describe('void1 importer', () => {
  it('maps wiki/projects/tasks/conversations/messages, idempotently', async () => {
    const r1 = await importVoid1({ sqlitePath: dbPath, spaceId });
    expect(r1.pages).toBeGreaterThanOrEqual(1);
    expect(r1.projects).toBe(1);
    expect(r1.tasks).toBe(1);
    expect(r1.conversations).toBe(1);
    expect(r1.messages).toBe(2);
    const page = await pool.query(`SELECT title FROM pages WHERE space_id=$1 AND title='Home'`, [spaceId]);
    expect(page.rows.length).toBe(1);
    const r2 = await importVoid1({ sqlitePath: dbPath, spaceId });   // re-run
    expect(r2.pages + r2.projects + r2.tasks + r2.conversations + r2.messages).toBe(0);
  });
});
  • Step 2: Run → FAIL.
  • Step 3: Implement migrate/sources/void1.js
import { DatabaseSync } from 'node:sqlite';
import { pool } from '../../lib/db/pool.js';
import * as pages from '../../lib/db/repos/pages.js';
import * as projects from '../../lib/db/repos/projects.js';
import * as tasks from '../../lib/db/repos/tasks.js';
import * as messages from '../../lib/db/repos/messages.js';
import * as map from '../../lib/db/repos/migration_map.js';

const SYS = { kind: 'system', id: null };
const slugify = (s) => (s || '').toLowerCase().replace(/[^a-z0-9]+/g, '-').replace(/(^-|-$)/g, '').slice(0, 60) || 'item';

// conversations.create() has no space_id param; insert directly so migrated chats
// live in the void1 Space.
async function createConversation(title, spaceId) {
  const { rows: [r] } = await pool.query(
    `INSERT INTO conversations(title, space_id, metadata) VALUES($1,$2,'{}') RETURNING id`, [title || 'Conversation', spaceId]);
  return r.id;
}

export async function importVoid1({ sqlitePath, spaceId, dryRun = false }) {
  const db = new DatabaseSync(sqlitePath, { readOnly: true });
  const n = { pages: 0, projects: 0, tasks: 0, conversations: 0, messages: 0 };
  const projIds = new Map();   // void1 project id -> void2 project id

  for (const w of db.prepare(`SELECT * FROM wiki_pages`).all()) {
    if (await map.seen('void1', `wiki_pages:${w.id}`, 'page')) continue;
    n.pages++; if (dryRun) continue;
    const p = await pages.create({ space_id: spaceId, slug: slugify(w.slug || w.title), title: w.title || 'Untitled', body_md: w.content || '' }, SYS);
    await map.record('void1', `wiki_pages:${w.id}`, 'page', p.id);
  }
  for (const pr of db.prepare(`SELECT * FROM projects`).all()) {
    const existing = await map.seen('void1', `projects:${pr.id}`, 'project');
    if (existing) { projIds.set(pr.id, existing); continue; }
    n.projects++; if (dryRun) continue;
    const created = await projects.create({ space_id: spaceId, slug: slugify(pr.name), name: pr.name || 'Project', description: pr.description || null, status: pr.status === 'archived' ? 'archived' : 'active' }, SYS);
    projIds.set(pr.id, created.id);
    await map.record('void1', `projects:${pr.id}`, 'project', created.id);
  }
  for (const t of db.prepare(`SELECT * FROM project_tasks`).all()) {
    if (await map.seen('void1', `project_tasks:${t.id}`, 'task')) continue;
    n.tasks++; if (dryRun) continue;
    const created = await tasks.create({ space_id: spaceId, project_id: projIds.get(t.project_id) || null, title: t.text || 'Task', position: t.position ?? 0 }, SYS);
    await map.record('void1', `project_tasks:${t.id}`, 'task', created.id);
  }
  for (const j of db.prepare(`SELECT * FROM project_journal`).all()) {
    if (await map.seen('void1', `project_journal:${j.id}`, 'page')) continue;
    n.pages++; if (dryRun) continue;
    const p = await pages.create({ space_id: spaceId, slug: slugify(`journal-${j.id}`), title: `Journal ${j.id}`, body_md: j.content || '' }, SYS);
    await map.record('void1', `project_journal:${j.id}`, 'page', p.id);
  }
  for (const c of db.prepare(`SELECT * FROM conversations`).all()) {
    let convId = await map.seen('void1', `conversations:${c.id}`, 'conversation');
    if (!convId) {
      n.conversations++; if (dryRun) continue;
      convId = await createConversation(c.title, spaceId);
      await map.record('void1', `conversations:${c.id}`, 'conversation', convId);
    }
    for (const m of db.prepare(`SELECT * FROM messages WHERE conversation_id=? ORDER BY created_at`).all(c.id)) {
      if (await map.seen('void1', `messages:${m.id}`, 'message')) continue;
      n.messages++; if (dryRun) continue;
      const appended = await messages.append(convId, { role: m.role === 'assistant' ? 'assistant' : 'user', body: m.content || '' });
      await map.record('void1', `messages:${m.id}`, 'message', appended.id);
    }
  }
  db.close();
  return n;
}
  • Step 4: Run → PASS. Commit feat(migrate): Void 1 SQLite importer

Task 5: Karakeep importer (refs via upsertByExternal)

Files: Create migrate/sources/karakeep.js; Test tests/migrate/karakeep.test.js

  • Step 1: Failing test
import { describe, it, expect, beforeAll, vi } from 'vitest';
import { pool } from '../../lib/db/pool.js';
import { resetDb } from '../helpers/db.js';
import { migrateUp } from '../../lib/db/migrate.js';
import { ensureSpace } from '../../migrate/spaces.js';
import { importKarakeep } from '../../migrate/sources/karakeep.js';

let spaceId;
beforeAll(async () => { await resetDb(); await migrateUp(); spaceId = await ensureSpace('bookmarks', 'Bookmarks'); });

describe('karakeep importer', () => {
  it('imports bookmarks as url refs, idempotently', async () => {
    const page = { bookmarks: [
      { id: 'b1', content: { type: 'link', url: 'https://a.test', title: 'A' } },
      { id: 'b2', content: { type: 'link', url: 'https://b.test', title: 'B' } }
    ], nextCursor: null };
    const fetchImpl = vi.fn(async () => ({ ok: true, json: async () => page }));
    const r1 = await importKarakeep({ apiUrl: 'http://k', token: 't', spaceId, fetchImpl });
    expect(r1.created).toBe(2);
    const { rows } = await pool.query(`SELECT url FROM refs WHERE space_id=$1 ORDER BY url`, [spaceId]);
    expect(rows.map(r => r.url)).toEqual(['https://a.test', 'https://b.test']);
    const r2 = await importKarakeep({ apiUrl: 'http://k', token: 't', spaceId, fetchImpl }); // upsert, no dupes
    const { rows: again } = await pool.query(`SELECT count(*)::int n FROM refs WHERE space_id=$1`, [spaceId]);
    expect(again[0].n).toBe(2);
  });
});
  • Step 2: Run → FAIL.
  • Step 3: Implement migrate/sources/karakeep.js
import * as refs from '../../lib/db/repos/refs.js';

const SYS = { kind: 'system', id: null };

// Karakeep REST: GET /api/v1/bookmarks?cursor=... → { bookmarks:[{id,content:{url,title}}], nextCursor }.
export async function importKarakeep({ apiUrl = process.env.KARAKEEP_URL, token = process.env.KARAKEEP_TOKEN, spaceId, dryRun = false, fetchImpl = fetch }) {
  let cursor = null, created = 0;
  do {
    const u = `${apiUrl}/api/v1/bookmarks?limit=100${cursor ? `&cursor=${encodeURIComponent(cursor)}` : ''}`;
    const res = await fetchImpl(u, { headers: { Authorization: `Bearer ${token}` } });
    if (!res.ok) throw new Error(`karakeep ${res.status}`);
    const body = await res.json();
    for (const b of (body.bookmarks || [])) {
      const url = b.content?.url; if (!url) continue;
      created++;
      if (dryRun) continue;
      await refs.upsertByExternal({ space_id: spaceId, kind: 'url', url, title: b.content?.title || url, source_kind: 'karakeep', external_id: String(b.id) }, SYS);
    }
    cursor = body.nextCursor || null;
  } while (cursor);
  return { created };
}
  • Step 4: Run → PASS. Commit feat(migrate): Karakeep bookmarks importer

Task 6: BookStack importer

Files: Create migrate/sources/bookstack.js; Test tests/migrate/bookstack.test.js

  • Step 1: Failing test
import { describe, it, expect, beforeAll, vi } from 'vitest';
import { pool } from '../../lib/db/pool.js';
import { resetDb } from '../helpers/db.js';
import { migrateUp } from '../../lib/db/migrate.js';
import { ensureSpace } from '../../migrate/spaces.js';
import { importBookstack } from '../../migrate/sources/bookstack.js';

let spaceId;
beforeAll(async () => { await resetDb(); await migrateUp(); spaceId = await ensureSpace('wiki', 'Wiki'); });

describe('bookstack importer', () => {
  it('imports pages with markdown body, idempotently', async () => {
    const list = { data: [{ id: 1, name: 'Page One' }, { id: 2, name: 'Page Two' }] };
    const detail = (id) => ({ id, name: id === 1 ? 'Page One' : 'Page Two', markdown: `# P${id}\nbody`, html: `<p>body</p>` });
    const fetchImpl = vi.fn(async (u) => {
      if (u.endsWith('/api/pages')) return { ok: true, json: async () => list };
      const id = Number(u.split('/').pop());
      return { ok: true, json: async () => detail(id) };
    });
    const r1 = await importBookstack({ apiUrl: 'http://bs', tokenId: 'i', tokenSecret: 's', spaceId, fetchImpl });
    expect(r1.created).toBe(2);
    const { rows } = await pool.query(`SELECT title FROM pages WHERE space_id=$1 ORDER BY title`, [spaceId]);
    expect(rows.map(r => r.title)).toEqual(['Page One', 'Page Two']);
    const r2 = await importBookstack({ apiUrl: 'http://bs', tokenId: 'i', tokenSecret: 's', spaceId, fetchImpl });
    expect(r2.created).toBe(0);
  });
});
  • Step 2: Run → FAIL.
  • Step 3: Implement migrate/sources/bookstack.js
import * as pages from '../../lib/db/repos/pages.js';
import * as map from '../../lib/db/repos/migration_map.js';

const SYS = { kind: 'system', id: null };
const slugify = (s) => (s || '').toLowerCase().replace(/[^a-z0-9]+/g, '-').replace(/(^-|-$)/g, '').slice(0, 60) || 'page';

// BookStack REST: list GET /api/pages → {data:[{id,name}]}; detail GET /api/pages/:id
// → {id,name,markdown,html}. Auth header: "Token <id>:<secret>".
export async function importBookstack({ apiUrl = process.env.BOOKSTACK_URL, tokenId = process.env.BOOKSTACK_TOKEN_ID, tokenSecret = process.env.BOOKSTACK_TOKEN_SECRET, spaceId, dryRun = false, fetchImpl = fetch }) {
  const headers = { Authorization: `Token ${tokenId}:${tokenSecret}` };
  const listRes = await fetchImpl(`${apiUrl}/api/pages`, { headers });
  if (!listRes.ok) throw new Error(`bookstack list ${listRes.status}`);
  const { data = [] } = await listRes.json();
  let created = 0;
  for (const item of data) {
    if (await map.seen('bookstack', `page:${item.id}`, 'page')) continue;
    created++; if (dryRun) continue;
    const dRes = await fetchImpl(`${apiUrl}/api/pages/${item.id}`, { headers });
    if (!dRes.ok) throw new Error(`bookstack page ${item.id} ${dRes.status}`);
    const d = await dRes.json();
    const body = d.markdown && d.markdown.trim() ? d.markdown : (d.html || '');
    const p = await pages.create({ space_id: spaceId, slug: slugify(d.name), title: d.name || 'Untitled', body_md: body }, SYS);
    await map.record('bookstack', `page:${item.id}`, 'page', p.id);
  }
  return { created };
}
  • Step 4: Run → PASS. Commit feat(migrate): BookStack importer

Task 7: CLI + verify

Files: Create migrate/cli.js, migrate/verify.js; Test tests/migrate/verify.test.js

  • Step 1: migrate/verify.js — counts migrated rows per source/type from migration_map + refs:
import { pool } from '../lib/db/pool.js';

export async function verify() {
  const { rows: map } = await pool.query(
    `SELECT source, entity_type, count(*)::int n FROM migration_map GROUP BY source, entity_type ORDER BY source, entity_type`);
  const { rows: [{ n: bookmarks }] } = await pool.query(`SELECT count(*)::int n FROM refs WHERE source_kind='karakeep'`);
  return { map, bookmarks };
}
  • Step 2: Verify test tests/migrate/verify.test.js
import { describe, it, expect, beforeAll } from 'vitest';
import { resetDb } from '../helpers/db.js';
import { migrateUp } from '../../lib/db/migrate.js';
import * as map from '../../lib/db/repos/migration_map.js';
import { verify } from '../../migrate/verify.js';
import { randomUUID } from 'crypto';

beforeAll(async () => { await resetDb(); await migrateUp(); await map.record('plans', 'a.md', 'page', randomUUID()); });

describe('verify', () => {
  it('reports per-source/type counts', async () => {
    const out = await verify();
    expect(out.map.find(r => r.source === 'plans' && r.entity_type === 'page').n).toBe(1);
  });
});

  • Step 3: Run verify test → FAIL then PASS after creating verify.js.

  • Step 4: migrate/cli.js (dispatch; manual-run entry point):

import 'dotenv/config';
import { ensureSpace } from './spaces.js';
import { importPlans } from './sources/plans.js';
import { importVoid1 } from './sources/void1.js';
import { importKarakeep } from './sources/karakeep.js';
import { importBookstack } from './sources/bookstack.js';
import { verify } from './verify.js';

const [, , cmd, ...rest] = process.argv;
const dryRun = rest.includes('--dry-run');

const run = {
  async plans() { return importPlans({ dir: process.env.PLANS_DIR || '/root/.claude/plans', spaceId: await ensureSpace('plans', 'Plans'), dryRun }); },
  async void1() { return importVoid1({ sqlitePath: process.env.VOID1_DB || '/tmp/void1.db', spaceId: await ensureSpace('void1', 'Void 1'), dryRun }); },
  async karakeep() { return importKarakeep({ spaceId: await ensureSpace('bookmarks', 'Bookmarks'), dryRun }); },
  async bookstack() { return importBookstack({ spaceId: await ensureSpace('wiki', 'Wiki'), dryRun }); },
  async verify() { return verify(); }
};

if (!run[cmd]) { console.error(`usage: node migrate/cli.js <plans|void1|karakeep|bookstack|verify> [--dry-run]`); process.exit(1); }
run[cmd]().then(r => { console.log(JSON.stringify(r, null, 2)); process.exit(0); })
  .catch(e => { console.error('migrate failed:', e.message); process.exit(1); });
  • Step 5: Run → PASS. Commit feat(migrate): CLI dispatch + verify

Task 8: Run the migration (gated execution — owner-authorized)

Files: none. Requires: migration 018 applied to the void2 DB + source access.

  • Step 1: Apply 018 to prod — deploy (bash deploy/push.sh; runs 018_migration_map) OR npm run migrate against the void2 DB.
  • Step 2: Stage sources on the run host (CT 311 or dev box with the void2 DATABASE_URL):
    • scp Void 1's void.db from CT 301 → set VOID1_DB.
    • Set PLANS_DIR to the plans dir.
    • BookStack: create an API token (Settings → API Tokens) → BOOKSTACK_URL, BOOKSTACK_TOKEN_ID, BOOKSTACK_TOKEN_SECRET.
    • Karakeep: create an API key → KARAKEEP_URL, KARAKEEP_TOKEN.
  • Step 3: Dry-run each — node migrate/cli.js <src> --dry-run → review would-create counts.
  • Step 4: Import — node migrate/cli.js plans|void1|karakeep|bookstack.
  • Step 5: node migrate/cli.js verify — confirm counts vs sources; spot-check a few entities in the Void 2 UI (the 4 new Spaces).
  • Step 6: Memory — record migrated counts + that Plan 8a is done; note 8b (cutover) is the remaining gate to 2.0.0.

Self-Review

Spec coverage: migration_map→T1; ensureSpace/per-source Spaces→T2; plans→T3; void1 (wiki/projects/tasks/journal/conversations/messages)→T4; karakeep (refs upsertByExternal)→T5; bookstack→T6; CLI+dry-run+verify→T7; run/verify→T8. Idempotency tested in T3–T6 (re-run → 0). Out-of-scope (8b cutover) deferred. Covered.

Placeholder scan: T8 env values (tokens, paths) are runtime provisioning inputs, not code gaps. No TBDs in code.

Type consistency: importers return {created} (plans/karakeep/bookstack) and {pages,projects,tasks,conversations,messages} (void1) — tests match. map.seen(source,source_id,type)/map.record(...) consistent T1 def + T3–T6 use. ensureSpace(slug,name)→id consistent. Repo calls match verified signatures (pages.create({space_id,slug,title,body_md}), tasks.create({space_id,project_id,title,position}), refs.upsertByExternal({space_id,kind,url,title,source_kind,external_id}), messages.append(convId,{role,body})).

Reference in New Issue View Git Blame Copy Permalink