mirror of
https://github.com/Patiun/com.gregkilmer.www.git
synced 2026-11-07 19:58:24 +00:00
Add the content service and MCP server (phase 2)
services/content watches the vault's Site folder, runs every change through fixed rules, keeps the last accepted version of each note with its history, and serves one write API. The HTTP API, the MCP server for agents, and the admin page and editing tools to come all write through it. - SQLite store with history, review queue, redirects, links and full-text search - Rules: invalid, misplaced, duplicate name or slug, text loss, secrets, agent contact details, agent edits, agent publishing, agent batches - Watcher with debounce and a periodic full scan - HTTP API with reader, agent, editor and admin tokens - MCP tools: search, get_note, get_schema, list_held, set_status, add_library_item, update_fields, draft_post - Tests, including taking in all 545 migrated notes with nothing held Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012aSbo2D1qP8cMFQCykf7kx
This commit is contained in:
Claude
@@ -19,3 +19,4 @@ npm-debug.log*
|
|||||||
!.vscode/extensions.json
|
!.vscode/extensions.json
|
||||||
!.vscode/settings.json
|
!.vscode/settings.json
|
||||||
.idea/
|
.idea/
|
||||||
|
data/
|
||||||
|
|||||||
@@ -12,12 +12,12 @@ with the decisions behind it, is the
|
|||||||
| --- | --- |
|
| --- | --- |
|
||||||
| `packages/schema` | The one definition of every note type: fields, folders, file names, and the checks for one note and for the whole folder. The migration, the content service, the MCP server, the site and any later editing tool all use it. |
|
| `packages/schema` | The one definition of every note type: fields, folders, file names, and the checks for one note and for the whole folder. The migration, the content service, the MCP server, the site and any later editing tool all use it. |
|
||||||
| `vault-kit` | What goes into the vault: the Obsidian templates (generated from the schema) and the Site guide. |
|
| `vault-kit` | What goes into the vault: the Obsidian templates (generated from the schema) and the Site guide. |
|
||||||
|
| `services/content` | The content service: watches the Site folder, runs the rules, keeps the last good version and its history, and serves the one write API. The MCP server for agents is part of it. See [docs/content-service.md](docs/content-service.md). |
|
||||||
| `tools/migrate` | The one-time migration from the old site's data into Site folder notes. |
|
| `tools/migrate` | The one-time migration from the old site's data into Site folder notes. |
|
||||||
| `docs` | How the pieces fit. Start with [docs/content-model.md](docs/content-model.md). |
|
| `docs` | How the pieces fit. Start with [docs/content-model.md](docs/content-model.md). |
|
||||||
| `legacy` | The old site, kept for reference only. Nothing outside it imports from it except the migration. |
|
| `legacy` | The old site, kept for reference only. Nothing outside it imports from it except the migration. |
|
||||||
|
|
||||||
Coming in later phases: `services/content` (the content service and its MCP
|
Coming in later phases: the admin page, the component library, and the new site.
|
||||||
server), the admin page, and the new site.
|
|
||||||
|
|
||||||
## Commands
|
## Commands
|
||||||
|
|
||||||
@@ -29,6 +29,10 @@ npm run check # typecheck and tests
|
|||||||
npm run migrate # writes out/24.01_PersonalWebsite and out/MIGRATION-REPORT.md
|
npm run migrate # writes out/24.01_PersonalWebsite and out/MIGRATION-REPORT.md
|
||||||
npm run templates # regenerates vault-kit templates after a schema change
|
npm run templates # regenerates vault-kit templates after a schema change
|
||||||
npm run schema:json # writes a JSON Schema per note type to packages/schema/json/
|
npm run schema:json # writes a JSON Schema per note type to packages/schema/json/
|
||||||
|
|
||||||
|
# the content service, against a copy of the Site folder
|
||||||
|
SITE_DIR=out/24.01_PersonalWebsite/Site DATA_DIR=data \
|
||||||
|
CONTENT_TOKENS="reader:<24+ chars>,admin:<24+ chars>" npm run content
|
||||||
```
|
```
|
||||||
|
|
||||||
## The old site
|
## The old site
|
||||||
|
|||||||
@@ -0,0 +1,120 @@
|
|||||||
|
# Content service
|
||||||
|
|
||||||
|
The content service sits between the vault and everything that shows or
|
||||||
|
changes content. It is the only thing that reads or writes the Site folder.
|
||||||
|
|
||||||
|
```
|
||||||
|
Obsidian ──edits──▶ Site folder ──watch──▶ rules ──▶ accepted store ──▶ site, admin page
|
||||||
|
▲ ▲
|
||||||
|
└──── accepted API changes ────────┤
|
||||||
|
write API (HTTP)
|
||||||
|
▲ ▲
|
||||||
|
MCP server admin page, later editors
|
||||||
|
```
|
||||||
|
|
||||||
|
- **The vault is the source of truth.** Greg edits in Obsidian; the service
|
||||||
|
watches the folder and takes each change in.
|
||||||
|
- **Every change runs the same rules**, whether it came from Obsidian, an
|
||||||
|
agent or the admin page. A change that passes is accepted. A change that
|
||||||
|
fails is held for review, and the site keeps showing the last accepted
|
||||||
|
version, so a bad edit never reaches a page.
|
||||||
|
- **Changes through the API are written back into the vault** once accepted,
|
||||||
|
so Obsidian always shows what the site shows. The service marks its own
|
||||||
|
writes as seen before writing, so the watcher does not take them in twice.
|
||||||
|
- **Deleting or moving a file unpublishes the note.** Its history stays, and
|
||||||
|
any version can be rolled back.
|
||||||
|
|
||||||
|
## The rules
|
||||||
|
|
||||||
|
Pure functions in [`services/content/src/rules.ts`](../services/content/src/rules.ts),
|
||||||
|
run in this order. The rules live in code, not in the vault, so no agent can
|
||||||
|
loosen them.
|
||||||
|
|
||||||
|
| Code | Holds a change when |
|
||||||
|
| --- | --- |
|
||||||
|
| `invalid` | The note fails its type's schema. |
|
||||||
|
| `misplaced` | A settings note (`_site`, `Now/_kinds`, `Store/_settings`) is not at its fixed path. |
|
||||||
|
| `duplicate-name`, `duplicate-slug` | Another live note already has the file name, or the same type already uses the address. |
|
||||||
|
| `text-loss` | A body of 400 or more characters loses more than half in one edit. |
|
||||||
|
| `secret` | New text looks like a key, token or password. |
|
||||||
|
| `contact` | An agent adds an email address or phone number. |
|
||||||
|
| `agent-edit` | An agent changes a note it did not write, unless the note has `agentEdits: true`. |
|
||||||
|
| `agent-publish` | An agent publishes anything but a status or a Library item. |
|
||||||
|
| `batch` | An agent makes more than 10 changes in 10 minutes. |
|
||||||
|
|
||||||
|
Invalid input through the API is turned down at once with the problems, rather
|
||||||
|
than held, so callers fix it themselves. A change from Obsidian that fails is
|
||||||
|
held, because the file is already on disk.
|
||||||
|
|
||||||
|
## Addresses
|
||||||
|
|
||||||
|
A slug change or a rename keeps the old address working as a redirect. When a
|
||||||
|
note disappears and a note of the same type with the same text or title
|
||||||
|
appears within ten minutes, that is treated as a move: redirects to the old
|
||||||
|
path are pointed at the new one, so redirects never chain. A slug that comes
|
||||||
|
back into use belongs to its note, not to the redirect.
|
||||||
|
|
||||||
|
## The write API
|
||||||
|
|
||||||
|
`ContentService.apply` takes one change, `create` (type, fields, body) or
|
||||||
|
`update` (path, the fields to change, `null` to remove one, and optionally
|
||||||
|
the body), along with who made it. Every client uses it:
|
||||||
|
|
||||||
|
- the MCP server, as `agent`;
|
||||||
|
- the admin page and later editing tools, as `greg` (editor token) or `admin`;
|
||||||
|
- the review queue: `acceptHeld`, `rejectHeld` (optionally restoring the live
|
||||||
|
version into the vault), and `rollback` to any earlier revision.
|
||||||
|
|
||||||
|
An update first takes in any unsaved Obsidian edit to that file, so an API
|
||||||
|
change never overwrites one. A note with a change waiting for review takes no
|
||||||
|
other API change until that one is decided. A note's type never changes.
|
||||||
|
|
||||||
|
### HTTP
|
||||||
|
|
||||||
|
Every route but `/health` needs `Authorization: Bearer <token>`. Tokens map to
|
||||||
|
roles in `CONTENT_TOKENS`.
|
||||||
|
|
||||||
|
| Route | Who | What |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| `GET /health` | anyone | Note, published and held counts. |
|
||||||
|
| `GET /api/schema` | all | Every note type with its JSON Schema. |
|
||||||
|
| `GET /api/notes?type=&q=&limit=&offset=&full=` | all | List or search. Readers see published notes only. |
|
||||||
|
| `GET /api/note?path=` or `?type=&slug=` | all | One note with its backlinks. An old slug answers with `{ redirect }`. |
|
||||||
|
| `GET /api/backlinks?name=` | all | Notes linking to a note. |
|
||||||
|
| `GET /api/redirects` | all | Every redirect. |
|
||||||
|
| `GET /attachments/<file>` | all | An image from `_attachments`. |
|
||||||
|
| `GET /api/note/history?path=`, `/api/note/revision?path=&rev=` | agent, editor, admin | History and old versions. |
|
||||||
|
| `GET /api/problems` | agent, editor, admin | Broken links and other whole-folder problems. |
|
||||||
|
| `GET /api/held`, `/api/held/<id>` | agent, editor, admin | The review queue; one entry comes with the current text for comparison. |
|
||||||
|
| `POST /api/changes` | agent, editor, admin | One change. 422 with reasons when turned down. |
|
||||||
|
| `POST /api/held/<id>/accept`, `/reject` | editor, admin | Decide a held change. `{ "restore": true }` on reject puts the live version back into the vault. |
|
||||||
|
| `POST /api/note/rollback` | admin | `{ path, rev }` |
|
||||||
|
| `POST /api/scan` | editor, admin | Compare the whole folder now. |
|
||||||
|
| `/mcp` | agent, editor, admin | The MCP server. |
|
||||||
|
|
||||||
|
### MCP
|
||||||
|
|
||||||
|
Streamable HTTP at `/mcp`, stateless. Tools: `search`, `get_note`,
|
||||||
|
`get_schema`, `list_held`, `set_status`, `add_library_item`, `update_fields`
|
||||||
|
and `draft_post` (always unpublished). There is no tool to delete, rename,
|
||||||
|
move or write a raw file.
|
||||||
|
|
||||||
|
## Running it
|
||||||
|
|
||||||
|
| Variable | Default | |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| `SITE_DIR` | required | The vault's Site folder. |
|
||||||
|
| `DATA_DIR` | `data` | Holds `content.db`. Back it up: history and the review queue cannot be rebuilt from the vault. |
|
||||||
|
| `PORT`, `HOST` | `4310`, `0.0.0.0` | |
|
||||||
|
| `CONTENT_TOKENS` | none | `role:token` pairs, comma separated. Roles: reader, agent, editor, admin. Tokens are 24 characters or more. |
|
||||||
|
| `ALLOWED_EMAILS` | none | Addresses an agent may add without a hold. |
|
||||||
|
| `HEALTHCHECK_URL` | none | Pinged after every full scan. |
|
||||||
|
| `NOTIFY_WEBHOOK_URL` | none | Receives `{ event: "held", held }` when a change is held. |
|
||||||
|
| `SCAN_MINUTES` | `5` | Full scan interval, on top of the file watch. |
|
||||||
|
|
||||||
|
```sh
|
||||||
|
SITE_DIR=/path/to/Site CONTENT_TOKENS="reader:…,agent:…,editor:…,admin:…" npm run content
|
||||||
|
```
|
||||||
|
|
||||||
|
The store is SQLite (`node:sqlite`) with full-text search, so the service has
|
||||||
|
no database server to run.
|
||||||
Generated
+1206
-29
File diff suppressed because it is too large
Load Diff
+4
-2
@@ -5,18 +5,20 @@
|
|||||||
"description": "GregKilmer.com: the note schema, content service and site that serve Greg's vault notes. The old site lives in legacy/ for reference only.",
|
"description": "GregKilmer.com: the note schema, content service and site that serve Greg's vault notes. The old site lives in legacy/ for reference only.",
|
||||||
"workspaces": [
|
"workspaces": [
|
||||||
"packages/*",
|
"packages/*",
|
||||||
|
"services/*",
|
||||||
"tools/*"
|
"tools/*"
|
||||||
],
|
],
|
||||||
"engines": {
|
"engines": {
|
||||||
"node": ">=22"
|
"node": ">=22"
|
||||||
},
|
},
|
||||||
"scripts": {
|
"scripts": {
|
||||||
"typecheck": "tsc -p packages/schema && tsc -p tools/migrate",
|
"typecheck": "tsc -p packages/schema && tsc -p tools/migrate && tsc -p services/content",
|
||||||
"test": "vitest run",
|
"test": "vitest run",
|
||||||
"check": "npm run typecheck && npm test",
|
"check": "npm run typecheck && npm test",
|
||||||
"schema:json": "npm run json -w @gk/schema",
|
"schema:json": "npm run json -w @gk/schema",
|
||||||
"templates": "npm run templates -w @gk/schema",
|
"templates": "npm run templates -w @gk/schema",
|
||||||
"migrate": "npm run migrate -w @gk/migrate --"
|
"migrate": "npm run migrate -w @gk/migrate --",
|
||||||
|
"content": "npm run start -w @gk/content"
|
||||||
},
|
},
|
||||||
"devDependencies": {
|
"devDependencies": {
|
||||||
"@types/node": "^22.10.0",
|
"@types/node": "^22.10.0",
|
||||||
|
|||||||
@@ -87,6 +87,19 @@ function orderKeys(def: NoteTypeDef, data: Record<string, unknown>): Record<stri
|
|||||||
return out;
|
return out;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Format a note's text without checking it, so a caller can run checkNote on
|
||||||
|
* the result and report problems field by field.
|
||||||
|
*/
|
||||||
|
export function formatNote(data: Record<string, unknown>, body = ""): string {
|
||||||
|
const def = typeDef(String(data.type));
|
||||||
|
const ordered = def ? orderKeys(def, data) : data;
|
||||||
|
// Write what was given (not schema defaults), so notes stay short.
|
||||||
|
const yaml = stringifyYaml(ordered, { lineWidth: 0 }).trimEnd();
|
||||||
|
const text = body.trim() ? `\n${body.trim()}\n` : "";
|
||||||
|
return `---\n${yaml}\n---\n${text}`;
|
||||||
|
}
|
||||||
|
|
||||||
/** Write a note's text. The data is checked first; this throws when it fails. */
|
/** Write a note's text. The data is checked first; this throws when it fails. */
|
||||||
export function writeNote(data: Record<string, unknown>, body = ""): string {
|
export function writeNote(data: Record<string, unknown>, body = ""): string {
|
||||||
const def = typeDef(String(data.type));
|
const def = typeDef(String(data.type));
|
||||||
@@ -96,10 +109,7 @@ export function writeNote(data: Record<string, unknown>, body = ""): string {
|
|||||||
const problems = result.error.issues.map((i) => `${i.path.join(".") || "note"}: ${i.message}`).join("; ");
|
const problems = result.error.issues.map((i) => `${i.path.join(".") || "note"}: ${i.message}`).join("; ");
|
||||||
throw new Error(`"${String(data.title)}" is not a valid ${def.label}: ${problems}`);
|
throw new Error(`"${String(data.title)}" is not a valid ${def.label}: ${problems}`);
|
||||||
}
|
}
|
||||||
// Write what was given (not schema defaults), so notes stay short.
|
return formatNote(data, body);
|
||||||
const yaml = stringifyYaml(orderKeys(def, data), { lineWidth: 0 }).trimEnd();
|
|
||||||
const text = body.trim() ? `\n${body.trim()}\n` : "";
|
|
||||||
return `---\n${yaml}\n---\n${text}`;
|
|
||||||
}
|
}
|
||||||
|
|
||||||
/** Where a note of this type lives, relative to the Site folder, with `.md`. */
|
/** Where a note of this type lives, relative to the Site folder, with `.md`. */
|
||||||
|
|||||||
@@ -0,0 +1,17 @@
|
|||||||
|
{
|
||||||
|
"name": "@gk/content",
|
||||||
|
"version": "0.1.0",
|
||||||
|
"private": true,
|
||||||
|
"type": "module",
|
||||||
|
"description": "The content service: watches the vault's Site folder, checks every change, keeps the last good version of each note, and serves the one write API that the MCP server, the admin page and later editing tools share.",
|
||||||
|
"scripts": {
|
||||||
|
"start": "node --disable-warning=ExperimentalWarning --import tsx src/main.ts",
|
||||||
|
"dev": "node --disable-warning=ExperimentalWarning --watch --import tsx src/main.ts"
|
||||||
|
},
|
||||||
|
"dependencies": {
|
||||||
|
"@gk/schema": "*",
|
||||||
|
"@modelcontextprotocol/sdk": "^1.32.1",
|
||||||
|
"tsx": "^4.19.0",
|
||||||
|
"zod": "^4.1.0"
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,65 @@
|
|||||||
|
/**
|
||||||
|
* Settings come from the environment, so the same image runs on Cosmos and in
|
||||||
|
* tests. Nothing here is read from the vault: an agent that can edit notes
|
||||||
|
* cannot change who may write or what the rules allow.
|
||||||
|
*/
|
||||||
|
import { resolve } from "node:path";
|
||||||
|
|
||||||
|
export type Role = "reader" | "agent" | "editor" | "admin";
|
||||||
|
|
||||||
|
export interface Config {
|
||||||
|
/** The vault's Site folder. */
|
||||||
|
siteDir: string;
|
||||||
|
/** The SQLite file holding accepted notes, history and the review queue. */
|
||||||
|
dbFile: string;
|
||||||
|
port: number;
|
||||||
|
host: string;
|
||||||
|
/** Bearer token to role. */
|
||||||
|
tokens: Map<string, Role>;
|
||||||
|
/** Pinged after every full scan, so a stopped watcher is noticed. */
|
||||||
|
healthcheckUrl?: string;
|
||||||
|
/** Receives a JSON message whenever a change is held. */
|
||||||
|
notifyWebhookUrl?: string;
|
||||||
|
allowedEmails: string[];
|
||||||
|
scanMinutes: number;
|
||||||
|
}
|
||||||
|
|
||||||
|
const roles = new Set<Role>(["reader", "agent", "editor", "admin"]);
|
||||||
|
|
||||||
|
/** CONTENT_TOKENS is a comma separated list of role:token pairs. */
|
||||||
|
export function parseTokens(raw: string | undefined): Map<string, Role> {
|
||||||
|
const tokens = new Map<string, Role>();
|
||||||
|
for (const pair of (raw ?? "").split(",").map((s) => s.trim()).filter(Boolean)) {
|
||||||
|
const at = pair.indexOf(":");
|
||||||
|
const role = pair.slice(0, at) as Role;
|
||||||
|
const token = pair.slice(at + 1);
|
||||||
|
if (at < 1 || !roles.has(role)) throw new Error(`CONTENT_TOKENS: "${pair.slice(0, Math.max(at, 0))}" is not a role`);
|
||||||
|
if (token.length < 24) throw new Error(`CONTENT_TOKENS: the ${role} token is shorter than 24 characters`);
|
||||||
|
tokens.set(token, role);
|
||||||
|
}
|
||||||
|
return tokens;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function loadConfig(env: NodeJS.ProcessEnv = process.env): Config {
|
||||||
|
if (!env.SITE_DIR) throw new Error("SITE_DIR is required: the vault's Site folder");
|
||||||
|
const dataDir = resolve(env.DATA_DIR ?? "data");
|
||||||
|
return {
|
||||||
|
siteDir: resolve(env.SITE_DIR),
|
||||||
|
dbFile: resolve(dataDir, "content.db"),
|
||||||
|
port: Number(env.PORT ?? 4310),
|
||||||
|
host: env.HOST ?? "0.0.0.0",
|
||||||
|
tokens: parseTokens(env.CONTENT_TOKENS),
|
||||||
|
healthcheckUrl: env.HEALTHCHECK_URL || undefined,
|
||||||
|
notifyWebhookUrl: env.NOTIFY_WEBHOOK_URL || undefined,
|
||||||
|
allowedEmails: (env.ALLOWED_EMAILS ?? "").split(",").map((s) => s.trim().toLowerCase()).filter(Boolean),
|
||||||
|
scanMinutes: Number(env.SCAN_MINUTES ?? 5),
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
/** What each role may do. Checked by the HTTP API and the MCP server alike. */
|
||||||
|
export const can = {
|
||||||
|
readDrafts: (r: Role) => r !== "reader",
|
||||||
|
write: (r: Role) => r === "agent" || r === "editor" || r === "admin",
|
||||||
|
review: (r: Role) => r === "editor" || r === "admin",
|
||||||
|
rollback: (r: Role) => r === "admin",
|
||||||
|
};
|
||||||
@@ -0,0 +1,262 @@
|
|||||||
|
/**
|
||||||
|
* The HTTP API. The site reads through it with the reader token; the admin
|
||||||
|
* page and later editing tools read and write through it; agents use the MCP
|
||||||
|
* server mounted at /mcp. Every write lands in `ContentService.apply` or a
|
||||||
|
* review method, so the rules are the same whichever door a change comes in by.
|
||||||
|
*/
|
||||||
|
import { createReadStream } from "node:fs";
|
||||||
|
import { createServer, type IncomingMessage, type ServerResponse } from "node:http";
|
||||||
|
import { extname } from "node:path";
|
||||||
|
import { z } from "zod";
|
||||||
|
import { noteTypes, typeDef } from "@gk/schema";
|
||||||
|
import { can, type Config, type Role } from "./config.ts";
|
||||||
|
import { handleMcp } from "./mcp.ts";
|
||||||
|
import type { Change, ContentService } from "./service.ts";
|
||||||
|
import type { Author, StoredNote } from "./store.ts";
|
||||||
|
import { heldView, noteView, outcomeView, revisionView, summaryView } from "./views.ts";
|
||||||
|
|
||||||
|
const MAX_BODY = 1_000_000;
|
||||||
|
|
||||||
|
class HttpError extends Error {
|
||||||
|
constructor(readonly status: number, message: string) {
|
||||||
|
super(message);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
export const authorFor = (role: Role): Author => (role === "agent" ? "agent" : role === "admin" ? "admin" : "greg");
|
||||||
|
|
||||||
|
const changeSchema = z.discriminatedUnion("op", [
|
||||||
|
z.object({ op: z.literal("create"), data: z.record(z.string(), z.unknown()), body: z.string().optional() }),
|
||||||
|
z.object({
|
||||||
|
op: z.literal("update"),
|
||||||
|
path: z.string(),
|
||||||
|
fields: z.record(z.string(), z.unknown()).optional(),
|
||||||
|
body: z.string().optional(),
|
||||||
|
}),
|
||||||
|
]);
|
||||||
|
|
||||||
|
const types = {
|
||||||
|
".png": "image/png",
|
||||||
|
".jpg": "image/jpeg",
|
||||||
|
".jpeg": "image/jpeg",
|
||||||
|
".webp": "image/webp",
|
||||||
|
".gif": "image/gif",
|
||||||
|
".svg": "image/svg+xml",
|
||||||
|
".avif": "image/avif",
|
||||||
|
".pdf": "application/pdf",
|
||||||
|
} as Record<string, string>;
|
||||||
|
|
||||||
|
/** Every note type with its JSON Schema, for the admin page, editors and agents. */
|
||||||
|
export function schemaList() {
|
||||||
|
return Object.values(noteTypes).map((def) => ({
|
||||||
|
type: def.type,
|
||||||
|
label: def.label,
|
||||||
|
description: def.description,
|
||||||
|
folder: typeof def.folder === "string" ? def.folder : null,
|
||||||
|
singleton: def.singleton?.fileName ?? null,
|
||||||
|
body: def.body,
|
||||||
|
schema: z.toJSONSchema(def.schema, { io: "input", unrepresentable: "any" }),
|
||||||
|
}));
|
||||||
|
}
|
||||||
|
|
||||||
|
export function createHttpServer(service: ContentService, config: Pick<Config, "tokens">) {
|
||||||
|
const { store } = service;
|
||||||
|
|
||||||
|
const visible = (n: StoredNote | null, role: Role) => (n && !n.removed && (n.publish || can.readDrafts(role)) ? n : null);
|
||||||
|
|
||||||
|
async function route(req: IncomingMessage, res: ServerResponse) {
|
||||||
|
const url = new URL(req.url ?? "/", "http://localhost");
|
||||||
|
const path = url.pathname;
|
||||||
|
const q = (k: string) => url.searchParams.get(k) ?? undefined;
|
||||||
|
const method = req.method ?? "GET";
|
||||||
|
|
||||||
|
if (path === "/health") {
|
||||||
|
return send(res, 200, { ok: true, ...store.count() });
|
||||||
|
}
|
||||||
|
|
||||||
|
const role = authenticate(req, config.tokens);
|
||||||
|
if (!role) throw new HttpError(401, "A valid bearer token is required");
|
||||||
|
const need = (ok: boolean) => {
|
||||||
|
if (!ok) throw new HttpError(403, `The ${role} token cannot do this`);
|
||||||
|
};
|
||||||
|
|
||||||
|
if (path === "/mcp") {
|
||||||
|
need(can.write(role));
|
||||||
|
const body = method === "POST" ? await readJson(req) : undefined;
|
||||||
|
return handleMcp(req, res, body, service, authorFor(role));
|
||||||
|
}
|
||||||
|
|
||||||
|
if (method === "GET") {
|
||||||
|
if (path === "/api/schema") return send(res, 200, schemaList());
|
||||||
|
|
||||||
|
if (path === "/api/notes") {
|
||||||
|
const limit = Math.min(Number(q("limit") ?? 100), 1000);
|
||||||
|
const publishedOnly = !can.readDrafts(role) || q("published") === "true";
|
||||||
|
const type = q("type");
|
||||||
|
if (type && !typeDef(type)) throw new HttpError(400, `Unknown type "${type}"`);
|
||||||
|
const query = q("q");
|
||||||
|
const notes = query
|
||||||
|
? store.search(query, { publishedOnly, limit }).filter((n) => !type || n.type === type)
|
||||||
|
: store.list({ type, publishedOnly, limit, offset: Number(q("offset") ?? 0) });
|
||||||
|
// full=true returns whole notes, so a list page needs one request.
|
||||||
|
return send(res, 200, notes.map((n) => (q("full") === "true" ? noteView(n) : summaryView(n))));
|
||||||
|
}
|
||||||
|
|
||||||
|
if (path === "/api/note") {
|
||||||
|
const p = q("path");
|
||||||
|
const type = q("type");
|
||||||
|
const slug = q("slug");
|
||||||
|
let note: StoredNote | null = null;
|
||||||
|
if (p) note = store.get(p);
|
||||||
|
else if (type && slug) {
|
||||||
|
note = store.bySlug(type, slug);
|
||||||
|
if (!visible(note, role)) {
|
||||||
|
const to = store.redirect(type, slug);
|
||||||
|
const target = to ? visible(store.get(to), role) : null;
|
||||||
|
if (target) return send(res, 200, { redirect: { type: target.type, slug: target.slug, path: target.path } });
|
||||||
|
}
|
||||||
|
} else throw new HttpError(400, "Give path, or type and slug");
|
||||||
|
const found = visible(note, role);
|
||||||
|
if (!found) throw new HttpError(404, "No such note");
|
||||||
|
return send(res, 200, {
|
||||||
|
...noteView(found, { text: can.readDrafts(role) }),
|
||||||
|
backlinks: store.backlinks(found.name).filter((b) => visible(b, role)).map(summaryView),
|
||||||
|
...(can.readDrafts(role) ? { pending: store.pendingFor(found.path) ? heldView(store.pendingFor(found.path)!) : null } : {}),
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
if (path === "/api/note/history") {
|
||||||
|
need(can.readDrafts(role));
|
||||||
|
return send(res, 200, store.history(required(q("path"), "path")).map((r) => revisionView(r)));
|
||||||
|
}
|
||||||
|
|
||||||
|
if (path === "/api/note/revision") {
|
||||||
|
need(can.readDrafts(role));
|
||||||
|
const r = store.revision(required(q("path"), "path"), Number(required(q("rev"), "rev")));
|
||||||
|
if (!r) throw new HttpError(404, "No such revision");
|
||||||
|
return send(res, 200, revisionView(r, { text: true }));
|
||||||
|
}
|
||||||
|
|
||||||
|
if (path === "/api/backlinks") {
|
||||||
|
return send(res, 200, store.backlinks(required(q("name"), "name")).filter((b) => visible(b, role)).map(summaryView));
|
||||||
|
}
|
||||||
|
|
||||||
|
if (path === "/api/redirects") return send(res, 200, store.redirects());
|
||||||
|
|
||||||
|
if (path === "/api/problems") {
|
||||||
|
need(can.readDrafts(role));
|
||||||
|
return send(res, 200, service.problems());
|
||||||
|
}
|
||||||
|
|
||||||
|
if (path === "/api/held") {
|
||||||
|
need(can.readDrafts(role));
|
||||||
|
const status = (q("status") ?? "pending") as "pending";
|
||||||
|
return send(res, 200, store.heldList(status).map((h) => heldView(h)));
|
||||||
|
}
|
||||||
|
|
||||||
|
const heldOne = path.match(/^\/api\/held\/(\d+)$/);
|
||||||
|
if (heldOne) {
|
||||||
|
need(can.readDrafts(role));
|
||||||
|
const h = store.heldById(Number(heldOne[1]));
|
||||||
|
if (!h) throw new HttpError(404, "No such change");
|
||||||
|
const current = store.get(h.path);
|
||||||
|
// The current text comes along so a reviewer can see the difference.
|
||||||
|
return send(res, 200, { ...heldView(h, { text: true }), current: current && !current.removed ? current.text : null });
|
||||||
|
}
|
||||||
|
|
||||||
|
const attachment = path.match(/^\/attachments\/([^/]+)$/);
|
||||||
|
if (attachment) {
|
||||||
|
let name = "";
|
||||||
|
try {
|
||||||
|
name = decodeURIComponent(attachment[1] ?? "");
|
||||||
|
} catch {
|
||||||
|
throw new HttpError(400, "Bad attachment name");
|
||||||
|
}
|
||||||
|
const file = name.includes("..") ? null : service.vault.attachmentPath(name);
|
||||||
|
if (!file) throw new HttpError(404, "No such attachment");
|
||||||
|
res.writeHead(200, {
|
||||||
|
"content-type": types[extname(file).toLowerCase()] ?? "application/octet-stream",
|
||||||
|
"cache-control": "public, max-age=300",
|
||||||
|
"x-content-type-options": "nosniff",
|
||||||
|
});
|
||||||
|
createReadStream(file).pipe(res);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if (method === "POST") {
|
||||||
|
if (path === "/api/changes") {
|
||||||
|
need(can.write(role));
|
||||||
|
const parsed = changeSchema.safeParse(await readJson(req));
|
||||||
|
if (!parsed.success) throw new HttpError(400, z.prettifyError(parsed.error));
|
||||||
|
const outcome = service.apply(parsed.data as Change, authorFor(role));
|
||||||
|
return send(res, outcome.status === "rejected" ? 422 : 200, outcomeView(outcome));
|
||||||
|
}
|
||||||
|
|
||||||
|
const decide = path.match(/^\/api\/held\/(\d+)\/(accept|reject)$/);
|
||||||
|
if (decide) {
|
||||||
|
need(can.review(role));
|
||||||
|
const id = Number(decide[1]);
|
||||||
|
const body = (await readJson(req).catch(() => ({}))) as { restore?: boolean } | undefined;
|
||||||
|
const outcome = decide[2] === "accept" ? service.acceptHeld(id) : service.rejectHeld(id, { restore: body?.restore === true });
|
||||||
|
return send(res, outcome.status === "rejected" ? 422 : 200, outcomeView(outcome));
|
||||||
|
}
|
||||||
|
|
||||||
|
if (path === "/api/note/rollback") {
|
||||||
|
need(can.rollback(role));
|
||||||
|
const body = z.object({ path: z.string(), rev: z.number().int() }).safeParse(await readJson(req));
|
||||||
|
if (!body.success) throw new HttpError(400, z.prettifyError(body.error));
|
||||||
|
const outcome = service.rollback(body.data.path, body.data.rev);
|
||||||
|
return send(res, outcome.status === "rejected" ? 422 : 200, outcomeView(outcome));
|
||||||
|
}
|
||||||
|
|
||||||
|
if (path === "/api/scan") {
|
||||||
|
need(can.review(role));
|
||||||
|
return send(res, 200, service.scan());
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
throw new HttpError(404, "Not found");
|
||||||
|
}
|
||||||
|
|
||||||
|
return createServer((req, res) => {
|
||||||
|
route(req, res).catch((err: unknown) => {
|
||||||
|
if (res.headersSent) return res.end();
|
||||||
|
if (err instanceof HttpError) return send(res, err.status, { error: err.message });
|
||||||
|
console.error(err);
|
||||||
|
send(res, 500, { error: "Something went wrong in the content service" });
|
||||||
|
});
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
function authenticate(req: IncomingMessage, tokens: Map<string, Role>): Role | null {
|
||||||
|
const m = /^Bearer\s+(.+)$/i.exec(req.headers.authorization ?? "");
|
||||||
|
return m ? (tokens.get((m[1] ?? "").trim()) ?? null) : null;
|
||||||
|
}
|
||||||
|
|
||||||
|
function required(v: string | undefined, name: string): string {
|
||||||
|
if (!v) throw new HttpError(400, `${name} is required`);
|
||||||
|
return v;
|
||||||
|
}
|
||||||
|
|
||||||
|
async function readJson(req: IncomingMessage): Promise<unknown> {
|
||||||
|
let size = 0;
|
||||||
|
const chunks: Buffer[] = [];
|
||||||
|
for await (const chunk of req) {
|
||||||
|
size += (chunk as Buffer).length;
|
||||||
|
if (size > MAX_BODY) throw new HttpError(413, "The request is too large");
|
||||||
|
chunks.push(chunk as Buffer);
|
||||||
|
}
|
||||||
|
const text = Buffer.concat(chunks).toString("utf8");
|
||||||
|
if (!text) return undefined;
|
||||||
|
try {
|
||||||
|
return JSON.parse(text);
|
||||||
|
} catch {
|
||||||
|
throw new HttpError(400, "The body is not valid JSON");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function send(res: ServerResponse, status: number, body: unknown) {
|
||||||
|
res.writeHead(status, { "content-type": "application/json; charset=utf-8", "cache-control": "no-store" });
|
||||||
|
res.end(JSON.stringify(body));
|
||||||
|
}
|
||||||
@@ -0,0 +1,54 @@
|
|||||||
|
/**
|
||||||
|
* Starts the content service: open the store, scan the Site folder, watch it,
|
||||||
|
* and serve the API and the MCP server.
|
||||||
|
*/
|
||||||
|
import { loadConfig } from "./config.ts";
|
||||||
|
import { createHttpServer } from "./http.ts";
|
||||||
|
import { ContentService } from "./service.ts";
|
||||||
|
import { Store } from "./store.ts";
|
||||||
|
import { Vault } from "./vault.ts";
|
||||||
|
import { Watcher } from "./watcher.ts";
|
||||||
|
import { heldView } from "./views.ts";
|
||||||
|
|
||||||
|
const config = loadConfig();
|
||||||
|
const store = new Store(config.dbFile);
|
||||||
|
const service = new ContentService({
|
||||||
|
store,
|
||||||
|
vault: new Vault(config.siteDir),
|
||||||
|
rules: { allowedEmails: config.allowedEmails },
|
||||||
|
onHold: (held) => {
|
||||||
|
console.log(`Held ${held.path}: ${held.reasons.map((r) => r.code).join(", ") || "waiting for review"}`);
|
||||||
|
if (config.notifyWebhookUrl) {
|
||||||
|
fetch(config.notifyWebhookUrl, {
|
||||||
|
method: "POST",
|
||||||
|
headers: { "content-type": "application/json" },
|
||||||
|
body: JSON.stringify({ event: "held", held: heldView(held) }),
|
||||||
|
}).catch((err) => console.error("Notify failed:", err));
|
||||||
|
}
|
||||||
|
},
|
||||||
|
});
|
||||||
|
|
||||||
|
const watcher = new Watcher(service, config.siteDir, {
|
||||||
|
scanMinutes: config.scanMinutes,
|
||||||
|
onScan: (r) => {
|
||||||
|
if (r.changed) console.log(`Scan: ${r.changed} changed, ${r.held} held`);
|
||||||
|
if (config.healthcheckUrl) fetch(config.healthcheckUrl).catch(() => {});
|
||||||
|
},
|
||||||
|
onError: (err) => console.error("Watcher:", err),
|
||||||
|
});
|
||||||
|
watcher.start();
|
||||||
|
|
||||||
|
const server = createHttpServer(service, config);
|
||||||
|
server.listen(config.port, config.host, () => {
|
||||||
|
const c = store.count();
|
||||||
|
console.log(`Content service on ${config.host}:${config.port}: ${c.notes} notes, ${c.published} published, ${c.held} held`);
|
||||||
|
});
|
||||||
|
|
||||||
|
const stop = () => {
|
||||||
|
watcher.stop();
|
||||||
|
server.close();
|
||||||
|
store.close();
|
||||||
|
process.exit(0);
|
||||||
|
};
|
||||||
|
process.on("SIGTERM", stop);
|
||||||
|
process.on("SIGINT", stop);
|
||||||
@@ -0,0 +1,229 @@
|
|||||||
|
/**
|
||||||
|
* The MCP server agents use. It is one more client of the write API: every
|
||||||
|
* tool that changes something calls `ContentService.apply`, so agents meet
|
||||||
|
* the same rules as everyone else, plus the agent rules.
|
||||||
|
*
|
||||||
|
* Agents can find, read and add. They cannot delete, rename, move or write a
|
||||||
|
* raw file, and a draft post always starts unpublished.
|
||||||
|
*/
|
||||||
|
import type { IncomingMessage, ServerResponse } from "node:http";
|
||||||
|
import { McpServer } from "@modelcontextprotocol/sdk/server/mcp.js";
|
||||||
|
import { StreamableHTTPServerTransport } from "@modelcontextprotocol/sdk/server/streamableHttp.js";
|
||||||
|
import { z } from "zod";
|
||||||
|
import { libraryStates, media, noteTypes, topics, typeDef } from "@gk/schema";
|
||||||
|
import type { Change, ContentService, Outcome } from "./service.ts";
|
||||||
|
import type { Author } from "./store.ts";
|
||||||
|
import { heldView, noteView, outcomeView, summaryView } from "./views.ts";
|
||||||
|
|
||||||
|
const json = (value: unknown) => ({ content: [{ type: "text" as const, text: JSON.stringify(value, null, 2) }] });
|
||||||
|
const fail = (message: string) => ({ content: [{ type: "text" as const, text: message }], isError: true });
|
||||||
|
|
||||||
|
/** Turn an outcome into a tool result an agent can act on. */
|
||||||
|
function result(o: Outcome) {
|
||||||
|
if (o.status === "rejected") return fail(`Not saved:\n${o.reasons.map((r) => `- ${r.code}: ${r.message}`).join("\n")}`);
|
||||||
|
if (o.status === "held") {
|
||||||
|
return json({
|
||||||
|
...outcomeView(o),
|
||||||
|
message: "Saved for Greg to review. It is not live yet. Do not retry; the change is waiting.",
|
||||||
|
});
|
||||||
|
}
|
||||||
|
return json(outcomeView(o));
|
||||||
|
}
|
||||||
|
|
||||||
|
const today = () => new Date().toISOString().slice(0, 10);
|
||||||
|
|
||||||
|
export function createMcpServer(service: ContentService, author: Author) {
|
||||||
|
const { store } = service;
|
||||||
|
const server = new McpServer(
|
||||||
|
{ name: "gregkilmer-content", version: "0.1.0" },
|
||||||
|
{
|
||||||
|
instructions:
|
||||||
|
"Content for GregKilmer.com. Search and read freely. Writes go through rules: some are saved straight away, " +
|
||||||
|
"others wait for Greg's review, and a held change should not be retried. Call get_schema before writing a type " +
|
||||||
|
"for the first time. Never invent facts, numbers or dates you were not given.",
|
||||||
|
},
|
||||||
|
);
|
||||||
|
const apply = (change: Change) => result(service.apply(change, author));
|
||||||
|
|
||||||
|
server.registerTool(
|
||||||
|
"search",
|
||||||
|
{
|
||||||
|
title: "Search notes",
|
||||||
|
description: "Full-text search over every note's title and body, drafts included. Optionally limit to one type.",
|
||||||
|
inputSchema: {
|
||||||
|
query: z.string().min(1),
|
||||||
|
type: z.enum(Object.keys(noteTypes) as [string, ...string[]]).optional(),
|
||||||
|
limit: z.number().int().min(1).max(50).optional(),
|
||||||
|
},
|
||||||
|
annotations: { readOnlyHint: true },
|
||||||
|
},
|
||||||
|
async ({ query, type, limit }) =>
|
||||||
|
json(store.search(query, { limit: (limit ?? 10) * (type ? 5 : 1) }).filter((n) => !type || n.type === type).slice(0, limit ?? 10).map(summaryView)),
|
||||||
|
);
|
||||||
|
|
||||||
|
server.registerTool(
|
||||||
|
"get_note",
|
||||||
|
{
|
||||||
|
title: "Read a note",
|
||||||
|
description: "One note by its path (like Library/Books/Mistborn.md) or its name (the file name without .md), with what links to it.",
|
||||||
|
inputSchema: { path: z.string().optional(), name: z.string().optional() },
|
||||||
|
annotations: { readOnlyHint: true },
|
||||||
|
},
|
||||||
|
async ({ path, name }) => {
|
||||||
|
const n = path ? store.get(path) : name ? store.byName(name) : null;
|
||||||
|
if (!n || n.removed) return fail("No such note. Use search to find it.");
|
||||||
|
const pending = store.pendingFor(n.path);
|
||||||
|
return json({
|
||||||
|
...noteView(n),
|
||||||
|
backlinks: store.backlinks(n.name).map(summaryView),
|
||||||
|
pending: pending ? heldView(pending) : null,
|
||||||
|
});
|
||||||
|
},
|
||||||
|
);
|
||||||
|
|
||||||
|
server.registerTool(
|
||||||
|
"get_schema",
|
||||||
|
{
|
||||||
|
title: "Note types",
|
||||||
|
description: "Without a type: every note type and what it is for. With a type: its fields as JSON Schema.",
|
||||||
|
inputSchema: { type: z.enum(Object.keys(noteTypes) as [string, ...string[]]).optional() },
|
||||||
|
annotations: { readOnlyHint: true },
|
||||||
|
},
|
||||||
|
async ({ type }) => {
|
||||||
|
if (!type) return json(Object.values(noteTypes).map((d) => ({ type: d.type, label: d.label, description: d.description })));
|
||||||
|
const def = typeDef(type)!;
|
||||||
|
return json({
|
||||||
|
type: def.type,
|
||||||
|
description: def.description,
|
||||||
|
body: def.body,
|
||||||
|
schema: z.toJSONSchema(def.schema, { io: "input", unrepresentable: "any" }),
|
||||||
|
});
|
||||||
|
},
|
||||||
|
);
|
||||||
|
|
||||||
|
server.registerTool(
|
||||||
|
"list_held",
|
||||||
|
{
|
||||||
|
title: "Changes waiting for review",
|
||||||
|
description: "Changes held for Greg's review, with the reasons. Useful before retrying anything.",
|
||||||
|
inputSchema: {},
|
||||||
|
annotations: { readOnlyHint: true },
|
||||||
|
},
|
||||||
|
async () => json(store.heldList("pending").map((h) => heldView(h))),
|
||||||
|
);
|
||||||
|
|
||||||
|
server.registerTool(
|
||||||
|
"set_status",
|
||||||
|
{
|
||||||
|
title: "Post a status",
|
||||||
|
description:
|
||||||
|
"Add a line to the Now status log, like reading a book or starting a build. kind must be one of the kinds in Now/_kinds. " +
|
||||||
|
"subject links the note it is about, by name.",
|
||||||
|
inputSchema: {
|
||||||
|
kind: z.string().min(1),
|
||||||
|
title: z.string().min(1),
|
||||||
|
note: z.string().optional(),
|
||||||
|
subject: z.string().optional().describe("Name of the note this is about, without brackets"),
|
||||||
|
date: z.string().optional().describe("YYYY-MM-DD, default today"),
|
||||||
|
publish: z.boolean().optional().describe("Default true"),
|
||||||
|
},
|
||||||
|
},
|
||||||
|
async ({ kind, title, note, subject, date, publish }) => {
|
||||||
|
const kinds = store.list({ type: "status-kinds" })[0]?.data.kinds as { label: string }[] | undefined;
|
||||||
|
if (kinds && !kinds.some((k) => k.label.toLowerCase() === kind.toLowerCase())) {
|
||||||
|
return fail(`Unknown kind "${kind}". Kinds: ${kinds.map((k) => k.label).join(", ")}`);
|
||||||
|
}
|
||||||
|
const label = kinds?.find((k) => k.label.toLowerCase() === kind.toLowerCase())?.label ?? kind;
|
||||||
|
return apply({
|
||||||
|
op: "create",
|
||||||
|
data: {
|
||||||
|
type: "status",
|
||||||
|
title,
|
||||||
|
kind: label,
|
||||||
|
date: date ?? today(),
|
||||||
|
...(note ? { note } : {}),
|
||||||
|
...(subject ? { subject: `[[${subject.replace(/^\[\[|\]\]$/g, "")}]]` } : {}),
|
||||||
|
publish: publish ?? true,
|
||||||
|
},
|
||||||
|
});
|
||||||
|
},
|
||||||
|
);
|
||||||
|
|
||||||
|
server.registerTool(
|
||||||
|
"add_library_item",
|
||||||
|
{
|
||||||
|
title: "Add to the Library",
|
||||||
|
description: "Add a book, series, show, film or game. To change one that exists, use update_fields.",
|
||||||
|
inputSchema: {
|
||||||
|
title: z.string().min(1),
|
||||||
|
medium: z.enum(media),
|
||||||
|
state: z.enum(libraryStates),
|
||||||
|
creator: z.string().optional(),
|
||||||
|
group: z.string().optional(),
|
||||||
|
year: z.number().int().optional(),
|
||||||
|
platform: z.string().optional(),
|
||||||
|
verdict: z.string().optional(),
|
||||||
|
note: z.string().optional(),
|
||||||
|
progress: z.string().optional(),
|
||||||
|
anilistId: z.number().int().optional(),
|
||||||
|
publish: z.boolean().optional().describe("Default true"),
|
||||||
|
},
|
||||||
|
},
|
||||||
|
async ({ publish, ...fields }) => {
|
||||||
|
const existing = store.search(fields.title, { limit: 5 }).find((n) => n.type === "library-item" && n.title.toLowerCase() === fields.title.toLowerCase());
|
||||||
|
if (existing) return fail(`Already in the Library at ${existing.path}. Use update_fields to change it.`);
|
||||||
|
const data = Object.fromEntries(Object.entries(fields).filter(([, v]) => v !== undefined));
|
||||||
|
return apply({ op: "create", data: { type: "library-item", ...data, publish: publish ?? true } });
|
||||||
|
},
|
||||||
|
);
|
||||||
|
|
||||||
|
server.registerTool(
|
||||||
|
"update_fields",
|
||||||
|
{
|
||||||
|
title: "Change a note's fields",
|
||||||
|
description:
|
||||||
|
"Change some fields of an existing note; fields not named stay as they are, and null removes one. " +
|
||||||
|
"Give body only to replace the whole body. Greg's own notes need agentEdits: true before an agent may change them.",
|
||||||
|
inputSchema: {
|
||||||
|
path: z.string(),
|
||||||
|
fields: z.record(z.string(), z.unknown()),
|
||||||
|
body: z.string().optional(),
|
||||||
|
},
|
||||||
|
},
|
||||||
|
async ({ path, fields, body }) => apply({ op: "update", path, fields, ...(body !== undefined ? { body } : {}) }),
|
||||||
|
);
|
||||||
|
|
||||||
|
server.registerTool(
|
||||||
|
"draft_post",
|
||||||
|
{
|
||||||
|
title: "Draft a blog post",
|
||||||
|
description: "Start a blog post as an unpublished draft for Greg to finish. It never goes live from here.",
|
||||||
|
inputSchema: {
|
||||||
|
title: z.string().min(1),
|
||||||
|
topic: z.enum(topics),
|
||||||
|
summary: z.string().min(1),
|
||||||
|
body: z.string().min(1),
|
||||||
|
postType: z.enum(["article", "build log", "case study"]).optional(),
|
||||||
|
tags: z.array(z.string()).optional(),
|
||||||
|
},
|
||||||
|
},
|
||||||
|
async ({ body, ...fields }) => {
|
||||||
|
const data = Object.fromEntries(Object.entries(fields).filter(([, v]) => v !== undefined));
|
||||||
|
return apply({ op: "create", data: { type: "post", date: today(), ...data, publish: false }, body });
|
||||||
|
},
|
||||||
|
);
|
||||||
|
|
||||||
|
return server;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** One stateless MCP exchange per request: nothing is kept between calls. */
|
||||||
|
export async function handleMcp(req: IncomingMessage, res: ServerResponse, body: unknown, service: ContentService, author: Author) {
|
||||||
|
const server = createMcpServer(service, author);
|
||||||
|
const transport = new StreamableHTTPServerTransport({ sessionIdGenerator: undefined, enableJsonResponse: true });
|
||||||
|
res.on("close", () => {
|
||||||
|
void transport.close();
|
||||||
|
void server.close();
|
||||||
|
});
|
||||||
|
await server.connect(transport);
|
||||||
|
await transport.handleRequest(req, res, body);
|
||||||
|
}
|
||||||
@@ -0,0 +1,153 @@
|
|||||||
|
/**
|
||||||
|
* The rules every change runs, in a fixed order. Pure functions: the same
|
||||||
|
* change against the same accepted state always gets the same answer, and no
|
||||||
|
* model decides anything.
|
||||||
|
*
|
||||||
|
* The rules live here in code, not in the vault, so no agent can loosen them.
|
||||||
|
*/
|
||||||
|
import { notePath, typeDef, type CheckedNote } from "@gk/schema";
|
||||||
|
import type { Author, HeldReason, StoredNote } from "./store.ts";
|
||||||
|
|
||||||
|
export interface RuleSettings {
|
||||||
|
/** A note with at least this much body text… */
|
||||||
|
textLossMinChars: number;
|
||||||
|
/** …is held when an edit keeps less than this share of it. */
|
||||||
|
textLossKeepRatio: number;
|
||||||
|
/** An agent making more changes than this inside the window is held. */
|
||||||
|
agentBatchLimit: number;
|
||||||
|
agentBatchWindowMinutes: number;
|
||||||
|
/** Note types an agent may publish without review. */
|
||||||
|
agentAutoPublish: string[];
|
||||||
|
/** Email addresses that may appear in notes. */
|
||||||
|
allowedEmails: string[];
|
||||||
|
}
|
||||||
|
|
||||||
|
export const defaultRules: RuleSettings = {
|
||||||
|
textLossMinChars: 400,
|
||||||
|
textLossKeepRatio: 0.5,
|
||||||
|
agentBatchLimit: 10,
|
||||||
|
agentBatchWindowMinutes: 10,
|
||||||
|
agentAutoPublish: ["status", "library-item"],
|
||||||
|
allowedEmails: [],
|
||||||
|
};
|
||||||
|
|
||||||
|
export interface Proposed {
|
||||||
|
path: string;
|
||||||
|
name: string;
|
||||||
|
slug: string;
|
||||||
|
text: string;
|
||||||
|
author: Author;
|
||||||
|
checked: CheckedNote;
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface RuleContext {
|
||||||
|
/** The accepted version at this path, if any. */
|
||||||
|
current: StoredNote | null;
|
||||||
|
/** Another live note with the same file name, if any. */
|
||||||
|
nameTakenBy: string | null;
|
||||||
|
/** Another live note of the same type with the same slug, if any. */
|
||||||
|
slugTakenBy: string | null;
|
||||||
|
/** The agent's changes inside the batch window, before this one. */
|
||||||
|
recentAgentChanges: number;
|
||||||
|
settings: RuleSettings;
|
||||||
|
}
|
||||||
|
|
||||||
|
export type Verdict = { decision: "accept" } | { decision: "hold"; reasons: HeldReason[] };
|
||||||
|
|
||||||
|
/** Strings that look like credentials. Matching is on new text only. */
|
||||||
|
const SECRET_PATTERNS: [string, RegExp][] = [
|
||||||
|
["a private key", /-----BEGIN [A-Z ]*PRIVATE KEY-----/],
|
||||||
|
["an AWS key", /\bAKIA[0-9A-Z]{16}\b/],
|
||||||
|
["a GitHub token", /\b(?:ghp|gho|ghs|ghu|ghr)_[A-Za-z0-9]{30,}\b|\bgithub_pat_[A-Za-z0-9_]{30,}\b/],
|
||||||
|
["a Slack token", /\bxox[abpors]-[A-Za-z0-9-]{10,}\b/],
|
||||||
|
["an API key", /\bsk-[A-Za-z0-9_-]{20,}\b/],
|
||||||
|
["a bearer token", /\bBearer\s+[A-Za-z0-9._~+/-]{20,}=*/],
|
||||||
|
["a password or key value", /\b(?:password|passwd|secret|api[_-]?key|token)\s*[:=]\s*["']?[^\s"']{8,}/i],
|
||||||
|
];
|
||||||
|
const EMAIL = /[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}/g;
|
||||||
|
const PHONE = /(?:\+?1[\s.-]?)?\(?\b\d{3}\)?[\s.-]\d{3}[\s.-]\d{4}\b/g;
|
||||||
|
|
||||||
|
function newMatches(re: RegExp, text: string, before: string): string[] {
|
||||||
|
const flags = re.flags.includes("g") ? re.flags : `${re.flags}g`;
|
||||||
|
const g = new RegExp(re.source, flags);
|
||||||
|
const old = new Set(before.match(g) ?? []);
|
||||||
|
return [...new Set(text.match(g) ?? [])].filter((m) => !old.has(m));
|
||||||
|
}
|
||||||
|
|
||||||
|
const bodyOf = (text: string) => text.replace(/^---\r?\n[\s\S]*?\r?\n---\r?\n?/, "").trim();
|
||||||
|
|
||||||
|
export function evaluate(p: Proposed, ctx: RuleContext): Verdict {
|
||||||
|
const reasons: HeldReason[] = [];
|
||||||
|
const { current, settings } = ctx;
|
||||||
|
const live = current && !current.removed ? current : null;
|
||||||
|
|
||||||
|
// 1. The note must be valid. Nothing else is judged on a broken note.
|
||||||
|
if (!p.checked.ok) {
|
||||||
|
return {
|
||||||
|
decision: "hold",
|
||||||
|
reasons: [
|
||||||
|
{
|
||||||
|
code: "invalid",
|
||||||
|
message: p.checked.problems.map((x) => (x.field ? `${x.field}: ${x.message}` : x.message)).join("; "),
|
||||||
|
},
|
||||||
|
],
|
||||||
|
};
|
||||||
|
}
|
||||||
|
const data = p.checked.data;
|
||||||
|
|
||||||
|
// 2. Settings notes live at one fixed path, so there is only ever one.
|
||||||
|
if (typeDef(p.checked.type)?.singleton && notePath(data) !== p.path) {
|
||||||
|
reasons.push({ code: "misplaced", message: `A ${p.checked.type} note belongs at ${notePath(data)}` });
|
||||||
|
}
|
||||||
|
|
||||||
|
// 3. Names and slugs must stay unambiguous.
|
||||||
|
if (ctx.nameTakenBy) {
|
||||||
|
reasons.push({ code: "duplicate-name", message: `Another note already uses this file name: ${ctx.nameTakenBy}` });
|
||||||
|
}
|
||||||
|
if (ctx.slugTakenBy) {
|
||||||
|
reasons.push({ code: "duplicate-slug", message: `Another ${p.checked.type} already uses the address "${p.slug}": ${ctx.slugTakenBy}` });
|
||||||
|
}
|
||||||
|
|
||||||
|
// 4. A large loss of text in one edit.
|
||||||
|
if (live) {
|
||||||
|
const before = bodyOf(live.text).length;
|
||||||
|
const after = bodyOf(p.text).length;
|
||||||
|
if (before >= settings.textLossMinChars && after < before * settings.textLossKeepRatio) {
|
||||||
|
reasons.push({
|
||||||
|
code: "text-loss",
|
||||||
|
message: `The text shrank from ${before} to ${after} characters in one edit`,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// 5. Anything that looks like a secret, or contact details from an agent.
|
||||||
|
const before = live?.text ?? "";
|
||||||
|
for (const [what, re] of SECRET_PATTERNS) {
|
||||||
|
if (newMatches(re, p.text, before).length) reasons.push({ code: "secret", message: `The note now contains what looks like ${what}` });
|
||||||
|
}
|
||||||
|
if (p.author === "agent") {
|
||||||
|
const allowed = new Set(settings.allowedEmails.map((e) => e.toLowerCase()));
|
||||||
|
const emails = newMatches(EMAIL, p.text, before).filter((e) => !allowed.has(e.toLowerCase()));
|
||||||
|
if (emails.length) reasons.push({ code: "contact", message: `An agent added an email address: ${emails.join(", ")}` });
|
||||||
|
if (newMatches(PHONE, p.text, before).length) reasons.push({ code: "contact", message: "An agent added a phone number" });
|
||||||
|
}
|
||||||
|
|
||||||
|
// 6. Agents stay inside their lane.
|
||||||
|
if (p.author === "agent") {
|
||||||
|
if (live && live.author !== "agent" && live.data.agentEdits !== true) {
|
||||||
|
reasons.push({ code: "agent-edit", message: "An agent changed a note you wrote, which needs your review" });
|
||||||
|
}
|
||||||
|
const publishing = data.publish === true && !(live?.publish ?? false);
|
||||||
|
if (publishing && !settings.agentAutoPublish.includes(p.checked.type)) {
|
||||||
|
reasons.push({ code: "agent-publish", message: `An agent wants to publish a ${p.checked.type}, which needs your approval` });
|
||||||
|
}
|
||||||
|
if (ctx.recentAgentChanges >= settings.agentBatchLimit) {
|
||||||
|
reasons.push({
|
||||||
|
code: "batch",
|
||||||
|
message: `More than ${settings.agentBatchLimit} agent changes in ${settings.agentBatchWindowMinutes} minutes`,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return reasons.length ? { decision: "hold", reasons } : { decision: "accept" };
|
||||||
|
}
|
||||||
@@ -0,0 +1,335 @@
|
|||||||
|
/**
|
||||||
|
* The content service core. Every change, from the vault or from the write
|
||||||
|
* API, goes through `propose`, which runs the rules and either accepts the
|
||||||
|
* change into the store or holds it for review.
|
||||||
|
*
|
||||||
|
* The HTTP API, the MCP server, the admin page and any later editing tool all
|
||||||
|
* call these methods; none of them write notes any other way.
|
||||||
|
*/
|
||||||
|
import { basename } from "node:path";
|
||||||
|
import { checkNote, checkVault, formatNote, notePath, outgoingLinks, slugify, typeDef, type CheckedNote } from "@gk/schema";
|
||||||
|
import { defaultRules, evaluate, type RuleSettings } from "./rules.ts";
|
||||||
|
import type { Author, HeldChange, HeldReason, StoredNote, Store } from "./store.ts";
|
||||||
|
import { hashText, isNotePath, type Vault } from "./vault.ts";
|
||||||
|
|
||||||
|
export type Outcome =
|
||||||
|
| { status: "accepted"; path: string; note: StoredNote }
|
||||||
|
| { status: "held"; path: string; held: HeldChange }
|
||||||
|
| { status: "rejected"; path: string | null; reasons: HeldReason[] }
|
||||||
|
| { status: "unchanged"; path: string };
|
||||||
|
|
||||||
|
export type Change =
|
||||||
|
| { op: "create"; data: Record<string, unknown>; body?: string }
|
||||||
|
| { op: "update"; path: string; fields?: Record<string, unknown>; body?: string };
|
||||||
|
|
||||||
|
export interface ServiceOptions {
|
||||||
|
store: Store;
|
||||||
|
vault: Vault;
|
||||||
|
rules?: Partial<RuleSettings>;
|
||||||
|
/** Injectable clock, for tests. */
|
||||||
|
now?: () => Date;
|
||||||
|
/** Called whenever a change is held, for notifications. */
|
||||||
|
onHold?: (held: HeldChange) => void;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** How long after a note disappears a new note can still claim its old address. */
|
||||||
|
const MOVE_WINDOW_MS = 10 * 60_000;
|
||||||
|
|
||||||
|
export class ContentService {
|
||||||
|
readonly store: Store;
|
||||||
|
readonly vault: Vault;
|
||||||
|
readonly rules: RuleSettings;
|
||||||
|
private readonly now: () => Date;
|
||||||
|
private readonly onHold?: (held: HeldChange) => void;
|
||||||
|
|
||||||
|
constructor(o: ServiceOptions) {
|
||||||
|
this.store = o.store;
|
||||||
|
this.vault = o.vault;
|
||||||
|
this.rules = { ...defaultRules, ...o.rules };
|
||||||
|
this.now = o.now ?? (() => new Date());
|
||||||
|
this.onHold = o.onHold;
|
||||||
|
}
|
||||||
|
|
||||||
|
private iso = () => this.now().toISOString();
|
||||||
|
private today = () => this.iso().slice(0, 10);
|
||||||
|
|
||||||
|
// ------------------------------------------------------------ from the vault
|
||||||
|
|
||||||
|
/** Read one file from the vault and take it in if it changed. */
|
||||||
|
ingest(path: string): Outcome {
|
||||||
|
const text = this.vault.read(path);
|
||||||
|
if (text === null) return this.removed(path);
|
||||||
|
const hash = hashText(text);
|
||||||
|
if (this.store.seenHash(path) === hash) return { status: "unchanged", path };
|
||||||
|
this.store.markSeen(path, hash);
|
||||||
|
const current = this.store.get(path);
|
||||||
|
if (current && !current.removed && current.hash === hash) return { status: "unchanged", path };
|
||||||
|
return this.propose(path, text, "greg", "vault");
|
||||||
|
}
|
||||||
|
|
||||||
|
/** The file is gone. The note leaves the site; its history stays. */
|
||||||
|
removed(path: string): Outcome {
|
||||||
|
this.store.markSeen(path, null);
|
||||||
|
this.store.supersede(path, this.iso());
|
||||||
|
const note = this.store.remove(path, "greg", this.iso());
|
||||||
|
return note ? { status: "accepted", path, note } : { status: "unchanged", path };
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Compare the whole folder with what was seen before. Removals go first, so
|
||||||
|
* a note moved between folders frees its name before it is read again.
|
||||||
|
*/
|
||||||
|
scan(): { changed: number; held: number } {
|
||||||
|
const present = new Set(this.vault.listNotes());
|
||||||
|
let changed = 0;
|
||||||
|
let held = 0;
|
||||||
|
for (const path of this.store.seenPaths()) {
|
||||||
|
if (!present.has(path)) {
|
||||||
|
this.removed(path);
|
||||||
|
changed++;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for (const path of present) {
|
||||||
|
const o = this.ingest(path);
|
||||||
|
if (o.status !== "unchanged") changed++;
|
||||||
|
if (o.status === "held") held++;
|
||||||
|
}
|
||||||
|
return { changed, held };
|
||||||
|
}
|
||||||
|
|
||||||
|
// ------------------------------------------------------------ the write API
|
||||||
|
|
||||||
|
/** One change from the API, made as `author`. Accepted changes are written into the vault. */
|
||||||
|
apply(change: Change, author: Author): Outcome {
|
||||||
|
let data: Record<string, unknown>;
|
||||||
|
let body: string;
|
||||||
|
let path: string;
|
||||||
|
|
||||||
|
if (change.op === "create") {
|
||||||
|
data = { ...change.data, publish: change.data.publish ?? false, created: change.data.created ?? this.today() };
|
||||||
|
body = change.body ?? "";
|
||||||
|
if (!typeDef(String(data.type))) return this.reject(null, "invalid", `Unknown type "${String(data.type)}"`);
|
||||||
|
if (typeof data.title !== "string" || !data.title.trim()) return this.reject(null, "invalid", "title: A title is required");
|
||||||
|
path = notePath(data);
|
||||||
|
if (this.vault.exists(path) || (this.store.get(path) && !this.store.get(path)!.removed)) {
|
||||||
|
return this.reject(path, "exists", `A note already exists at ${path}; update it instead`);
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
path = change.path;
|
||||||
|
if (!isNotePath(path) || path.startsWith("/") || path.split("/").includes("..")) {
|
||||||
|
return this.reject(null, "invalid", "path: Not a note path inside the Site folder");
|
||||||
|
}
|
||||||
|
// Take in any edit made in Obsidian first, so it is never overwritten.
|
||||||
|
this.ingest(path);
|
||||||
|
const current = this.store.get(path);
|
||||||
|
if (!current || current.removed) return this.reject(path, "missing", `No note at ${path}`);
|
||||||
|
if (this.store.pendingFor(path)) {
|
||||||
|
return this.reject(path, "waiting", "This note has a change waiting for review; decide on that first");
|
||||||
|
}
|
||||||
|
data = { ...current.data };
|
||||||
|
for (const [k, v] of Object.entries(change.fields ?? {})) {
|
||||||
|
if (k === "type") return this.reject(path, "invalid", "type: A note's type cannot change");
|
||||||
|
if (v === null) delete data[k];
|
||||||
|
else data[k] = v;
|
||||||
|
}
|
||||||
|
body = change.body ?? current.body;
|
||||||
|
}
|
||||||
|
|
||||||
|
data.updated = this.today();
|
||||||
|
data.updatedBy = author;
|
||||||
|
const text = formatNote(data, body);
|
||||||
|
return this.propose(path, text, author, "api");
|
||||||
|
}
|
||||||
|
|
||||||
|
// ------------------------------------------------------------ review
|
||||||
|
|
||||||
|
acceptHeld(id: number): Outcome {
|
||||||
|
const h = this.store.heldById(id);
|
||||||
|
if (!h || h.status !== "pending") return this.reject(h?.path ?? null, "missing", "Nothing is waiting with that id");
|
||||||
|
const checked = checkNote(h.text);
|
||||||
|
if (!checked.ok) {
|
||||||
|
return this.reject(h.path, "invalid", "This change is not a valid note, so it cannot go live. Fix the note in Obsidian.");
|
||||||
|
}
|
||||||
|
const clash = this.clashes(h.path, checked);
|
||||||
|
if (clash.length) return { status: "rejected", path: h.path, reasons: clash };
|
||||||
|
if (h.source === "api") {
|
||||||
|
const onDisk = this.vault.read(h.path);
|
||||||
|
const current = this.store.get(h.path);
|
||||||
|
if (onDisk !== null && hashText(onDisk) !== (current && !current.removed ? current.hash : null)) {
|
||||||
|
return this.reject(h.path, "conflict", "The note changed in Obsidian since this was proposed");
|
||||||
|
}
|
||||||
|
this.writeToVault(h.path, h.text);
|
||||||
|
}
|
||||||
|
const note = this.acceptText(h.path, h.text, checked, h.author, "accept");
|
||||||
|
this.store.decide(id, "accepted", this.iso());
|
||||||
|
return { status: "accepted", path: h.path, note };
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Turn a held change down. With `restore`, a held edit from Obsidian is
|
||||||
|
* replaced in the vault by the version the site is showing.
|
||||||
|
*/
|
||||||
|
rejectHeld(id: number, opts: { restore?: boolean } = {}): Outcome {
|
||||||
|
const h = this.store.heldById(id);
|
||||||
|
if (!h || h.status !== "pending") return this.reject(h?.path ?? null, "missing", "Nothing is waiting with that id");
|
||||||
|
this.store.decide(id, "rejected", this.iso());
|
||||||
|
const current = this.store.get(h.path);
|
||||||
|
if (opts.restore && h.source === "vault" && current && !current.removed) this.writeToVault(h.path, current.text);
|
||||||
|
return current ? { status: "accepted", path: h.path, note: current } : { status: "unchanged", path: h.path };
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Put an earlier accepted version back, in the vault and on the site. */
|
||||||
|
rollback(path: string, rev: number): Outcome {
|
||||||
|
const r = this.store.revision(path, rev);
|
||||||
|
if (!r) return this.reject(path, "missing", `No revision ${rev} for ${path}`);
|
||||||
|
const checked = checkNote(r.text);
|
||||||
|
if (!checked.ok) return this.reject(path, "invalid", "That version no longer passes the note rules");
|
||||||
|
const clash = this.clashes(path, checked);
|
||||||
|
if (clash.length) return { status: "rejected", path, reasons: clash };
|
||||||
|
this.store.supersede(path, this.iso());
|
||||||
|
this.writeToVault(path, r.text);
|
||||||
|
const note = this.acceptText(path, r.text, checked, "admin", "rollback");
|
||||||
|
return { status: "accepted", path, note };
|
||||||
|
}
|
||||||
|
|
||||||
|
// ------------------------------------------------------------ reading
|
||||||
|
|
||||||
|
/** Problems across the whole folder: links to notes that do not exist, and the like. */
|
||||||
|
problems() {
|
||||||
|
const notes = this.store.list().map((n) => ({ path: n.path, type: n.type, data: n.data, body: n.body }));
|
||||||
|
return checkVault(notes, this.vault.listAttachments());
|
||||||
|
}
|
||||||
|
|
||||||
|
// ------------------------------------------------------------ internals
|
||||||
|
|
||||||
|
private reject(path: string | null, code: string, message: string): Outcome {
|
||||||
|
return { status: "rejected", path, reasons: [{ code, message }] };
|
||||||
|
}
|
||||||
|
|
||||||
|
private slugOf(checked: CheckedNote): string {
|
||||||
|
if (!checked.ok) return "";
|
||||||
|
return String(checked.data.slug ?? slugify(String(checked.data.title)));
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Name and slug clashes with other live notes. */
|
||||||
|
private clashes(path: string, checked: CheckedNote): HeldReason[] {
|
||||||
|
const reasons: HeldReason[] = [];
|
||||||
|
if (!checked.ok) return reasons;
|
||||||
|
const other = this.liveOther(this.store.byName(basename(path, ".md")), path);
|
||||||
|
if (other) reasons.push({ code: "duplicate-name", message: `Another note already uses this file name: ${other}` });
|
||||||
|
const def = typeDef(checked.type);
|
||||||
|
if (!def?.singleton) {
|
||||||
|
const s = this.liveOther(this.store.bySlug(checked.type, this.slugOf(checked)), path);
|
||||||
|
if (s) reasons.push({ code: "duplicate-slug", message: `Another ${checked.type} already uses this address: ${s}` });
|
||||||
|
}
|
||||||
|
return reasons;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Another live note's path, after making sure that note's file still exists. */
|
||||||
|
private liveOther(note: StoredNote | null, path: string): string | null {
|
||||||
|
if (!note || note.path === path) return null;
|
||||||
|
if (!this.vault.exists(note.path)) {
|
||||||
|
// Its file is gone (moved or deleted) and the watcher has not caught up.
|
||||||
|
this.removed(note.path);
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
return note.path;
|
||||||
|
}
|
||||||
|
|
||||||
|
private propose(path: string, text: string, author: Author, source: "vault" | "api"): Outcome {
|
||||||
|
const checked = checkNote(text);
|
||||||
|
const name = basename(path, ".md");
|
||||||
|
const slug = this.slugOf(checked);
|
||||||
|
const window = new Date(this.now().getTime() - this.rules.agentBatchWindowMinutes * 60_000).toISOString();
|
||||||
|
const current = this.store.get(path);
|
||||||
|
const nameTakenBy = checked.ok ? this.liveOther(this.store.byName(name), path) : null;
|
||||||
|
const slugTakenBy =
|
||||||
|
checked.ok && !typeDef(checked.type)?.singleton ? this.liveOther(this.store.bySlug(checked.type, slug), path) : null;
|
||||||
|
|
||||||
|
const verdict = evaluate(
|
||||||
|
{ path, name, slug, text, author, checked },
|
||||||
|
{
|
||||||
|
current,
|
||||||
|
nameTakenBy,
|
||||||
|
slugTakenBy,
|
||||||
|
recentAgentChanges: author === "agent" ? this.store.changesSince("agent", window) : 0,
|
||||||
|
settings: this.rules,
|
||||||
|
},
|
||||||
|
);
|
||||||
|
|
||||||
|
if (verdict.decision === "accept" && checked.ok) {
|
||||||
|
if (source === "api") this.writeToVault(path, text);
|
||||||
|
const note = this.acceptText(path, text, checked, author, "accept");
|
||||||
|
return { status: "accepted", path, note };
|
||||||
|
}
|
||||||
|
if (verdict.decision === "hold" && source === "api" && verdict.reasons.some((r) => r.code === "invalid")) {
|
||||||
|
// An API caller gets the problems back at once instead of a broken note in the queue.
|
||||||
|
return { status: "rejected", path, reasons: verdict.reasons };
|
||||||
|
}
|
||||||
|
const reasons = verdict.decision === "hold" ? verdict.reasons : [];
|
||||||
|
const held = this.store.hold({
|
||||||
|
path,
|
||||||
|
author,
|
||||||
|
source,
|
||||||
|
text,
|
||||||
|
baseHash: current && !current.removed ? current.hash : null,
|
||||||
|
reasons,
|
||||||
|
at: this.iso(),
|
||||||
|
});
|
||||||
|
this.onHold?.(held);
|
||||||
|
return { status: "held", path, held };
|
||||||
|
}
|
||||||
|
|
||||||
|
private writeToVault(path: string, text: string) {
|
||||||
|
// Mark it seen first, so the watcher recognises the service's own write.
|
||||||
|
this.store.markSeen(path, hashText(text));
|
||||||
|
this.vault.write(path, text);
|
||||||
|
}
|
||||||
|
|
||||||
|
private acceptText(path: string, text: string, checked: CheckedNote & { ok: true }, author: Author, action: "accept" | "rollback"): StoredNote {
|
||||||
|
const at = this.iso();
|
||||||
|
const data = checked.data;
|
||||||
|
const slug = this.slugOf(checked);
|
||||||
|
const hash = hashText(text);
|
||||||
|
const vaultNote = { path, type: checked.type, data, body: checked.body };
|
||||||
|
|
||||||
|
return this.store.transaction(() => {
|
||||||
|
const prev = this.store.get(path);
|
||||||
|
// The old address keeps working after a rename or a slug change.
|
||||||
|
if (prev && !prev.removed && prev.type === checked.type && prev.slug !== slug) {
|
||||||
|
this.store.addRedirect(prev.type, prev.slug, path, at);
|
||||||
|
}
|
||||||
|
if (!prev || prev.removed) {
|
||||||
|
const since = new Date(this.now().getTime() - MOVE_WINDOW_MS).toISOString();
|
||||||
|
const moved = this.store
|
||||||
|
.recentlyRemoved(since)
|
||||||
|
.find((n) => n.path !== path && n.type === checked.type && (n.hash === hash || n.title === data.title));
|
||||||
|
if (moved) {
|
||||||
|
this.store.retarget(moved.path, path);
|
||||||
|
if (moved.slug !== slug) this.store.addRedirect(moved.type, moved.slug, path, at);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// A slug in use again belongs to its note, not to a redirect.
|
||||||
|
this.store.dropRedirect(checked.type, slug);
|
||||||
|
this.store.supersede(path, at);
|
||||||
|
return this.store.accept(
|
||||||
|
{
|
||||||
|
path,
|
||||||
|
type: checked.type,
|
||||||
|
name: basename(path, ".md"),
|
||||||
|
slug,
|
||||||
|
title: String(data.title),
|
||||||
|
publish: data.publish === true,
|
||||||
|
data,
|
||||||
|
body: checked.body,
|
||||||
|
text,
|
||||||
|
hash,
|
||||||
|
author,
|
||||||
|
acceptedAt: at,
|
||||||
|
},
|
||||||
|
action,
|
||||||
|
outgoingLinks(vaultNote),
|
||||||
|
);
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,382 @@
|
|||||||
|
/**
|
||||||
|
* The accepted store: the last good version of every note, its history,
|
||||||
|
* changes waiting for review, redirects, links and search.
|
||||||
|
*
|
||||||
|
* Everything the site shows comes from here, never from the vault directly.
|
||||||
|
* The history and review queue cannot be rebuilt from the vault, so this
|
||||||
|
* database carries the backup.dump=sqlite label on Cosmos.
|
||||||
|
*/
|
||||||
|
import { DatabaseSync } from "node:sqlite";
|
||||||
|
import { mkdirSync } from "node:fs";
|
||||||
|
import { dirname } from "node:path";
|
||||||
|
|
||||||
|
export type Author = "greg" | "agent" | "admin";
|
||||||
|
|
||||||
|
export interface StoredNote {
|
||||||
|
path: string;
|
||||||
|
type: string;
|
||||||
|
name: string;
|
||||||
|
slug: string;
|
||||||
|
title: string;
|
||||||
|
publish: boolean;
|
||||||
|
removed: boolean;
|
||||||
|
data: Record<string, unknown>;
|
||||||
|
body: string;
|
||||||
|
text: string;
|
||||||
|
hash: string;
|
||||||
|
author: Author;
|
||||||
|
acceptedAt: string;
|
||||||
|
rev: number;
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface Revision {
|
||||||
|
rev: number;
|
||||||
|
path: string;
|
||||||
|
hash: string;
|
||||||
|
text: string;
|
||||||
|
author: Author;
|
||||||
|
at: string;
|
||||||
|
action: "accept" | "remove" | "rollback" | "restore";
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface HeldReason {
|
||||||
|
code: string;
|
||||||
|
message: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface HeldChange {
|
||||||
|
id: number;
|
||||||
|
path: string;
|
||||||
|
author: Author;
|
||||||
|
source: "vault" | "api";
|
||||||
|
/** The proposed note text. */
|
||||||
|
text: string;
|
||||||
|
/** Hash of the accepted version this change was made against, if any. */
|
||||||
|
baseHash: string | null;
|
||||||
|
reasons: HeldReason[];
|
||||||
|
at: string;
|
||||||
|
status: "pending" | "accepted" | "rejected" | "superseded";
|
||||||
|
decidedAt: string | null;
|
||||||
|
}
|
||||||
|
|
||||||
|
const SCHEMA = `
|
||||||
|
create table if not exists notes (
|
||||||
|
path text primary key,
|
||||||
|
type text not null,
|
||||||
|
name text not null,
|
||||||
|
slug text not null,
|
||||||
|
title text not null,
|
||||||
|
publish integer not null,
|
||||||
|
removed integer not null default 0,
|
||||||
|
data text not null,
|
||||||
|
body text not null,
|
||||||
|
text text not null,
|
||||||
|
hash text not null,
|
||||||
|
author text not null,
|
||||||
|
accepted_at text not null,
|
||||||
|
rev integer not null
|
||||||
|
);
|
||||||
|
create index if not exists notes_type on notes(type, publish, removed);
|
||||||
|
create index if not exists notes_name on notes(name collate nocase);
|
||||||
|
create table if not exists revisions (
|
||||||
|
path text not null,
|
||||||
|
rev integer not null,
|
||||||
|
hash text not null,
|
||||||
|
text text not null,
|
||||||
|
author text not null,
|
||||||
|
at text not null,
|
||||||
|
action text not null,
|
||||||
|
primary key (path, rev)
|
||||||
|
);
|
||||||
|
create table if not exists held (
|
||||||
|
id integer primary key autoincrement,
|
||||||
|
path text not null,
|
||||||
|
author text not null,
|
||||||
|
source text not null,
|
||||||
|
text text not null,
|
||||||
|
base_hash text,
|
||||||
|
reasons text not null,
|
||||||
|
at text not null,
|
||||||
|
status text not null default 'pending',
|
||||||
|
decided_at text
|
||||||
|
);
|
||||||
|
create index if not exists held_status on held(status, path);
|
||||||
|
create table if not exists redirects (
|
||||||
|
type text not null,
|
||||||
|
from_slug text not null,
|
||||||
|
to_path text not null,
|
||||||
|
at text not null,
|
||||||
|
primary key (type, from_slug)
|
||||||
|
);
|
||||||
|
create table if not exists links (
|
||||||
|
from_path text not null,
|
||||||
|
to_name text not null collate nocase,
|
||||||
|
primary key (from_path, to_name)
|
||||||
|
);
|
||||||
|
create table if not exists seen (
|
||||||
|
path text primary key,
|
||||||
|
hash text not null
|
||||||
|
);
|
||||||
|
create virtual table if not exists search using fts5(path unindexed, title, body, tokenize = 'porter unicode61');
|
||||||
|
`;
|
||||||
|
|
||||||
|
type Row = Record<string, any>;
|
||||||
|
|
||||||
|
const toNote = (r: Row): StoredNote => ({
|
||||||
|
path: r.path,
|
||||||
|
type: r.type,
|
||||||
|
name: r.name,
|
||||||
|
slug: r.slug,
|
||||||
|
title: r.title,
|
||||||
|
publish: !!r.publish,
|
||||||
|
removed: !!r.removed,
|
||||||
|
data: JSON.parse(r.data),
|
||||||
|
body: r.body,
|
||||||
|
text: r.text,
|
||||||
|
hash: r.hash,
|
||||||
|
author: r.author,
|
||||||
|
acceptedAt: r.accepted_at,
|
||||||
|
rev: r.rev,
|
||||||
|
});
|
||||||
|
|
||||||
|
const toHeld = (r: Row): HeldChange => ({
|
||||||
|
id: r.id,
|
||||||
|
path: r.path,
|
||||||
|
author: r.author,
|
||||||
|
source: r.source,
|
||||||
|
text: r.text,
|
||||||
|
baseHash: r.base_hash,
|
||||||
|
reasons: JSON.parse(r.reasons),
|
||||||
|
at: r.at,
|
||||||
|
status: r.status,
|
||||||
|
decidedAt: r.decided_at,
|
||||||
|
});
|
||||||
|
|
||||||
|
export class Store {
|
||||||
|
readonly db: DatabaseSync;
|
||||||
|
|
||||||
|
constructor(file: string) {
|
||||||
|
if (file !== ":memory:") mkdirSync(dirname(file), { recursive: true });
|
||||||
|
this.db = new DatabaseSync(file);
|
||||||
|
this.db.exec("pragma journal_mode = wal; pragma foreign_keys = on;");
|
||||||
|
this.db.exec(SCHEMA);
|
||||||
|
}
|
||||||
|
|
||||||
|
close() {
|
||||||
|
this.db.close();
|
||||||
|
}
|
||||||
|
|
||||||
|
transaction<T>(fn: () => T): T {
|
||||||
|
this.db.exec("begin");
|
||||||
|
try {
|
||||||
|
const out = fn();
|
||||||
|
this.db.exec("commit");
|
||||||
|
return out;
|
||||||
|
} catch (e) {
|
||||||
|
this.db.exec("rollback");
|
||||||
|
throw e;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// ------------------------------------------------------------ notes
|
||||||
|
|
||||||
|
get(path: string): StoredNote | null {
|
||||||
|
const r = this.db.prepare("select * from notes where path = ?").get(path) as Row | undefined;
|
||||||
|
return r ? toNote(r) : null;
|
||||||
|
}
|
||||||
|
|
||||||
|
byName(name: string): StoredNote | null {
|
||||||
|
const r = this.db.prepare("select * from notes where name = ? collate nocase and removed = 0").get(name) as Row | undefined;
|
||||||
|
return r ? toNote(r) : null;
|
||||||
|
}
|
||||||
|
|
||||||
|
bySlug(type: string, slug: string): StoredNote | null {
|
||||||
|
const r = this.db.prepare("select * from notes where type = ? and slug = ? and removed = 0").get(type, slug) as Row | undefined;
|
||||||
|
return r ? toNote(r) : null;
|
||||||
|
}
|
||||||
|
|
||||||
|
list(opts: { type?: string; publishedOnly?: boolean; includeRemoved?: boolean; limit?: number; offset?: number } = {}): StoredNote[] {
|
||||||
|
const where: string[] = [];
|
||||||
|
const args: (string | number)[] = [];
|
||||||
|
if (opts.type) {
|
||||||
|
where.push("type = ?");
|
||||||
|
args.push(opts.type);
|
||||||
|
}
|
||||||
|
if (opts.publishedOnly) where.push("publish = 1");
|
||||||
|
if (!opts.includeRemoved) where.push("removed = 0");
|
||||||
|
const sql = `select * from notes ${where.length ? `where ${where.join(" and ")}` : ""} order by path limit ? offset ?`;
|
||||||
|
return (this.db.prepare(sql).all(...args, opts.limit ?? 10_000, opts.offset ?? 0) as Row[]).map(toNote);
|
||||||
|
}
|
||||||
|
|
||||||
|
count(): { notes: number; published: number; held: number } {
|
||||||
|
const c = this.db.prepare("select count(*) n, sum(publish) p from notes where removed = 0").get() as Row;
|
||||||
|
const h = this.db.prepare("select count(*) n from held where status = 'pending'").get() as Row;
|
||||||
|
return { notes: c.n, published: c.p ?? 0, held: h.n };
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Save an accepted version and its revision. */
|
||||||
|
accept(note: Omit<StoredNote, "rev" | "removed">, action: Revision["action"], links: string[]): StoredNote {
|
||||||
|
const prev = this.get(note.path);
|
||||||
|
const rev = (prev?.rev ?? 0) + 1;
|
||||||
|
this.db
|
||||||
|
.prepare(
|
||||||
|
`insert into notes (path, type, name, slug, title, publish, removed, data, body, text, hash, author, accepted_at, rev)
|
||||||
|
values (?, ?, ?, ?, ?, ?, 0, ?, ?, ?, ?, ?, ?, ?)
|
||||||
|
on conflict(path) do update set type = excluded.type, name = excluded.name, slug = excluded.slug,
|
||||||
|
title = excluded.title, publish = excluded.publish, removed = 0, data = excluded.data, body = excluded.body,
|
||||||
|
text = excluded.text, hash = excluded.hash, author = excluded.author, accepted_at = excluded.accepted_at, rev = excluded.rev`,
|
||||||
|
)
|
||||||
|
.run(
|
||||||
|
note.path, note.type, note.name, note.slug, note.title, note.publish ? 1 : 0, JSON.stringify(note.data),
|
||||||
|
note.body, note.text, note.hash, note.author, note.acceptedAt, rev,
|
||||||
|
);
|
||||||
|
this.db
|
||||||
|
.prepare("insert into revisions (path, rev, hash, text, author, at, action) values (?, ?, ?, ?, ?, ?, ?)")
|
||||||
|
.run(note.path, rev, note.hash, note.text, note.author, note.acceptedAt, action);
|
||||||
|
this.db.prepare("delete from links where from_path = ?").run(note.path);
|
||||||
|
const addLink = this.db.prepare("insert or ignore into links (from_path, to_name) values (?, ?)");
|
||||||
|
for (const l of links) addLink.run(note.path, l);
|
||||||
|
this.db.prepare("delete from search where path = ?").run(note.path);
|
||||||
|
this.db.prepare("insert into search (path, title, body) values (?, ?, ?)").run(note.path, note.title, note.body);
|
||||||
|
return this.get(note.path)!;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** The file is gone from the vault: hide the note, keep its history. */
|
||||||
|
remove(path: string, author: Author, at: string): StoredNote | null {
|
||||||
|
const prev = this.get(path);
|
||||||
|
if (!prev || prev.removed) return prev;
|
||||||
|
const rev = prev.rev + 1;
|
||||||
|
this.db.prepare("update notes set removed = 1, rev = ?, accepted_at = ?, author = ? where path = ?").run(rev, at, author, path);
|
||||||
|
this.db
|
||||||
|
.prepare("insert into revisions (path, rev, hash, text, author, at, action) values (?, ?, ?, ?, ?, ?, 'remove')")
|
||||||
|
.run(path, rev, prev.hash, prev.text, author, at);
|
||||||
|
this.db.prepare("delete from links where from_path = ?").run(path);
|
||||||
|
this.db.prepare("delete from search where path = ?").run(path);
|
||||||
|
return this.get(path);
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Notes removed recently, newest first; used to recognise a move. */
|
||||||
|
recentlyRemoved(since: string): StoredNote[] {
|
||||||
|
return (this.db.prepare("select * from notes where removed = 1 and accepted_at >= ? order by accepted_at desc").all(since) as Row[]).map(toNote);
|
||||||
|
}
|
||||||
|
|
||||||
|
history(path: string): Revision[] {
|
||||||
|
return this.db.prepare("select * from revisions where path = ? order by rev desc").all(path) as unknown as Revision[];
|
||||||
|
}
|
||||||
|
|
||||||
|
revision(path: string, rev: number): Revision | null {
|
||||||
|
return (this.db.prepare("select * from revisions where path = ? and rev = ?").get(path, rev) as unknown as Revision) ?? null;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Accepted changes by an author since a time, for the batch guard. */
|
||||||
|
changesSince(author: Author, since: string): number {
|
||||||
|
const r = this.db.prepare("select count(*) n from revisions where author = ? and at >= ?").get(author, since) as Row;
|
||||||
|
const h = this.db.prepare("select count(*) n from held where author = ? and at >= ?").get(author, since) as Row;
|
||||||
|
return r.n + h.n;
|
||||||
|
}
|
||||||
|
|
||||||
|
// ------------------------------------------------------------ links, search, redirects
|
||||||
|
|
||||||
|
backlinks(name: string): StoredNote[] {
|
||||||
|
return (
|
||||||
|
this.db
|
||||||
|
.prepare("select n.* from links l join notes n on n.path = l.from_path where l.to_name = ? collate nocase and n.removed = 0 order by n.path")
|
||||||
|
.all(name) as Row[]
|
||||||
|
).map(toNote);
|
||||||
|
}
|
||||||
|
|
||||||
|
search(query: string, opts: { publishedOnly?: boolean; limit?: number } = {}): StoredNote[] {
|
||||||
|
// Quote each word so punctuation in a query can never become FTS syntax.
|
||||||
|
const fts = query
|
||||||
|
.split(/\s+/)
|
||||||
|
.filter(Boolean)
|
||||||
|
.map((w) => `"${w.replace(/"/g, '""')}"*`)
|
||||||
|
.join(" ");
|
||||||
|
if (!fts) return [];
|
||||||
|
const rows = this.db
|
||||||
|
.prepare(
|
||||||
|
`select n.* from search s join notes n on n.path = s.path
|
||||||
|
where search match ? and n.removed = 0 ${opts.publishedOnly ? "and n.publish = 1" : ""}
|
||||||
|
order by bm25(search, 0, 4, 1) limit ?`,
|
||||||
|
)
|
||||||
|
.all(fts, opts.limit ?? 25) as Row[];
|
||||||
|
return rows.map(toNote);
|
||||||
|
}
|
||||||
|
|
||||||
|
addRedirect(type: string, fromSlug: string, toPath: string, at: string) {
|
||||||
|
this.db
|
||||||
|
.prepare("insert into redirects (type, from_slug, to_path, at) values (?, ?, ?, ?) on conflict(type, from_slug) do update set to_path = excluded.to_path, at = excluded.at")
|
||||||
|
.run(type, fromSlug, toPath, at);
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Point every redirect that led to `fromPath` at `toPath`, so redirects never chain. */
|
||||||
|
retarget(fromPath: string, toPath: string) {
|
||||||
|
this.db.prepare("update redirects set to_path = ? where to_path = ?").run(toPath, fromPath);
|
||||||
|
}
|
||||||
|
|
||||||
|
dropRedirect(type: string, slug: string) {
|
||||||
|
this.db.prepare("delete from redirects where type = ? and from_slug = ?").run(type, slug);
|
||||||
|
}
|
||||||
|
|
||||||
|
redirect(type: string, slug: string): string | null {
|
||||||
|
const r = this.db.prepare("select to_path from redirects where type = ? and from_slug = ?").get(type, slug) as Row | undefined;
|
||||||
|
return (r?.to_path as string) ?? null;
|
||||||
|
}
|
||||||
|
|
||||||
|
redirects(): { type: string; fromSlug: string; toPath: string; at: string }[] {
|
||||||
|
return (this.db.prepare("select * from redirects order by at desc").all() as Row[]).map((r) => ({
|
||||||
|
type: r.type,
|
||||||
|
fromSlug: r.from_slug,
|
||||||
|
toPath: r.to_path,
|
||||||
|
at: r.at,
|
||||||
|
}));
|
||||||
|
}
|
||||||
|
|
||||||
|
// ------------------------------------------------------------ held changes
|
||||||
|
|
||||||
|
hold(h: Omit<HeldChange, "id" | "status" | "decidedAt">): HeldChange {
|
||||||
|
this.supersede(h.path, h.at);
|
||||||
|
const r = this.db
|
||||||
|
.prepare("insert into held (path, author, source, text, base_hash, reasons, at) values (?, ?, ?, ?, ?, ?, ?)")
|
||||||
|
.run(h.path, h.author, h.source, h.text, h.baseHash, JSON.stringify(h.reasons), h.at);
|
||||||
|
return this.heldById(Number(r.lastInsertRowid))!;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** A newer change to the same note replaces whatever was waiting. */
|
||||||
|
supersede(path: string, at: string) {
|
||||||
|
this.db.prepare("update held set status = 'superseded', decided_at = ? where path = ? and status = 'pending'").run(at, path);
|
||||||
|
}
|
||||||
|
|
||||||
|
heldById(id: number): HeldChange | null {
|
||||||
|
const r = this.db.prepare("select * from held where id = ?").get(id) as Row | undefined;
|
||||||
|
return r ? toHeld(r) : null;
|
||||||
|
}
|
||||||
|
|
||||||
|
pendingFor(path: string): HeldChange | null {
|
||||||
|
const r = this.db.prepare("select * from held where path = ? and status = 'pending' order by id desc").get(path) as Row | undefined;
|
||||||
|
return r ? toHeld(r) : null;
|
||||||
|
}
|
||||||
|
|
||||||
|
heldList(status: HeldChange["status"] = "pending"): HeldChange[] {
|
||||||
|
return (this.db.prepare("select * from held where status = ? order by id desc").all(status) as Row[]).map(toHeld);
|
||||||
|
}
|
||||||
|
|
||||||
|
decide(id: number, status: "accepted" | "rejected", at: string) {
|
||||||
|
this.db.prepare("update held set status = ?, decided_at = ? where id = ?").run(status, at, id);
|
||||||
|
}
|
||||||
|
|
||||||
|
// ------------------------------------------------------------ what the watcher has seen
|
||||||
|
|
||||||
|
seenHash(path: string): string | null {
|
||||||
|
return ((this.db.prepare("select hash from seen where path = ?").get(path) as Row | undefined)?.hash as string) ?? null;
|
||||||
|
}
|
||||||
|
|
||||||
|
markSeen(path: string, hash: string | null) {
|
||||||
|
if (hash === null) this.db.prepare("delete from seen where path = ?").run(path);
|
||||||
|
else this.db.prepare("insert into seen (path, hash) values (?, ?) on conflict(path) do update set hash = excluded.hash").run(path, hash);
|
||||||
|
}
|
||||||
|
|
||||||
|
seenPaths(): string[] {
|
||||||
|
return (this.db.prepare("select path from seen").all() as Row[]).map((r) => r.path);
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,95 @@
|
|||||||
|
/**
|
||||||
|
* The Site folder on disk. Only the content service reads or writes it.
|
||||||
|
*/
|
||||||
|
import { createHash } from "node:crypto";
|
||||||
|
import { existsSync, mkdirSync, readFileSync, readdirSync, renameSync, statSync, writeFileSync } from "node:fs";
|
||||||
|
import { basename, dirname, join, relative, sep } from "node:path";
|
||||||
|
|
||||||
|
export const hashText = (text: string) => createHash("sha256").update(text).digest("hex");
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Whether a path under the Site folder can hold a note. Folders starting with
|
||||||
|
* `_` or `.` hold templates, attachments and Obsidian's own files; files
|
||||||
|
* starting with `_` are settings notes and do count. Sync conflict copies
|
||||||
|
* never count, so a conflict can never overwrite a page.
|
||||||
|
*/
|
||||||
|
export function isNotePath(rel: string): boolean {
|
||||||
|
const parts = rel.split(/[\\/]/);
|
||||||
|
const file = parts.pop() ?? "";
|
||||||
|
if (!file.endsWith(".md") || file.startsWith(".")) return false;
|
||||||
|
if (parts.some((p) => p.startsWith("_") || p.startsWith("."))) return false;
|
||||||
|
if (/\(conflicted copy|\.sync-conflict-/i.test(file)) return false;
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
|
||||||
|
export class Vault {
|
||||||
|
constructor(readonly dir: string) {}
|
||||||
|
|
||||||
|
read(rel: string): string | null {
|
||||||
|
try {
|
||||||
|
return readFileSync(join(this.dir, rel), "utf8");
|
||||||
|
} catch {
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
exists(rel: string): boolean {
|
||||||
|
return existsSync(join(this.dir, rel));
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Write through a hidden temp file and a rename, so a reader never sees half a note. */
|
||||||
|
write(rel: string, text: string) {
|
||||||
|
const full = join(this.dir, rel);
|
||||||
|
mkdirSync(dirname(full), { recursive: true });
|
||||||
|
const tmp = join(dirname(full), `.${basename(full)}.${process.pid}.tmp`);
|
||||||
|
writeFileSync(tmp, text);
|
||||||
|
renameSync(tmp, full);
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Every note path under the Site folder, relative, with `/` separators. */
|
||||||
|
listNotes(): string[] {
|
||||||
|
const out: string[] = [];
|
||||||
|
const walk = (d: string) => {
|
||||||
|
for (const e of readdirSync(d, { withFileTypes: true })) {
|
||||||
|
if (e.name.startsWith(".")) continue;
|
||||||
|
const full = join(d, e.name);
|
||||||
|
const rel = relative(this.dir, full).split(sep).join("/");
|
||||||
|
if (e.isDirectory()) {
|
||||||
|
if (!e.name.startsWith("_")) walk(full);
|
||||||
|
} else if (isNotePath(rel)) out.push(rel);
|
||||||
|
}
|
||||||
|
};
|
||||||
|
if (existsSync(this.dir)) walk(this.dir);
|
||||||
|
return out.sort();
|
||||||
|
}
|
||||||
|
|
||||||
|
/** File names in `_attachments`, for checking image links. */
|
||||||
|
listAttachments(): string[] {
|
||||||
|
const out: string[] = [];
|
||||||
|
const walk = (d: string) => {
|
||||||
|
if (!existsSync(d)) return;
|
||||||
|
for (const e of readdirSync(d, { withFileTypes: true })) {
|
||||||
|
if (e.isDirectory()) walk(join(d, e.name));
|
||||||
|
else out.push(e.name);
|
||||||
|
}
|
||||||
|
};
|
||||||
|
walk(join(this.dir, "_attachments"));
|
||||||
|
return out;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Absolute path of an attachment by file name, for serving images. */
|
||||||
|
attachmentPath(name: string): string | null {
|
||||||
|
const walk = (d: string): string | null => {
|
||||||
|
if (!existsSync(d)) return null;
|
||||||
|
for (const e of readdirSync(d, { withFileTypes: true })) {
|
||||||
|
const full = join(d, e.name);
|
||||||
|
if (e.isDirectory()) {
|
||||||
|
const found = walk(full);
|
||||||
|
if (found) return found;
|
||||||
|
} else if (e.name === name && statSync(full).isFile()) return full;
|
||||||
|
}
|
||||||
|
return null;
|
||||||
|
};
|
||||||
|
return walk(join(this.dir, "_attachments"));
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,61 @@
|
|||||||
|
/**
|
||||||
|
* The shapes the HTTP API and the MCP server hand out. Both use these, so a
|
||||||
|
* note looks the same to the site, the admin page and an agent.
|
||||||
|
*/
|
||||||
|
import type { HeldChange, Revision, StoredNote } from "./store.ts";
|
||||||
|
import type { Outcome } from "./service.ts";
|
||||||
|
|
||||||
|
export const noteView = (n: StoredNote, opts: { text?: boolean } = {}) => ({
|
||||||
|
path: n.path,
|
||||||
|
type: n.type,
|
||||||
|
name: n.name,
|
||||||
|
slug: n.slug,
|
||||||
|
title: n.title,
|
||||||
|
publish: n.publish,
|
||||||
|
data: n.data,
|
||||||
|
body: n.body,
|
||||||
|
rev: n.rev,
|
||||||
|
acceptedAt: n.acceptedAt,
|
||||||
|
author: n.author,
|
||||||
|
...(opts.text ? { text: n.text } : {}),
|
||||||
|
});
|
||||||
|
|
||||||
|
export const summaryView = (n: StoredNote) => ({
|
||||||
|
path: n.path,
|
||||||
|
type: n.type,
|
||||||
|
title: n.title,
|
||||||
|
slug: n.slug,
|
||||||
|
publish: n.publish,
|
||||||
|
acceptedAt: n.acceptedAt,
|
||||||
|
});
|
||||||
|
|
||||||
|
export const revisionView = (r: Revision, opts: { text?: boolean } = {}) => ({
|
||||||
|
rev: r.rev,
|
||||||
|
author: r.author,
|
||||||
|
at: r.at,
|
||||||
|
action: r.action,
|
||||||
|
...(opts.text ? { text: r.text } : {}),
|
||||||
|
});
|
||||||
|
|
||||||
|
export const heldView = (h: HeldChange, opts: { text?: boolean } = {}) => ({
|
||||||
|
id: h.id,
|
||||||
|
path: h.path,
|
||||||
|
author: h.author,
|
||||||
|
source: h.source,
|
||||||
|
reasons: h.reasons,
|
||||||
|
at: h.at,
|
||||||
|
status: h.status,
|
||||||
|
decidedAt: h.decidedAt,
|
||||||
|
...(opts.text ? { text: h.text } : {}),
|
||||||
|
});
|
||||||
|
|
||||||
|
export const outcomeView = (o: Outcome) => {
|
||||||
|
switch (o.status) {
|
||||||
|
case "accepted":
|
||||||
|
return { status: o.status, path: o.path, note: summaryView(o.note) };
|
||||||
|
case "held":
|
||||||
|
return { status: o.status, path: o.path, held: heldView(o.held) };
|
||||||
|
default:
|
||||||
|
return o;
|
||||||
|
}
|
||||||
|
};
|
||||||
@@ -0,0 +1,76 @@
|
|||||||
|
/**
|
||||||
|
* Watches the Site folder and hands each changed file to the service after it
|
||||||
|
* has been quiet for a moment, so a sync tool writing a file in pieces is read
|
||||||
|
* once, whole. A full scan at start and on a timer catches anything the
|
||||||
|
* operating system's watch missed.
|
||||||
|
*/
|
||||||
|
import { watch, type FSWatcher } from "node:fs";
|
||||||
|
import { isNotePath } from "./vault.ts";
|
||||||
|
import type { ContentService } from "./service.ts";
|
||||||
|
|
||||||
|
const SCAN = "\0scan";
|
||||||
|
|
||||||
|
export interface WatcherOptions {
|
||||||
|
debounceMs?: number;
|
||||||
|
scanMinutes?: number;
|
||||||
|
/** Called after every full scan. */
|
||||||
|
onScan?: (result: { changed: number; held: number }) => void;
|
||||||
|
onError?: (err: unknown) => void;
|
||||||
|
}
|
||||||
|
|
||||||
|
export class Watcher {
|
||||||
|
private fsw?: FSWatcher;
|
||||||
|
private timer?: NodeJS.Timeout;
|
||||||
|
private readonly pending = new Map<string, NodeJS.Timeout>();
|
||||||
|
|
||||||
|
constructor(
|
||||||
|
private readonly service: ContentService,
|
||||||
|
private readonly dir: string,
|
||||||
|
private readonly o: WatcherOptions = {},
|
||||||
|
) {}
|
||||||
|
|
||||||
|
start() {
|
||||||
|
this.scan();
|
||||||
|
this.fsw = watch(this.dir, { recursive: true }, (_event, file) => {
|
||||||
|
if (!file) return this.scan();
|
||||||
|
const rel = file.toString().split("\\").join("/");
|
||||||
|
if (isNotePath(rel)) this.queue(rel);
|
||||||
|
// A folder renamed or moved reports only the folder, so look at everything.
|
||||||
|
else if (!rel.endsWith(".md") && !rel.split("/").some((p) => p.startsWith(".") || p.startsWith("_"))) this.queue(SCAN);
|
||||||
|
});
|
||||||
|
this.fsw.on("error", (err) => this.o.onError?.(err));
|
||||||
|
const every = (this.o.scanMinutes ?? 5) * 60_000;
|
||||||
|
if (every > 0) this.timer = setInterval(() => this.scan(), every);
|
||||||
|
}
|
||||||
|
|
||||||
|
stop() {
|
||||||
|
this.fsw?.close();
|
||||||
|
clearInterval(this.timer);
|
||||||
|
for (const t of this.pending.values()) clearTimeout(t);
|
||||||
|
this.pending.clear();
|
||||||
|
}
|
||||||
|
|
||||||
|
scan() {
|
||||||
|
try {
|
||||||
|
this.o.onScan?.(this.service.scan());
|
||||||
|
} catch (err) {
|
||||||
|
this.o.onError?.(err);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
private queue(rel: string) {
|
||||||
|
clearTimeout(this.pending.get(rel));
|
||||||
|
this.pending.set(
|
||||||
|
rel,
|
||||||
|
setTimeout(() => {
|
||||||
|
this.pending.delete(rel);
|
||||||
|
if (rel === SCAN) return this.scan();
|
||||||
|
try {
|
||||||
|
this.service.ingest(rel);
|
||||||
|
} catch (err) {
|
||||||
|
this.o.onError?.(err);
|
||||||
|
}
|
||||||
|
}, this.o.debounceMs ?? 2000),
|
||||||
|
);
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,78 @@
|
|||||||
|
import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs";
|
||||||
|
import { tmpdir } from "node:os";
|
||||||
|
import { dirname, join } from "node:path";
|
||||||
|
import { notePath, writeNote } from "@gk/schema";
|
||||||
|
import { ContentService } from "../src/service.ts";
|
||||||
|
import { Store } from "../src/store.ts";
|
||||||
|
import { Vault } from "../src/vault.ts";
|
||||||
|
|
||||||
|
export interface Rig {
|
||||||
|
dir: string;
|
||||||
|
site: string;
|
||||||
|
store: Store;
|
||||||
|
vault: Vault;
|
||||||
|
service: ContentService;
|
||||||
|
clock: { t: number; advance(minutes: number): void };
|
||||||
|
/** Write a note into the vault the way Obsidian would, returning its path. */
|
||||||
|
put(data: Record<string, unknown>, body?: string): string;
|
||||||
|
putText(path: string, text: string): void;
|
||||||
|
done(): void;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function rig(): Rig {
|
||||||
|
const dir = mkdtempSync(join(tmpdir(), "gk-content-"));
|
||||||
|
const site = join(dir, "Site");
|
||||||
|
mkdirSync(site);
|
||||||
|
const store = new Store(join(dir, "content.db"));
|
||||||
|
const vault = new Vault(site);
|
||||||
|
const clock = {
|
||||||
|
t: Date.parse("2026-10-11T12:00:00Z"),
|
||||||
|
advance(minutes: number) {
|
||||||
|
this.t += minutes * 60_000;
|
||||||
|
},
|
||||||
|
};
|
||||||
|
const service = new ContentService({ store, vault, now: () => new Date(clock.t) });
|
||||||
|
const putText = (path: string, text: string) => {
|
||||||
|
mkdirSync(dirname(join(site, path)), { recursive: true });
|
||||||
|
writeFileSync(join(site, path), text);
|
||||||
|
};
|
||||||
|
return {
|
||||||
|
dir,
|
||||||
|
site,
|
||||||
|
store,
|
||||||
|
vault,
|
||||||
|
service,
|
||||||
|
clock,
|
||||||
|
put(data, body = "") {
|
||||||
|
const path = notePath(data);
|
||||||
|
putText(path, writeNote(data, body));
|
||||||
|
return path;
|
||||||
|
},
|
||||||
|
putText,
|
||||||
|
done() {
|
||||||
|
store.close();
|
||||||
|
rmSync(dir, { recursive: true, force: true });
|
||||||
|
},
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
export const book = (title: string, extra: Record<string, unknown> = {}) => ({
|
||||||
|
type: "library-item",
|
||||||
|
title,
|
||||||
|
medium: "book",
|
||||||
|
state: "finished",
|
||||||
|
publish: true,
|
||||||
|
...extra,
|
||||||
|
});
|
||||||
|
|
||||||
|
export const post = (title: string, extra: Record<string, unknown> = {}) => ({
|
||||||
|
type: "post",
|
||||||
|
title,
|
||||||
|
date: "2026-10-01",
|
||||||
|
topic: "Making",
|
||||||
|
summary: "A summary.",
|
||||||
|
publish: true,
|
||||||
|
...extra,
|
||||||
|
});
|
||||||
|
|
||||||
|
export const long = (n = 600) => "Paragraph of real writing. ".repeat(Math.ceil(n / 27)).slice(0, n);
|
||||||
@@ -0,0 +1,96 @@
|
|||||||
|
import type { AddressInfo } from "node:net";
|
||||||
|
import { afterAll, beforeAll, describe, expect, it } from "vitest";
|
||||||
|
import { Client } from "@modelcontextprotocol/sdk/client/index.js";
|
||||||
|
import { StreamableHTTPClientTransport } from "@modelcontextprotocol/sdk/client/streamableHttp.js";
|
||||||
|
import { parseTokens } from "../src/config.ts";
|
||||||
|
import { createHttpServer } from "../src/http.ts";
|
||||||
|
import { book, post, rig, type Rig } from "./helpers.ts";
|
||||||
|
|
||||||
|
const T = { reader: "r".repeat(24), agent: "a".repeat(24), editor: "e".repeat(24), admin: "d".repeat(24) };
|
||||||
|
let r: Rig;
|
||||||
|
let base: string;
|
||||||
|
let server: ReturnType<typeof createHttpServer>;
|
||||||
|
|
||||||
|
beforeAll(async () => {
|
||||||
|
r = rig();
|
||||||
|
r.put({ type: "status-kinds", title: "Status kinds", publish: true, kinds: [{ label: "Reading", icon: "book" }] });
|
||||||
|
r.put(book("Mistborn"), "About [[Elantris]].");
|
||||||
|
r.put(book("Elantris"));
|
||||||
|
r.put(post("Secret draft", { publish: false }));
|
||||||
|
r.service.scan();
|
||||||
|
server = createHttpServer(r.service, { tokens: parseTokens(Object.entries(T).map(([k, v]) => `${k}:${v}`).join(",")) });
|
||||||
|
await new Promise<void>((ok) => server.listen(0, "127.0.0.1", ok));
|
||||||
|
base = `http://127.0.0.1:${(server.address() as AddressInfo).port}`;
|
||||||
|
});
|
||||||
|
afterAll(() => {
|
||||||
|
server.close();
|
||||||
|
r.done();
|
||||||
|
});
|
||||||
|
|
||||||
|
const get = (path: string, token?: string) =>
|
||||||
|
fetch(base + path, { headers: token ? { authorization: `Bearer ${token}` } : {} });
|
||||||
|
const post_ = (path: string, token: string, body: unknown) =>
|
||||||
|
fetch(base + path, { method: "POST", headers: { authorization: `Bearer ${token}`, "content-type": "application/json" }, body: JSON.stringify(body) });
|
||||||
|
|
||||||
|
describe("HTTP API", () => {
|
||||||
|
it("needs a token for everything but health", async () => {
|
||||||
|
expect((await get("/health")).status).toBe(200);
|
||||||
|
expect((await get("/api/notes")).status).toBe(401);
|
||||||
|
expect((await get("/api/notes", "wrong".repeat(6))).status).toBe(401);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("shows readers published notes only", async () => {
|
||||||
|
const reader = (await (await get("/api/notes?type=post", T.reader)).json()) as unknown[];
|
||||||
|
const editor = (await (await get("/api/notes?type=post", T.editor)).json()) as unknown[];
|
||||||
|
expect(reader).toHaveLength(0);
|
||||||
|
expect(editor).toHaveLength(1);
|
||||||
|
expect((await get("/api/note?path=Posts/Secret%20draft.md", T.reader)).status).toBe(404);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("finds a note by slug with its backlinks", async () => {
|
||||||
|
const n = (await (await get("/api/note?type=library-item&slug=elantris", T.reader)).json()) as { title: string; backlinks: { title: string }[] };
|
||||||
|
expect(n.title).toBe("Elantris");
|
||||||
|
expect(n.backlinks.map((b) => b.title)).toEqual(["Mistborn"]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("keeps readers from writing and agents from reviewing", async () => {
|
||||||
|
expect((await post_("/api/changes", T.reader, { op: "create", data: book("X") })).status).toBe(403);
|
||||||
|
expect((await post_("/api/held/1/accept", T.agent, {})).status).toBe(403);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("applies a change and reports rejections plainly", async () => {
|
||||||
|
const ok = await post_("/api/changes", T.editor, { op: "create", data: book("Warbreaker") });
|
||||||
|
expect(ok.status).toBe(200);
|
||||||
|
const bad = await post_("/api/changes", T.editor, { op: "create", data: { type: "library-item", title: "Y" } });
|
||||||
|
expect(bad.status).toBe(422);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("MCP server", () => {
|
||||||
|
it("lets an agent search, post a status and draft a post, but not publish it", async () => {
|
||||||
|
const client = new Client({ name: "test", version: "1.0.0" });
|
||||||
|
await client.connect(
|
||||||
|
new StreamableHTTPClientTransport(new URL(`${base}/mcp`), { requestInit: { headers: { authorization: `Bearer ${T.agent}` } } }),
|
||||||
|
);
|
||||||
|
const tools = (await client.listTools()).tools.map((t) => t.name).sort();
|
||||||
|
expect(tools).toEqual(["add_library_item", "draft_post", "get_note", "get_schema", "list_held", "search", "set_status", "update_fields"]);
|
||||||
|
|
||||||
|
const found = await client.callTool({ name: "search", arguments: { query: "mistborn" } });
|
||||||
|
expect(JSON.stringify(found.content)).toContain("Library/Books/Mistborn.md");
|
||||||
|
|
||||||
|
const status = await client.callTool({ name: "set_status", arguments: { kind: "reading", title: "Mistborn", subject: "Mistborn" } });
|
||||||
|
expect(status.isError).toBeFalsy();
|
||||||
|
const badKind = await client.callTool({ name: "set_status", arguments: { kind: "Juggling", title: "Balls" } });
|
||||||
|
expect(badKind.isError).toBe(true);
|
||||||
|
|
||||||
|
const draft = await client.callTool({
|
||||||
|
name: "draft_post",
|
||||||
|
arguments: { title: "Agent draft", topic: "Making", summary: "S.", body: "Text." },
|
||||||
|
});
|
||||||
|
expect(draft.isError).toBeFalsy();
|
||||||
|
const publish = await client.callTool({ name: "update_fields", arguments: { path: "Posts/Agent draft.md", fields: { publish: true } } });
|
||||||
|
expect(JSON.stringify(publish.content)).toContain("agent-publish");
|
||||||
|
expect(r.store.get("Posts/Agent draft.md")!.publish).toBe(false);
|
||||||
|
await client.close();
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -0,0 +1,35 @@
|
|||||||
|
import { copyFileSync, mkdirSync } from "node:fs";
|
||||||
|
import { dirname, join, resolve } from "node:path";
|
||||||
|
import { afterAll, beforeAll, describe, expect, it } from "vitest";
|
||||||
|
import { build, loadProducts } from "../../../tools/migrate/src/build.ts";
|
||||||
|
import { rig, type Rig } from "./helpers.ts";
|
||||||
|
|
||||||
|
let r: Rig;
|
||||||
|
let total = 0;
|
||||||
|
|
||||||
|
beforeAll(async () => {
|
||||||
|
const legacy = resolve(import.meta.dirname, "../../../legacy");
|
||||||
|
await loadProducts(legacy);
|
||||||
|
const result = build(legacy);
|
||||||
|
r = rig();
|
||||||
|
for (const n of result.notes) r.putText(n.path, n.text);
|
||||||
|
for (const a of result.attachments) {
|
||||||
|
mkdirSync(dirname(join(r.site, a.to)), { recursive: true });
|
||||||
|
copyFileSync(a.from, join(r.site, a.to));
|
||||||
|
}
|
||||||
|
total = result.notes.length;
|
||||||
|
});
|
||||||
|
afterAll(() => r.done());
|
||||||
|
|
||||||
|
describe("the migrated Site folder", () => {
|
||||||
|
it("is taken in whole, with nothing held and no problems", () => {
|
||||||
|
expect(r.service.scan()).toEqual({ changed: total, held: 0 });
|
||||||
|
expect(r.store.count().notes).toBe(total);
|
||||||
|
expect(r.service.problems()).toEqual([]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("is searchable and linked", () => {
|
||||||
|
expect(r.store.search("Spirit Island").some((n) => n.type === "board-game")).toBe(true);
|
||||||
|
expect(r.store.backlinks("Magic The Gathering").length).toBeGreaterThan(0);
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -0,0 +1,188 @@
|
|||||||
|
import { readFileSync, renameSync, rmSync } from "node:fs";
|
||||||
|
import { join } from "node:path";
|
||||||
|
import { afterEach, beforeEach, describe, expect, it } from "vitest";
|
||||||
|
import { evaluate, defaultRules } from "../src/rules.ts";
|
||||||
|
import { book, long, post, rig, type Rig } from "./helpers.ts";
|
||||||
|
|
||||||
|
let r: Rig;
|
||||||
|
beforeEach(() => {
|
||||||
|
r = rig();
|
||||||
|
});
|
||||||
|
afterEach(() => r.done());
|
||||||
|
|
||||||
|
const ok = (o: { status: string }) => expect(o.status).toBe("accepted");
|
||||||
|
|
||||||
|
describe("from the vault", () => {
|
||||||
|
it("takes in valid notes and indexes them for search and links", () => {
|
||||||
|
r.put(book("Mistborn", { creator: "Brandon Sanderson" }), "Allomancy, heists and [[Elantris]].");
|
||||||
|
r.put(book("Elantris"));
|
||||||
|
expect(r.service.scan()).toEqual({ changed: 2, held: 0 });
|
||||||
|
expect(r.store.search("allomancy").map((n) => n.title)).toEqual(["Mistborn"]);
|
||||||
|
expect(r.store.backlinks("Elantris").map((n) => n.title)).toEqual(["Mistborn"]);
|
||||||
|
expect(r.service.scan()).toEqual({ changed: 0, held: 0 });
|
||||||
|
});
|
||||||
|
|
||||||
|
it("holds a broken edit and keeps showing the last good version", () => {
|
||||||
|
const path = r.put(book("Mistborn"));
|
||||||
|
ok(r.service.ingest(path));
|
||||||
|
r.putText(path, "---\ntype: library-item\ntitle: Mistborn\nmedium: scroll\n---\n");
|
||||||
|
const o = r.service.ingest(path);
|
||||||
|
expect(o.status).toBe("held");
|
||||||
|
expect(o.status === "held" && o.held.reasons[0]!.code).toBe("invalid");
|
||||||
|
expect(r.store.get(path)!.data.medium).toBe("book");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("holds a large loss of text until it is accepted", () => {
|
||||||
|
const path = r.put(post("Building a bench"), long());
|
||||||
|
ok(r.service.ingest(path));
|
||||||
|
r.put(post("Building a bench"), "Short now.");
|
||||||
|
const o = r.service.ingest(path);
|
||||||
|
expect(o.status === "held" && o.held.reasons.map((x) => x.code)).toEqual(["text-loss"]);
|
||||||
|
expect(r.store.get(path)!.body.trim()).toBe(long());
|
||||||
|
ok(r.service.acceptHeld(o.status === "held" ? o.held.id : 0));
|
||||||
|
expect(r.store.get(path)!.body.trim()).toBe("Short now.");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("holds a secret pasted into a note", () => {
|
||||||
|
const path = r.put(post("Homelab notes"), "Setup.");
|
||||||
|
ok(r.service.ingest(path));
|
||||||
|
r.put(post("Homelab notes"), "Setup. api_key = abcd1234efgh5678");
|
||||||
|
expect(r.service.ingest(path).status).toBe("held");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("unpublishes a deleted note but keeps its history for a rollback", () => {
|
||||||
|
const path = r.put(book("Mistborn"));
|
||||||
|
ok(r.service.ingest(path));
|
||||||
|
rmSync(join(r.site, path));
|
||||||
|
r.service.scan();
|
||||||
|
expect(r.store.get(path)!.removed).toBe(true);
|
||||||
|
expect(r.store.list({ publishedOnly: true })).toHaveLength(0);
|
||||||
|
ok(r.service.rollback(path, 1));
|
||||||
|
expect(r.store.get(path)!.removed).toBe(false);
|
||||||
|
expect(r.vault.exists(path)).toBe(true);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("keeps old addresses working after a slug change and a move", () => {
|
||||||
|
const path = r.put(post("Bench build", { slug: "bench" }));
|
||||||
|
ok(r.service.ingest(path));
|
||||||
|
r.put(post("Bench build", { slug: "workbench" }));
|
||||||
|
ok(r.service.ingest(path));
|
||||||
|
expect(r.store.redirect("post", "bench")).toBe(path);
|
||||||
|
|
||||||
|
// Renaming the file: the old path goes, the same note appears under a new name.
|
||||||
|
renameSync(join(r.site, path), join(r.site, "Posts/Workbench build.md"));
|
||||||
|
r.service.scan();
|
||||||
|
expect(r.store.redirect("post", "bench")).toBe("Posts/Workbench build.md");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("holds a second note that takes a used address", () => {
|
||||||
|
ok(r.service.ingest(r.put(post("Bench build", { slug: "bench" }))));
|
||||||
|
const o = r.service.ingest(r.put(post("Another bench", { slug: "bench" })));
|
||||||
|
expect(o.status === "held" && o.held.reasons.map((x) => x.code)).toEqual(["duplicate-slug"]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("ignores conflict copies and template folders", () => {
|
||||||
|
r.putText("Posts/Bench (conflicted copy 2026-10-11).md", "junk");
|
||||||
|
r.putText("_templates/Post.md", "junk");
|
||||||
|
expect(r.service.scan()).toEqual({ changed: 0, held: 0 });
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("the write API", () => {
|
||||||
|
it("writes an accepted change into the vault and skips it when the watcher sees it", () => {
|
||||||
|
const o = r.service.apply({ op: "create", data: book("Warbreaker") }, "agent");
|
||||||
|
ok(o);
|
||||||
|
const text = readFileSync(join(r.site, o.path!), "utf8");
|
||||||
|
expect(text).toContain("updatedBy: agent");
|
||||||
|
expect(r.service.ingest(o.path!)).toEqual({ status: "unchanged", path: o.path });
|
||||||
|
});
|
||||||
|
|
||||||
|
it("rejects invalid input at once instead of holding it", () => {
|
||||||
|
const o = r.service.apply({ op: "create", data: { type: "library-item", title: "X", medium: "scroll", state: "now" } }, "agent");
|
||||||
|
expect(o.status).toBe("rejected");
|
||||||
|
expect(r.store.heldList()).toHaveLength(0);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("refuses paths outside the Site folder and type changes", () => {
|
||||||
|
expect(r.service.apply({ op: "update", path: "../secrets.md", fields: {} }, "admin").status).toBe("rejected");
|
||||||
|
const path = r.put(book("Mistborn"));
|
||||||
|
r.service.ingest(path);
|
||||||
|
expect(r.service.apply({ op: "update", path, fields: { type: "post" } }, "admin").status).toBe("rejected");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("takes in an Obsidian edit before applying an API change on top of it", () => {
|
||||||
|
const path = r.put(book("Mistborn"));
|
||||||
|
r.service.ingest(path);
|
||||||
|
r.put(book("Mistborn", { verdict: "Great heist." }));
|
||||||
|
ok(r.service.apply({ op: "update", path, fields: { state: "now" } }, "greg"));
|
||||||
|
expect(r.store.get(path)!.data).toMatchObject({ verdict: "Great heist.", state: "now" });
|
||||||
|
});
|
||||||
|
|
||||||
|
it("holds an agent editing Greg's note, and writes it to the vault only once accepted", () => {
|
||||||
|
const path = r.put(book("Mistborn"));
|
||||||
|
r.service.ingest(path);
|
||||||
|
const before = readFileSync(join(r.site, path), "utf8");
|
||||||
|
const o = r.service.apply({ op: "update", path, fields: { verdict: "Meh." } }, "agent");
|
||||||
|
expect(o.status === "held" && o.held.reasons.map((x) => x.code)).toEqual(["agent-edit"]);
|
||||||
|
expect(readFileSync(join(r.site, path), "utf8")).toBe(before);
|
||||||
|
ok(r.service.acceptHeld(o.status === "held" ? o.held.id : 0));
|
||||||
|
expect(readFileSync(join(r.site, path), "utf8")).toContain("verdict: Meh.");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("lets an agent edit a note marked agentEdits, and its own notes", () => {
|
||||||
|
const path = r.put(book("Mistborn", { agentEdits: true }));
|
||||||
|
r.service.ingest(path);
|
||||||
|
ok(r.service.apply({ op: "update", path, fields: { progress: "2/3" } }, "agent"));
|
||||||
|
const own = r.service.apply({ op: "create", data: book("Warbreaker") }, "agent");
|
||||||
|
ok(r.service.apply({ op: "update", path: own.path!, fields: { state: "now" } }, "agent"));
|
||||||
|
});
|
||||||
|
|
||||||
|
it("holds an agent publishing a post", () => {
|
||||||
|
const o = r.service.apply({ op: "create", data: post("Draft", { publish: false }), body: "Hi." }, "agent");
|
||||||
|
ok(o);
|
||||||
|
const pub = r.service.apply({ op: "update", path: o.path!, fields: { publish: true } }, "agent");
|
||||||
|
expect(pub.status === "held" && pub.held.reasons.map((x) => x.code)).toEqual(["agent-publish"]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("holds an agent adding contact details", () => {
|
||||||
|
const o = r.service.apply({ op: "create", data: post("Draft", { publish: false }), body: "Mail me at someone@example.com" }, "agent");
|
||||||
|
expect(o.status === "held" && o.held.reasons.map((x) => x.code)).toEqual(["contact"]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("holds an agent making too many changes at once", () => {
|
||||||
|
for (let i = 0; i < 10; i++) ok(r.service.apply({ op: "create", data: book(`Book ${i}`) }, "agent"));
|
||||||
|
expect(r.service.apply({ op: "create", data: book("Book 10") }, "agent").status).toBe("held");
|
||||||
|
r.clock.advance(11);
|
||||||
|
ok(r.service.apply({ op: "create", data: book("Book 11") }, "agent"));
|
||||||
|
});
|
||||||
|
|
||||||
|
it("refuses a change while another waits for review", () => {
|
||||||
|
const path = r.put(book("Mistborn"));
|
||||||
|
r.service.ingest(path);
|
||||||
|
r.service.apply({ op: "update", path, fields: { verdict: "Meh." } }, "agent");
|
||||||
|
expect(r.service.apply({ op: "update", path, fields: { verdict: "Good." } }, "greg").status).toBe("rejected");
|
||||||
|
});
|
||||||
|
|
||||||
|
it("rejecting a held vault edit can restore the live version into the vault", () => {
|
||||||
|
const path = r.put(book("Mistborn"));
|
||||||
|
r.service.ingest(path);
|
||||||
|
const good = readFileSync(join(r.site, path), "utf8");
|
||||||
|
r.putText(path, "---\ntype: library-item\n---\n");
|
||||||
|
const o = r.service.ingest(path);
|
||||||
|
r.service.rejectHeld(o.status === "held" ? o.held.id : 0, { restore: true });
|
||||||
|
expect(readFileSync(join(r.site, path), "utf8")).toBe(good);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("rules", () => {
|
||||||
|
it("only flags secrets that are new in this edit", () => {
|
||||||
|
const text = "---\ntype: post\n---\npassword = hunter2hunter2";
|
||||||
|
const checked = { ok: true as const, type: "post", data: { type: "post", title: "x", publish: false }, body: "" };
|
||||||
|
const current = { text, removed: false, author: "greg", data: {}, publish: false } as never;
|
||||||
|
const v = evaluate(
|
||||||
|
{ path: "Posts/x.md", name: "x", slug: "x", text, author: "greg", checked: checked as never },
|
||||||
|
{ current, nameTakenBy: null, slugTakenBy: null, recentAgentChanges: 0, settings: defaultRules },
|
||||||
|
);
|
||||||
|
expect(v.decision).toBe("accept");
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -0,0 +1,4 @@
|
|||||||
|
{
|
||||||
|
"extends": "../../tsconfig.base.json",
|
||||||
|
"include": ["src", "test"]
|
||||||
|
}
|
||||||
+1
-1
@@ -1,5 +1,5 @@
|
|||||||
import { defineConfig } from "vitest/config";
|
import { defineConfig } from "vitest/config";
|
||||||
|
|
||||||
export default defineConfig({
|
export default defineConfig({
|
||||||
test: { include: ["packages/*/test/**/*.test.ts", "tools/*/test/**/*.test.ts"] },
|
test: { include: ["packages/*/test/**/*.test.ts", "services/*/test/**/*.test.ts", "tools/*/test/**/*.test.ts"] },
|
||||||
});
|
});
|
||||||
|
|||||||
Reference in New Issue
Block a user