yjs-local-persistence.md33.4 KBView on GitHub # Local persistence + instant open for Yjs-backed editors
## 1) Introduction — goal, present state, future state
We want Yjs-backed editors (starting with the conversation agenda) to open instantly on the second visit and to keep working when the network is slow or offline, matching the Notion / Google Docs feel. Today the agenda waits ~2s for `documents.getDoc` on every mount, and a separate bug means re-opening a closed agent row paints an empty editor because the long-lived provider hands the new mount a populated Y.Doc and y-prosemirror's `_forceRerender` path silently fails. We will fix the re-open bug first by refcounting providers and tearing them down on the last unmount, then add `y-indexeddb` as a local persistence layer so the Y.Doc is hydrated from IndexedDB before the network round-trip resolves; React Query stops being the content cache (it keeps only metadata) and Yjs's own merge semantics handle reconciliation when the server-side snapshot arrives.
## 2) Present state
### 2.1 Architecture diagram
The Y.Doc is the in-memory hub. The editor and the tRPC provider are **peers** attached to it — neither is "under" the other. They communicate only through the Y.Doc's `update` event.
```text
┌───────────────────────────────────────┐
│ ProseMirror editor (TipTap) │
│ state.doc = { type:'doc', ... } │
└─────────────────┬─────────────────────┘
│
observeDeep ▲ ▼ dispatches keystrokes
(paint) │ │ as Y.Doc edits
│ │
┌───────────────────────────────────────┐
│ Y.Doc │
│ (CRDT, browser memory) │
│ owner: providerRegistry │
└────────────┬──────────────────────────┘
│
ydoc.on('update') fans out to every listener
│
┌───────────────────────┴───────────────────────┐
▼ ▼
┌──────────────────────┐ ┌──────────────────────┐
│ tRPC provider │ │ SSE channel │
│ (CedarYjsProvider) │ │ /doc-events │
│ │ │ │
│ local edits: │ │ inbound only: │
│ debounce → POST │ │ Y.applyUpdate( │
│ files.applyUpdate│ │ ydoc, bytes, │
│ │ │ 'remote') │
│ server-pushed │ │ │
│ updates: ignore │ │ │
└──────────┬───────────┘ └──────────▲───────────┘
│ │
│ POST binary delta broadcast │
▼ │
┌─────────────────────────────────────────────────────────────────┐
│ Postgres (server) │
│ documents row: contentYjs (Yjs blob) + version + metadata │
└─────────────────────────────────────────────────────────────────┘
Separate, orthogonal — does NOT touch the content path:
┌─────────────────────────────────────────────────────────────────┐
│ React Query cache → documents.getDoc(type, path) │
│ │
│ Today: re-fetched on every mount (staleTime: 0, │
│ refetchOnMount: 'always'); ships the full ~243 KB │
│ contentYjs blob every time even though we already │
│ have it in the live Y.Doc. │
└─────────────────────────────────────────────────────────────────┘
```
### 2.2 Step-by-step walkthrough
1. **Component mounts** — `ConversationAgendaDocument` at [ConversationAgendaDocument.tsx:70](apps/mail/modules/agentCanvas/components/ConversationAgendaDocument.tsx).
- Receives: `{ conversationId, aopAgentId }`.
- Fires `useQuery(trpc.documents.getDoc, { documentType: 'conversation_agenda', path })` with `staleTime: 0`, `refetchOnMount: 'always'` at [ConversationAgendaDocument.tsx:78](apps/mail/modules/agentCanvas/components/ConversationAgendaDocument.tsx).
2. **tRPC `documents.getDoc` resolves** — `getDocImpl` at [get-doc.ts:168](apps/server/src/services/document-store/get-doc.ts).
- Reads the row identified by `(documentType, path)`, returns the editor-relevant columns plus FS metadata.
- Data shape returned to the client:
```ts
{
id: string,
content: string | null, // markdown mirror
contentYjs: string | null, // base64 of the Yjs binary blob (~243 KB observed)
version: number,
yjsRevision: number,
title, emoji, parentId, lastEditedBy, updatedAt, ...
}
```
3. **Render-time Y.Doc adoption** — at [ConversationAgendaDocument.tsx:114-122](apps/mail/modules/agentCanvas/components/ConversationAgendaDocument.tsx).
- `ydocRef.current ??= getProvider(documentId)?.ydoc || new Y.Doc()`.
- On first open: provider is absent → fresh empty `Y.Doc`.
- On re-open: provider is still in the registry → returns the populated Y.Doc → triggers Phase B `_forceRerender` in y-prosemirror, which silently fails to paint custom nodes (`agendaTask`, `dateHeading`, `nextStepNode`). **This is the bug.**
4. **TipTap editor created** — `MarkdownEditor` mounts with extensions including `Collaboration.configure({ document: ydoc, field: 'prosemirror' })` at [ConversationAgendaDocument.tsx:256-260](apps/mail/modules/agentCanvas/components/ConversationAgendaDocument.tsx).
- y-prosemirror installs `ySyncPlugin`, calls `yXmlFragment.observeDeep(handler)`.
- If fragment is empty: subsequent `Y.applyUpdate` events stream in as incremental ProseMirror transactions (Phase A — works).
- If fragment is non-empty: `binding._forceRerender(view)` builds the whole doc from Yjs and dispatches one replace transaction (Phase B — broken for our custom-node schema).
5. **Provider effect runs** — at [ConversationAgendaDocument.tsx:125-143](apps/mail/modules/agentCanvas/components/ConversationAgendaDocument.tsx).
- Calls `getOrCreateProvider({ documentId, ydoc, contentYjsBase64 })` at [providerRegistry.ts:52](apps/mail/modules/files/yjs/providerRegistry.ts).
- If the provider exists and has no pending local edits, re-applies the server snapshot via `existing.applyInitialState(...)` at [providerRegistry.ts:58](apps/mail/modules/files/yjs/providerRegistry.ts).
- If new, constructs `CedarYjsProvider` and calls `provider.applyInitialState(stateBytes)` at [providerRegistry.ts:108-110](apps/mail/modules/files/yjs/providerRegistry.ts).
- `applyInitialState` at [CedarYjsProvider.ts:142](apps/mail/modules/files/yjs/CedarYjsProvider.ts) runs `Y.applyUpdate(ydoc, stateBytes, 'server-initial')`.
6. **First paint (cold open)** — y-prosemirror's `observeDeep` handler sees the inserts from step 5 and dispatches ~16 incremental ProseMirror transactions. Editor paints.
- Data after this step (editor's state.doc):
```json
{
"type": "doc",
"content": [
{ "type": "nextStepNode", "attrs": { ... } },
{ "type": "dateHeading", "attrs": { "date": "2026-05-19" } },
{ "type": "agendaTask", "attrs": { "taskId": "..." }, "content": [...] },
...
]
}
```
7. **Local edits** — keystroke fires Collaboration plugin → Y.Doc transaction with origin `null` → `ydoc.on('update', handleLocalUpdate)` at [CedarYjsProvider.ts:138](apps/mail/modules/files/yjs/CedarYjsProvider.ts).
- Origin not in `SERVER_PUSHED_ORIGINS` → `this.dirty = true; scheduleFlush()`.
- 800ms debounce → `flush()` POSTs `Y.encodeStateAsUpdate(ydoc, lastSyncedStateVector)` via `files.applyUpdate` at [files.ts:308](apps/server/src/trpc/routes/files.ts).
- Server response includes `broadcastUpdate`; applied with origin `'server-echo'`. State vector reset.
8. **Remote edits** — SSE channel `/doc-events` subscribed via `useDocEvents(documentId)` at [useDocEvents.ts:16](apps/mail/modules/files/yjs/useDocEvents.ts).
- On `doc:update` event, calls `provider.applyRemote(updateBytes, origin)` at [CedarYjsProvider.ts:158](apps/mail/modules/files/yjs/CedarYjsProvider.ts) → `Y.applyUpdate(ydoc, updateBytes, 'remote'|'agent'|'human'|'system')`.
- y-prosemirror observer streams the changes into the editor.
9. **Unmount** — at [ConversationAgendaDocument.tsx:173-182](apps/mail/modules/agentCanvas/components/ConversationAgendaDocument.tsx).
- Calls `provider.forceFlush()` to drain pending writes.
- **Does not** call `destroyProvider(documentId)`. The provider, its Y.Doc, and the 243 KB of state stay resident in the registry. Re-mount hits step 3's bug path.
## 3) Designed state
### 3.1 Architecture diagram
Same hub-and-peers model, with **IndexedDB added as a fourth peer** of the Y.Doc. All four peers (editor, IDB, tRPC provider, SSE) attach to the same Y.Doc; none of them know about each other; they coordinate purely through `ydoc.on('update')` events with origin tags.
```text
┌───────────────────────────────────────┐
│ ProseMirror editor (TipTap) │
│ binds to a FRESH empty Y.Doc │
│ on every mount │
└─────────────────┬─────────────────────┘
│
observeDeep ▲ ▼ dispatches keystrokes
│ │
│ │
┌───────────────────────────────────────┐
│ Y.Doc │
│ (CRDT, browser memory) │
│ owner: providerRegistry, refcounted │
└─┬──────────────┬──────────────┬───────┘
│ │ │
ydoc.on('update') fans out to every listener
│ │ │
▼ ▼ ▼
┌──────────────────┐ ┌────────────┐ ┌──────────────┐
│ tRPC provider │ │ IndexedDB │ │ SSE channel │
│ │ │ (y-index- │ │ /doc-events │
│ local edits → │ │ eddb) │ │ │
│ POST │ │ │ │ inbound: │
│ │ │ on init: │ │ Y.applyUpd │
│ server-pushed │ │ reads │ │ with origin │
│ → ignore │ │ stored │ │ 'remote'/ │
│ │ │ updates, │ │ 'agent'/... │
│ IDB-origin │ │ applies │ │ │
│ replays → ignore│ │ them │ │ │
│ │ │ │ │ │
│ │ │ on update: │ │ │
│ │ │ writes │ │ │
│ │ │ bytes │ │ │
└────────┬─────────┘ └─────┬──────┘ └──────▲───────┘
│ │ │
│ ▼ │
│ ┌────────────────────┐ │
│ │ IndexedDB (browser)│ │
│ │ cedar-doc-<id> │ │
│ └────────────────────┘ │
│ │
▼ POST binary delta broadcast │
┌─────────────────────────────────────────────────┐
│ Postgres (server) │
│ contentYjs (canonical snapshot) + metadata │
└─────────────────────────────────────────────────┘
Separate, orthogonal:
┌─────────────────────────────────────────────────────────────────┐
│ React Query cache → documents.getDocMeta(type, path) │
│ │
│ Lightweight: title, emoji, version, yjsRevision, updatedAt, │
│ lastEditedBy, parentId. No content. No contentYjs. │
│ │
│ Used by: SaveStatusBadge, FS tree, "last edited by" UI. │
│ staleTime: 30s, refetchOnMount: 'always' — safe to revalidate │
│ often because the payload is tiny. │
└─────────────────────────────────────────────────────────────────┘
```
### 3.2 Step-by-step walkthrough
1. **Component mounts** — `ConversationAgendaDocument` at [ConversationAgendaDocument.tsx:70](apps/mail/modules/agentCanvas/components/ConversationAgendaDocument.tsx).
- Calls `useQuery(trpc.documents.getDoc, …)` with `staleTime: Infinity`, `refetchOnMount: false`. Content freshness comes from SSE; cache is now purely a lookup table.
2. **Render-time Y.Doc creation** — always a **fresh empty `Y.Doc`**.
- We no longer adopt `getProvider(documentId)?.ydoc` during render. The render path is symmetric for first-open and re-open.
- The local `ydocRef` holds this empty Y.Doc for the lifetime of the mount.
3. **Provider acquire** — new `acquireProvider({ documentId, ydoc, contentYjsBase64 })` at [providerRegistry.ts](apps/mail/modules/files/yjs/providerRegistry.ts) (renamed from `getOrCreateProvider`).
- Increments refcount for `documentId`.
- If no provider exists yet: constructs `CedarYjsProvider` with the editor's fresh empty Y.Doc.
- Inside the constructor (new): attaches `new IndexeddbPersistence(\`cedar-doc-\${documentId}\`, ydoc)` and exposes `provider.idbReady: Promise<void>` (resolves on the first IDB `synced` event).
- If a provider already exists (another component is showing the same doc): the editor's fresh Y.Doc is **replaced** with the existing provider's Y.Doc *during the acquire call, before the editor finishes constructing*. (See §3.3 for the synchronisation contract.)
- Returns `{ provider, hadExisting: boolean }`.
4. **Cold open path** (IndexedDB empty, no provider).
- **Frame 0**: editor binds to empty Y.Doc → observer listening.
- **Frame 0**: `IndexeddbPersistence` reads `cedar-doc-<id>` — empty → `synced` event fires with no updates.
- **Frame ~30 (network)**: tRPC response arrives → `provider.applyInitialState(stateBytes)` at [CedarYjsProvider.ts:142](apps/mail/modules/files/yjs/CedarYjsProvider.ts) → `Y.applyUpdate(ydoc, …, 'server-initial')`.
- Editor observer streams in ~16 incremental transactions → first paint.
- In parallel, `Y.applyUpdate` fires `ydoc.on('update')` for both observers: tRPC provider's `handleLocalUpdate` (no-op because origin is `'server-initial'`, in `SERVER_PUSHED_ORIGINS`), and `IndexeddbPersistence`'s update listener — which **does** persist it. Subsequent opens have a populated IDB.
5. **Warm open path** (IndexedDB populated).
- **Frame 0**: editor binds to empty Y.Doc → observer listening.
- **Frame 1-5**: `IndexeddbPersistence` reads all stored updates from IDB and applies them via `Y.applyUpdate(ydoc, update, idbPersistence)` — origin is the persistence instance, not in `SERVER_PUSHED_ORIGINS`. **Edge case: see §3.4 — we add the persistence instance to the ignore set so its replays don't re-flush to the server.**
- Editor observer streams these in as ~16 incremental transactions → **instant paint** (single-digit ms).
- **Frame ~30 (network)**: tRPC response arrives. The Y.Doc is already populated. `provider.applyInitialState(stateBytes)` runs `Y.applyUpdate(ydoc, stateBytes, 'server-initial')`. Yjs CRDT merges: any updates the server has that IDB didn't are applied; any updates IDB had that the server also has are deduped.
- Net effect: if local IDB was up-to-date, this is a no-op. If the server has newer edits (e.g., an agent wrote while the user was away), they appear as incremental inserts via the editor observer. **Phase A on both sides.**
6. **Edit while online** — same as present state §2.2 step 7. Local edit → `handleLocalUpdate` (origin `null`) → debounced flush to `files.applyUpdate`. Simultaneously, the same Y.Doc update event fires `IndexeddbPersistence`'s listener, which persists the binary delta to IDB. No code change required — both listeners are attached to the same `ydoc.on('update')`.
7. **Edit while offline** — local edit → flush throws → `consecutiveFlushFailures` increments → exponential backoff at [CedarYjsProvider.ts:222-235](apps/mail/modules/files/yjs/CedarYjsProvider.ts).
- Crucially, the edit is already in IDB (step 6's fan-out). The user can close the tab and reopen; the edit reloads from IDB on warm open (step 5) and the dirty flag is re-armed because IDB-origin updates need to be re-flushed (see §3.4).
8. **Reconnect after offline edits** — backoff retry succeeds → `Y.encodeStateAsUpdate(ydoc, lastSyncedStateVector)` produces a delta containing all edits the server hasn't acknowledged. POST to `files.applyUpdate`. Server merges via Yjs (CRDT). Response includes `broadcastUpdate` for any concurrent server-side edits → applied with origin `'server-echo'` → editor observer streams them in.
9. **Conflict on reconnect** (user edited offline, agent wrote server-side during the outage).
- Local Y.Doc has user edits; server has agent edits. Both descend from a common state vector.
- On flush, server runs `Y.applyUpdate(serverYdoc, clientDelta)` then computes `broadcastUpdate = Y.encodeStateAsUpdate(serverYdoc, clientStateVector)`. The broadcast contains the agent's edits that the client didn't have.
- Client applies `broadcastUpdate` with origin `'server-echo'` → editor observer paints the merged result. Yjs guarantees the two states converge.
10. **Unmount** — at [ConversationAgendaDocument.tsx:173-182](apps/mail/modules/agentCanvas/components/ConversationAgendaDocument.tsx).
- Calls `provider.forceFlush()` (already happening).
- **New**: calls `releaseProvider(documentId)` in the same cleanup. Refcount decrements.
- If refcount → 0: provider's `forceFlush` runs once more, then `IndexeddbPersistence.destroy()`, then `provider.destroy()`, then the registry entry is removed.
- Re-mount goes through step 1 fresh. Symmetric.
### 3.3 The acquire-and-replace contract (§3 step 3 detail)
This handles the "two components want the same doc at once" edge case. The contract:
- The editor's `Collaboration.configure({ document: ydoc })` must be called with whatever Y.Doc the provider settles on.
- `acquireProvider` is called from `useEffect`, which runs **after** the editor is constructed. So a fresh empty Y.Doc has already been handed to the editor.
- If the provider already exists, `acquireProvider` calls `Y.applyUpdate(editorYDoc, Y.encodeStateAsUpdate(provider.ydoc))` to clone the existing state into the editor's Y.Doc. The editor observer streams it in as incremental inserts (Phase A — paints correctly).
- The provider does **not** swap its Y.Doc reference. The two components now hold two different Y.Doc instances. A small relay (`provider.ydoc.on('update', u => Y.applyUpdate(editorYDoc, u, 'remote'))` and vice-versa with an origin guard) keeps them in sync for the lifetime of the second mount.
**For Phase 1 we don't need this** — agenda is the only consumer. We assert refcount ≤ 1 and throw if it's higher. We design the API so Phase 4 can lift the assertion without breaking callers.
### 3.4 Origin handling for IndexeddbPersistence
`IndexeddbPersistence` uses the persistence instance itself as the transaction origin when it replays stored updates. We must:
- Not treat IDB replays as local edits to flush — otherwise the warm-open path would re-POST the entire document on every open. Add the `IndexeddbPersistence` instance to a per-provider ignore set checked in `handleLocalUpdate` at [CedarYjsProvider.ts:211](apps/mail/modules/files/yjs/CedarYjsProvider.ts).
- Not treat IDB replays as already-synced — they may include edits the user made offline that the server hasn't seen. After IDB replay completes, compare `lastSyncedStateVector` (persisted alongside the updates in IDB) against the doc's state vector; if they differ, set `dirty = true` and schedule a flush.
- Persist `lastSyncedStateVector` to IDB on every successful server flush so the next warm open knows exactly which bytes have been acknowledged.
### 3.5 React Query interaction
After Phase 3, `trpc.documents.getDoc` plays two roles, separated:
| Concern | Source of truth | Cache strategy |
|---|---|---|
| Content (Y.Doc state) | Yjs Y.Doc, mirrored in IDB locally and Postgres remotely | Not React-Query-cached. The tRPC response's `contentYjs` is consumed once on cold open by `applyInitialState`, then ignored. |
| Metadata (title, emoji, version, yjsRevision, updatedAt, lastEditedBy, parentId) | Postgres row | React Query, `staleTime: 30s`, `refetchOnMount: 'always'`. Used by `SaveStatusBadge`, FS tree, `setDocument(buildConversationAgendaFsNode(...))` at [ConversationAgendaDocument.tsx:156-167](apps/mail/modules/agentCanvas/components/ConversationAgendaDocument.tsx). |
Concrete: split the existing `documents.getDoc` consumer into two queries.
- `documents.getDoc({ … })` — kept as-is, but with `staleTime: Infinity` and consumed only as the **first-time** seed for IDB (cold open). After IDB is populated, this query is never re-fetched for that doc.
- `documents.getDocMeta({ … })` — new tRPC query returning everything **except** `content` and `contentYjs`. Cheap. Used by the FS-node mirror effect, save badge, and any UI that displays "last updated by X".
Why not a single query with TanStack `select`? Because `select` doesn't change what's fetched over the wire — we'd still ship 243 KB on every metadata revalidation. Two queries lets the metadata one stay light.
## 4) Implementation phases
### Phase 1 — Refcounted provider lifecycle (fixes re-open bug)
**Goal:** Re-opening a closed editor paints correctly. No new dependencies; no IDB yet.
- [x] Add `refcounts: Map<string, number>` to [providerRegistry.ts](apps/mail/modules/files/yjs/providerRegistry.ts).
- [x] ~~Rename `getOrCreateProvider` → `acquireProvider`.~~ **Deviation:** kept `getOrCreateProvider` working as-is (no refcount) alongside the new `acquireProvider`, because Phase 4 still needs to migrate other consumers (`AgendaDocument`, `components/document.tsx`). Throwing in dev now would break those.
- [x] Add `releaseProvider(documentId: string): Promise<void>` to [providerRegistry.ts](apps/mail/modules/files/yjs/providerRegistry.ts). Decrements refcount; if 0, `await provider.forceFlush()` then `provider.destroy()` then `registry.delete(documentId)`.
- [x] In [ConversationAgendaDocument.tsx](apps/mail/modules/agentCanvas/components/ConversationAgendaDocument.tsx), replace render-time Y.Doc adoption with `ydocRef.current ??= new Y.Doc()`. Drop the `getProvider(documentId)?.ydoc` branch entirely.
- [x] In [ConversationAgendaDocument.tsx](apps/mail/modules/agentCanvas/components/ConversationAgendaDocument.tsx), replace `getOrCreateProvider` call with `acquireProvider`.
- [x] In [ConversationAgendaDocument.tsx](apps/mail/modules/agentCanvas/components/ConversationAgendaDocument.tsx), add `releaseProvider(documentId)` to the cleanup function.
- [x] Update [providerRegistry.ts](apps/mail/modules/files/yjs/providerRegistry.ts) docstring to reflect the new contract.
- [x] Add a dev-only `assertSingleConsumer(documentId)` invoked inside `acquireProvider` that throws if refcount would exceed 1 (kept until Phase 4 lifts the restriction).
**Tests:**
- [x] Unit: new test file `apps/mail/modules/files/yjs/__tests__/providerRegistry.test.ts` covering acquire/release pairing, refcount math, and destroy-on-zero.
- [ ] Integration (manual, recorded in PR): open conversation agenda → collapse Next Steps row → re-expand → verify 16 agenda nodes paint (capture timeline via debugger).
- [x] `pnpm --filter @zero/mail test modules/files/yjs/__tests__/providerRegistry.test.ts`
- [x] `npx tsc --noEmit` (no new errors introduced; pre-existing baseline errors unchanged).
### Phase 2 — Add `y-indexeddb` persistence
**Goal:** Warm opens paint from IndexedDB before the network round-trip resolves. Cold opens behave the same as Phase 1.
- [x] Add `y-indexeddb` to `apps/mail/package.json` dependencies. Run `pnpm install`.
- [x] Extend `CedarYjsProviderOptions` in [CedarYjsProvider.ts](apps/mail/modules/files/yjs/CedarYjsProvider.ts) with `enableIndexedDB?: boolean` (default `true` when `indexedDB` is available).
- [x] In the `CedarYjsProvider` constructor, construct `new IndexeddbPersistence(\`cedar-doc-\${documentId}\`, ydoc)` and store on `this.idb`. Expose `this.idbReady = idb.whenSynced`.
- [x] In `handleLocalUpdate`, add `if (origin === this.idb) return;` before the `SERVER_PUSHED_ORIGINS` check.
- [x] After `idb.whenSynced` resolves, compare the Y.Doc's state vector to the persisted `lastSyncedSV`. If they differ, set `this.dirty = true; this.scheduleFlush();`.
- [x] Persist `lastSyncedStateVector` to IDB after every successful server flush via a `persistLastSyncedSV()` helper (called from `flush`, `applyRemote`, `applyInitialState`, `markAsSynced`).
- [x] Update `destroy()` to call `this.idb?.destroy()` before destroying the Y.Doc.
- [x] In `acquireProvider` / `getOrCreateProvider`, wire `contentYjsBase64` into a `provider.idbReady.then(() => provider.applyInitialState(...))` pattern so the server snapshot is applied *after* IDB has rehydrated.
- [x] **Bonus fix**: `applyInitialState` now sets `lastSyncedStateVector` to the SV implied by the server bytes (not the post-merge SV), so offline edits in IDB get flushed up on reconnect instead of being silently swallowed.
**Tests:**
- [x] Unit: `apps/mail/modules/files/yjs/__tests__/CedarYjsProvider.idb.test.ts` covering: IDB-origin updates don't re-flush; offline edits surface as a flush on hydration; `lastSyncedSV` persists; `applyInitialState` correctness with offline-edit divergence.
- [ ] Manual: open conversation agenda → reload page → verify timeline shows editor paint **before** the `documents.getDoc` tRPC response arrives.
- [ ] Manual offline: DevTools → Network → Offline → type into agenda → close tab → reopen tab while still offline → verify edits are present → restore network → verify flush succeeds and server reflects the edits.
- [x] `pnpm --filter @zero/mail test modules/files/yjs/__tests__/CedarYjsProvider.idb.test.ts`
### Phase 3 — Split `documents.getDoc` into content + metadata queries
**Goal:** Stop re-fetching 243 KB of `contentYjs` on every metadata revalidation. Make React Query a metadata-only cache.
- [x] ~~Add `documents.getDocMeta` route + `getDocMetaImpl`.~~ **Deviation:** added `omitContent?: boolean` flag to the existing `documents.getDoc` route ([documents.ts](apps/server/src/trpc/routes/documents.ts)) and threaded it through [`encodeRow`](apps/server/src/services/document-store/get-doc.ts) instead of duplicating routes. Same wire-byte savings, no duplicated find-or-create + reconcile logic, no separate route to maintain.
- [x] In [ConversationAgendaDocument.tsx](apps/mail/modules/agentCanvas/components/ConversationAgendaDocument.tsx), main React Query call now uses `omitContent: true`, `staleTime: 30_000`, `refetchOnMount: 'always'`. Source for `setDocument(buildConversationAgendaFsNode(...))` and `SaveStatusBadge`.
- [x] Imperative one-shot content fetch gated by `provider.idbReady` + empty state vector — only fires on a genuine cold open.
- [x] Audited other call sites of `documents.getDoc` — left alone (they aren't Yjs-backed editors; daily `AgendaDocument` and `components/document.tsx` are migrated in Phase 4).
**Tests:**
- [ ] Unit (server-side): skipped because the change is a 4-line conditional in `encodeRow`; rely on manual verification. Worth adding once the daily agenda is migrated and the surface grows.
- [ ] Manual: Network tab → confirm warm opens use `omitContent: true` (small payload) and no full content fetch fires.
- [ ] Manual: clear IDB → verify cold open fires one `documents.getDoc` (without `omitContent`) and paints.
- [x] `pnpm --filter @zero/mail test modules/files/yjs/__tests__/` (provider-side tests still green).
### Phase 4 — Extend to other Yjs-backed editors
**Goal:** Apply the same persistence model to the daily `AgendaDocument`, HTML brain docs, and any other `Collaboration`-using editor.
- [x] Audited consumers of `getOrCreateProvider` / `acquireProvider`. Two consumers found: [AgendaDocument.tsx](apps/mail/modules/agentCanvas/components/AgendaDocument.tsx) (daily agenda, multi-day stack) and [components/document.tsx](apps/mail/components/document.tsx) (brain doc editor).
- [x] [AgendaDocument.tsx](apps/mail/modules/agentCanvas/components/AgendaDocument.tsx): fresh empty Y.Doc per documentId (reset on dayKey transition), `acquireProvider` in effect, `releaseProvider` in cleanup. Dropped the `getProvider(documentId)?.ydoc` adoption that triggered y-prosemirror's `_forceRerender` path on multi-day stack collapse/expand.
- [x] [components/document.tsx](apps/mail/components/document.tsx): same pattern, plus the bytes-adoption effect now calls `provider.applyInitialState(...)` directly (gated by `!hasPendingLocalChanges()` + `idbReady`) instead of re-calling `getOrCreateProvider`.
- [x] Removed the legacy `getOrCreateProvider` entry point from [providerRegistry.ts](apps/mail/modules/files/yjs/providerRegistry.ts) and [index.ts](apps/mail/modules/files/yjs/index.ts) — no longer used. Docstring updated.
- [x] Skipped the daily agenda → `omitContent` swap. The daily agenda's `documents.getAgenda` / `getDoc` call lives outside the conversation_agenda surface and has nuances (multi-day stack mass-fetches) that warrant a separate change; Phase 2's IDB layer already makes warm re-opens instant for it, which was the primary user-visible win.
- [ ] **Deferred:** lifting `assertSingleConsumer`. Lifting the guard without the acquire-and-replace relay (§3.3) would just reproduce the populated-Y.Doc paint bug for the second consumer — net negative. Tracked as a follow-up; the guard stays.
**Tests:**
- [ ] Integration multi-consumer test — gated on the relay landing; not meaningful with `assertSingleConsumer` in place.
- [ ] Manual smoke: daily agenda multi-day collapse/expand verifies the dayKey transition (acquire/release pair). Brain HTML doc verifies single-document mount/unmount.
- [x] `pnpm --filter @zero/mail test modules/files/yjs/__tests__/` (14 unit tests, all green after Phase 4 migration).