table-documents.md170.8 KBView on GitHub # Table documents
## 1) Introduction — goal, present state, future state
A table is Cedar's primitive for parallel agent work, and a file type second. Any task with N
independent units — draft an outreach for every customer, pull every closing deal against a field
set, connect with everyone on a list and follow up when they accept — is a table: rows are the
units, an agent is spawned on each one, and it writes back into its own cells as it works. Rows
come from a query rather than from the model transcribing them, and columns can be bound to live
Cedar values so part of the table populates itself. Materializing the work list
*is* the fan-out plan, filling it *is* the execution, and reading it afterwards *is* the audit
trail; nothing else in Cedar makes a parallel run inspectable while it runs. Progress needs no new
machinery — an agent writing `🔍 researching` into its own `status` cell is an ordinary cell write
that rides the existing Y.js delta and SSE path. The rest of this document is what that requires.
We want a `table` file type in Cedar Docs: a spreadsheet whose columns the agent defines at
authoring time, whose cells can hold live Cedar objects (a conversation, a task, a draft) rather
than only text, which streams in row-by-row as the agent fills it, persists as an ordinary file,
opens as a selected artifact beside the chat, and round-trips to Excel. Today Cedar Docs already
has almost all the plumbing — every document is a Y.js-backed ProseMirror tree with a markdown
mirror (`documents.content`), agent writes go markdown → PM JSON → Y.Doc delta → `applyUpdate` →
`docEventBus` broadcast, GFM pipe tables already parse into TipTap `table`/`tableRow`/`tableCell`
nodes ([markdown-parser.ts:151](apps/server/src/services/document-saving/markdown-parser.ts)) and
serialize back out ([serialize.ts:272](apps/server/src/services/document-saving/serialize.ts)), and
inline object tokens already exist in two forms (`{{conversation: id}}` widgets and `[[doc:uuid]]`
file links) — but a markdown table is untyped, has no stable row identity, has no per-column
semantics, and an agent rewriting it re-emits the entire document, so rows cannot stream and
concurrent user edits are lost. This design adds `table` as a ninth `DocumentType` whose content is
a `Y.Array` of per-row `Y.Map`s living beside — not inside — the ProseMirror fragment, with a
schema `Y.Map`, stable `rowId`s, typed columns, and `[[task:]]` / `[[draft:]]` / `[[conversation:]]`
tokens parsed into live chips at render time; adds a
row-granular Y.js mutation path (`writeTableAsYjs`) plus a `table` agent tool exposing a small
spreadsheet verb set (`read` / `add_columns` / `add_rows` / `set` / `clear` / `delete_rows`) whose
bulk arguments are line-oriented rather than JSON arrays — one record per line, so a half-arrived
tool call can be split on `\n` and applied record by record, which is what makes streaming into a
specific table cheap (§3.2 step 5); makes every write address a stable `rowId` so N subagents can
each own one row and fill it concurrently without colliding (§3.2 step 6); renders it as a
virtualized data grid; and adds exceljs-backed export/import that is lossless because the row ids and column schema
travel with the workbook.
## 2) Present state
### 2.1 Architecture diagram
```text
AGENT SERVER CLIENT
┌──────────┐ markdown ┌──────────────────────┐ ┌───────────────────┐
│ write- │ ──────────────────► │ writeDocumentTool │ │ FileEditor │
│ document │ │ (validate + scope) │ │ (CompanyExplorer)│
└──────────┘ └──────────┬───────────┘ └─────────┬─────────┘
│ │
▼ │ documentId
┌──────────────────────┐ ▼
│ writeFileAsYjs │ ┌───────────────────┐
│ getMarkdownParser │ │ <Document/> │
│ md → PM JSON │ │ acquireProvider │
│ ensureNodeIds │ │ getDoc(seed) │
│ DELETE whole frag │ │ useDocEvents(SSE)│
│ + reinsert content │ └─────────┬─────────┘
└──────────┬───────────┘ │
▼ ▼
┌──────────────────────┐ ┌───────────────────┐
│ applyUpdate │ │ CedarYjsProvider │
│ merge into Y.Doc │ │ applyRemote() │
│ dispatchHooks │ │ Collaboration │
│ serialize → content │ │ → TipTap render │
│ persist documents │ └───────────────────┘
└──────────┬───────────┘ ▲
▼ │
┌──────────────────────┐ SSE /doc-events │
│ docEventBus.publish │ ─────────────────────┘
│ + chat docUpdate evt │
└──────────────────────┘
```
### 2.2 Step-by-step walkthrough
1. **Agent write entry** — `execute` at
[writeDocumentTool.ts:149](apps/server/src/mastra/tools/document/writeDocumentTool.ts)
- Receives: `{ path, type, mode, content, description, metadata }` where
`type` is one of `document | html | agenda | conversation_agenda | attachment`
([writeDocumentTool.ts:64](apps/server/src/mastra/tools/document/writeDocumentTool.ts)).
- Resolves the virtual path to (orgId, userId, documentId), creating the row + ancestor folders
when missing, then calls `writeFileAsYjs`.
- Data after this step:
```json
{
"documentId": "b3f1…",
"mode": "upsert",
"content": "| Company | Stage |\n| --- | --- |\n| Acme | Demo |\n"
}
```
2. **Content computation** — `writeFileAsYjs` at
[writeFileAsYjs.ts:51](apps/server/src/services/document-saving/writeFileAsYjs.ts)
- Reads `content`, `content_yjs`, `document_type`, `path` for the row in one query.
- Folds `mode` into a single new markdown string: `upsert` replaces, `append` concatenates with
`\n\n`, `patch` does a `String.replace` of `oldString`.
- Note the failure mode this design must avoid: `append` on a document ending in a pipe table
appends _after_ the table, and `upsert` re-sends every row.
3. **Markdown → ProseMirror JSON** — `getMarkdownParser` at
[markdown.ts:10](apps/server/src/services/document-saving/markdown.ts) → `parseMarkdown` at
[markdown-parser.ts:35](apps/server/src/services/document-saving/markdown-parser.ts)
- Peels leading YAML frontmatter via `extractFrontmatter`
([frontmatter.ts:34](apps/server/src/services/document-saving/frontmatter.ts)) and emits it as
a `codeBlock` with `language: 'frontmatter'`.
- Walks markdown-it tokens. `table_open`/`tr_open`/`th_open`/`td_open` push
`table` / `tableRow` / `tableHeader` / `tableCell`, each cell wrapping its inline content in a
`paragraph` ([markdown-parser.ts:151-188](apps/server/src/services/document-saving/markdown-parser.ts)).
- `appendTextWithWidgets` ([markdown-parser.ts:297](apps/server/src/services/document-saving/markdown-parser.ts))
splits text on `{{score: n}}` / `{{conversation: id}}` / `{{meeting: id}}` into inline atoms.
- Data after this step:
```json
{
"type": "doc",
"content": [
{
"type": "table",
"content": [
{
"type": "tableRow",
"content": [
{
"type": "tableHeader",
"attrs": { "colspan": 1, "rowspan": 1, "colwidth": null },
"content": [
{ "type": "paragraph", "content": [{ "type": "text", "text": "Company" }] }
]
}
]
}
]
}
]
}
```
4. **Node id assignment** — `ensureNodeIds` at
[hydrate.ts:223](apps/server/src/services/document-saving/hydrate.ts)
- Mints a `nodeId` attr on every node lacking one. Ids are fresh per parse — nothing anchors a
row across rewrites except the ad-hoc `taskId`→`nodeId` carry-over
`collectNodeIdsByTaskId` does for agenda tasks
([writeFileAsYjs.ts:261](apps/server/src/services/document-saving/writeFileAsYjs.ts)).
5. **Fragment replacement** — [writeFileAsYjs.ts:182-196](apps/server/src/services/document-saving/writeFileAsYjs.ts)
- Reconstructs the server `Y.Doc` from `content_yjs` (`reconstructYDoc` at
[hydrate.ts:259](apps/server/src/services/document-saving/hydrate.ts)), snapshots its state
vector, then inside `transact(..., 'agent')` **deletes every child of the `prosemirror`
XmlFragment and reinserts the new tree** (`prosemirrorJsonToYDoc` at
[hydrate.ts:106](apps/server/src/services/document-saving/hydrate.ts)).
- Encodes the delta against the pre-transaction state vector (or full state when the row had no
`content_yjs`).
- Data after this step: `Uint8Array` covering N tombstones + N fresh inserts — proportional to
the whole document, not to what changed.
6. **Save pipeline** — `applyUpdate` at
[applyUpdate.ts:58](apps/server/src/services/document-saving/applyUpdate.ts)
- Merges the update into the authoritative `Y.Doc`, runs `dispatchHooks`
([applyUpdate.ts:189](apps/server/src/services/document-saving/applyUpdate.ts)) over
`DOCUMENT_SAVE_HOOKS` ([registry.ts:15](apps/server/src/services/document-saving/registry.ts)),
re-serializes to the markdown mirror via `getMarkdownSerializer`
([serialize.ts:33](apps/server/src/services/document-saving/serialize.ts)), and persists
`content`, `content_json`, `content_yjs`, `yjs_revision`, `version`.
- Data after this step:
```json
{
"documentId": "b3f1…",
"yjsRevision": 42,
"version": 17,
"broadcastUpdate": "<bytes>",
"fullUpdate": "<bytes>",
"hooks": [{ "name": "searchIndex", "ok": true }]
}
```
7. **Broadcast** — `docEventBus.publish` at
[doc-event-bus.ts:41](apps/server/src/services/documents/doc-event-bus.ts), called from
[writeFileAsYjs.ts:232](apps/server/src/services/document-saving/writeFileAsYjs.ts)
- In-process EventEmitter keyed `doc:{documentId}`; the `/doc-events` SSE route
([app.ts:700](apps/server/src/http/app.ts)) fans it out to subscribers.
- The chat stream carries the same payload as a `docUpdate` event because the publishing process
is usually not the process holding the user's SSE connection
([writeDocumentTool.ts:520](apps/server/src/mastra/tools/document/writeDocumentTool.ts)).
8. **Client apply** — `docUpdateResponseProcessor` at
[docUpdateResponseProcessor.ts](apps/mail/modules/cedar-os/src/store/agentConnection/responseProcessors/docUpdateResponseProcessor.ts)
and `useDocEvents` at [useDocEvents.ts:16](apps/mail/modules/documents/yjs/useDocEvents.ts)
- Both funnel into `CedarYjsProvider.applyRemote`
([CedarYjsProvider.ts:327](apps/mail/modules/documents/yjs/CedarYjsProvider.ts)), which applies
the bytes with origin `'agent'` so the `Collaboration` extension's UndoManager captures it.
9. **Render** — `<Document />` at
[document.tsx:145](apps/mail/modules/documents/document.tsx), mounted by `FileEditor` at
[CompanyExplorer.tsx:1058](apps/mail/modules/company/components/CompanyExplorer.tsx)
- Binds a fresh `Y.Doc`, acquires the refcounted provider, hydrates from IndexedDB then seeds
from `documents.getDoc` ([documents.ts:709](apps/server/src/trpc/routes/documents.ts)).
- `FileEditor` branches on `documentType` for `attachment` and `html`
([CompanyExplorer.tsx:1677-1694](apps/mail/modules/company/components/CompanyExplorer.tsx));
every other type falls through to the shared markdown editor, where TipTap's `Table` /
`TableRow` / `TableCell` / `TableHeader` extensions
([markdown-editor.tsx:15-18](apps/mail/components/markdown-editor.tsx)) render the generic
table.
10. **Artifact panel** — `DisplayArtifactPanelInner` at
[DisplayArtifactPanel.tsx:49](apps/mail/modules/home/components/DisplayArtifactPanel.tsx)
- `ContextKind` already includes `file`
([MessageTypes.ts:245](apps/mail/modules/cedar-os/src/store/messages/MessageTypes.ts)), but a
`file` artifact falls to `FileArtifact`
([DisplayArtifactPanel.tsx:196](apps/mail/modules/home/components/DisplayArtifactPanel.tsx)),
which renders `documents.getDoc().content` inside a `<pre>` — raw markdown, not the document.
## 3) Designed state
### 3.1 Architecture diagram
```text
AGENT / CHAT LOOP SERVER CLIENT
┌──────────────────────────┐
│ run-chat-agent-sdk │ ONE RECORD PER LINE, so a half-arrived tool call is usable:
│ activeInputAccum += │ assignments = "r_07.outreach = [[draft: r-3821]]\n
│ delta.partial_json │ r_08.outreach = [[draft: r-38"
│ split('\n') → flush │ ↑ partial line held back
│ COMPLETE lines mid-call│
└────────────┬─────────────┘
│ per-line ops
▼
┌────────────┐ read | create ┌────────────────────────┐ ┌──────────────────────┐
│ table │ add_columns │ tableTool │ │ FileEditor │
│ verb set │ add_rows (pipe rows)│ resolve doc + schema │ │ documentType==='table'│
│ │ set | clear (assigns)│ parseRowLines() / │ └──────────┬───────────┘
│ │ delete_rows │ parseAssignmentLines()│ │
│ │ ────────────────────► │ ordinal → rowId, │ │
└────────────┘ (schema stays JSON) │ guarded by ifRevision │ │
└───────────┬────────────┘ │
┌────────────┐ full markdown │ ▼
│ write- │ (create / rewrite) ▼ ┌──────────────────────┐
│ document │ ────────────────────► ┌────────────────────────┐ │ TableGrid │
│ type=table │ │ writeTableAsYjs │ │ (TanStack table + │
└────────────┘ │ mutate ONLY the │ │ virtual, no TipTap)│
│ tableRows Y.Array: │ │ ├ header ← Y.Map │
┌──────────────────────┐ │ insert rows / set │ │ ├ virtualized rows │
│ tableMarkdown │ │ cells by rowId │ │ │ ← Y.Array │
│ parse: fm+pipe → │◄──────►│ → tiny Y delta │ │ └ CellRefChip │
│ rows │ └───────────┬────────────┘ └──────────┬───────────┘
│ serial: rows → │ ▼ │
│ fm+pipe │ ┌────────────────────────┐ │
└──────────────────────┘ │ applyUpdate │ │
(same pipe syntax the │ + tableStatsHook │ │
agent writes — no └───────────┬────────────┘ │
translation layer) ▼ │
┌────────────────────────┐ SSE / chat │
│ docEventBus.publish │ ──────────────────┘
└────────────────────────┘ row appears live
EXCEL ┌────────────────────────┐
┌──────────┐ documents.exportTable │ tableToWorkbook │ sheet "Data" + hidden _id col
│ .xlsx │ ◄──────────────────────│ (exceljs) │ sheet "_cedar" holds schema JSON
└────┬─────┘ └────────────────────────┘
│ upload → attachment row
▼ ┌────────────────────────┐
documents.importTable ────────────► │ workbookToTableMarkdown│ ──► table.create
└────────────────────────┘
```
### 3.2 Step-by-step walkthrough
1. **Type registration** — `DOCUMENT_TYPE` at
[document-types.ts:49](apps/server/src/services/documents/document-types.ts)
- Adds `TABLE: 'table'`. No migration: `documents.document_type` is a plain `text` column with
TypeScript-only enforcement ([documents-schema.ts:70](apps/server/src/db/documents-schema.ts)).
- `resolveDocType` ([doc-type-registry.ts:429](apps/server/src/services/documents/doc-type-registry.ts))
already preserves any value present in `DOCUMENT_TYPE` as-is and treats it as org-scoped with
a generic path parser, so `table` docs are addressable at any virtual path with no registry
entry. No `seedFn` / `reconcile` — a table's content is authored, never derived.
2. **Markdown mirror parse** — `parseTableMarkdown` in
`apps/server/src/services/documents/table/table-markdown.ts` (new), wired into
`getMarkdownParser` ([markdown.ts:10](apps/server/src/services/document-saving/markdown.ts))
- Input is YAML frontmatter (the column schema) followed by one GFM pipe table whose first
column is the reserved, undeclared `_id`.
- Header labels are matched positionally against `schema.columns`; the `_id` column is stripped
into `TableRow.rowId`. Cell values are stored verbatim — `[[type: id]]` tokens
(`CEDAR_DOC_REFERENCE_REGEX` at
[document-types.ts:113](apps/server/src/services/documents/document-types.ts)) stay as text in
the `Y.Map` and are parsed into chips by the grid at render time, so the parser has no inline
node model to maintain.
- Input:
```markdown
---
table:
version: 1
titleColumn: company
columns:
- { key=[redacted], label: Company, type: text }
- { key=[redacted], label: Deal, type: conversation }
- { key=[redacted], label: Outreach, type: draft }
---
| \_id | Company | Deal | Outreach |
| ---- | ------- | ----------------------- | ----------------- |
| r_01 | Acme | [[conversation: 9f2e…]] | [[draft: r-3821]] |
```
- Data after this step:
```json
{
"schema": { "version": 1, "titleColumn": "company", "columns": ["…"] },
"rows": [
{
"rowId": "r_01",
"cells": {
"company": "Acme",
"deal": "[[conversation: 9f2e…]]",
"outreach": "[[draft: r-3821]]"
}
}
]
}
```
3. **Markdown mirror serialize** — `serializeTableToMarkdown` in the same module, wired into
`getMarkdownSerializer` ([serialize.ts:33](apps/server/src/services/document-saving/serialize.ts))
- Emits the frontmatter from the schema `Y.Map`, then a pipe table with `_id` first. Cell values
pass through verbatim — because refs are stored as text, serialization is a projection of the
row rather than a tree walk, and there is no inline-node round trip to get wrong.
- The mirror is what `grep-documents` and every agent read, so a table stays greppable as text.
4. **Row-granular Y.js mutation** — `writeTableAsYjs` in
`apps/server/src/services/document-saving/writeTableAsYjs.ts` (new)
- Receives `{ documentId, ops, scope, actorLabel }`. Reconstructs the server `Y.Doc`
(`reconstructYDoc`), reads `ydoc.getMap('tableSchema')` and `ydoc.getArray('tableRows')`, and
inside `transact(..., 'agent')` applies each op **without touching any other row**:
- `add_columns` — update the schema map; set an empty value on each row `Y.Map` for added keys.
- `add_rows` — `push` new row `Y.Map`s onto the array.
- `set` / `clear` — find the row `Y.Map` by its `_id`, then a single `Y.Map.set(columnKey, …)`.
- `delete_rows` — `Y.Array.delete` at the matched indices.
- Encodes the delta against the pre-transaction state vector and hands off to `applyUpdate`,
exactly as `writeFileAsYjs` does at
[writeFileAsYjs.ts:215](apps/server/src/services/document-saving/writeFileAsYjs.ts), then
publishes on `docEventBus`.
- This is the load-bearing difference from `writeFileAsYjs`, which deletes and reinserts the
whole fragment ([writeFileAsYjs.ts:184](apps/server/src/services/document-saving/writeFileAsYjs.ts)):
that would destroy every `rowId`'s CRDT identity and clobber concurrent user cell edits. A
`table` document must never route a partial write through `writeFileAsYjs`.
- Data after a 3-row append:
```json
{ "updateByteLength": 412, "rowsAppended": 3, "yjsRevision": 43 }
```
5. **Agent tool** — `tableTool` in
`apps/server/src/mastra/tools/document/tableTool.ts` (new)
- Input is a `TableVerb` (§3.3) plus the target `path`. `create` provisions the document row and
the initial schema; every other verb resolves the document, then goes through the
`TableBackend` for that table — `CedarTableBackend` calls `writeTableAsYjs`.
- **The bulk payload is line-oriented text, not a JSON array.** `rows` and `cells` are single
multi-line strings, one record per line. `schema` stays a JSON object. The split is deliberate,
and it is what makes streaming into a specific table cheap:
- Cedar's chat loop already accumulates tool arguments as they arrive —
`activeInputAccum += evt.delta.partial_json`
([run-chat-agent-sdk.ts:267](apps/server/src/mastra/workflows/chat/run-chat-agent-sdk.ts))
— but only calls `JSON.parse` at `content_block_stop`
([run-chat-agent-sdk.ts:277](apps/server/src/mastra/workflows/chat/run-chat-agent-sdk.ts)),
because a JSON array is syntactically invalid at every intermediate state. Streaming rows out
of a JSON payload therefore requires a tolerant/partial-JSON parser.
- With one record per line, mid-call streaming is `accum.split('\n')` and emit every complete
line except the last. No new dependency, no partial-JSON parser, no schema-aware incremental
decoder.
- The schema is small, nested, and written once. None of the above applies to it, so JSON is
the right tool there and stays.
- **The tool is a small verb set, not a document-write tool with modes.** `read`, `add_columns`,
`add_rows`, `set`, `clear`, `delete_rows`. This is the SheetCopilot result: exposing atomic
spreadsheet actions scored 44.3% Pass@1 against 16.3% for asking the model to emit VBA that
produces the same edit. A model handed `set` behaves like it is calling an API it already knows
(Sheets, Airtable, openpyxl); a model handed `write-document(mode: patch)` has to reason about
text surgery on a serialized table first. The verbs are the abstraction the model is fluent in.
- **`set` is the workhorse and the reason random access matters.** One assignment per line,
addressed by row then column:
```text
r_07.outreach = [[draft: r-3821]]
r_08.outreach = [[draft: r-3822]]
r_08.status = Sent
```
"Draft an outreach for each of them" fills one column down many rows; "find why they bought"
fills one cell per conversation. Both are sparse writes into an existing grid, and neither is
row-shaped. Forcing them through a whole-row rewrite makes the agent restate cells it is not
changing — the exact laziness/verbosity failure the edit-format literature documents.
- **Rows are addressed by `rowId`, not by ordinal.** `fill(3,4) = value` is the natural way to
write this and it is the one part that will bite: row 3 is a position, not an identity. A user
sorting, filtering, or inserting a row between the agent's read and its write — routine in a
multiplayer Y.js document — silently redirects the write into a different company's row. The
`rowId` in the `_id` column is a durable handle on the entity, so `r_07.outreach` means the same
thing forever.
- **Ordinals are still accepted, under a revision guard.** `set` also takes `4.outreach` (1-based
data row) and A1-style `D4`, resolved to a `rowId` at parse time. Any call using an ordinal must
carry `ifRevision`, checked against the row's existing `documents.yjs_revision`
([documents-schema.ts:109](apps/server/src/db/documents-schema.ts)) — which `read` returns and
`applyUpdate` already advances on every save. A stale revision rejects the whole call and
returns the current one plus the row ids, so the agent retries against truth. Ergonomics of
coordinates, none of the silent-misfire risk.
- **The grid is unbounded from the agent's side: `set` upserts.** Writing `acme.company = Acme`
to a rowId that does not exist materializes the row, the same way writing to `A1000` in a
spreadsheet just works. There is no mandatory `add_rows` step and no "row does not exist"
error to reason about — the agent picks a meaningful id (`acme`, `globex`) and writes. What is
*not* done is preallocating rows: storage stays sparse, so the Y.Doc and the markdown mirror
only ever hold rows that have content. An Excel-style preallocated grid would bloat both for
nothing, and every empty row would be a `Y.Map` the CRDT has to carry forever.
- Vivification is guarded — a `set` that creates a row must also assign the schema's
`titleColumn` in the same call, so a mistyped rowId errors instead of silently spawning a junk
row. Ordinals never vivify: an out-of-range ordinal is an error, because `47.company` on a
12-row table is far more likely a miscount than an intent to create 35 blank rows.
- **`add_rows` keeps the pipe-row grammar** and is now a bulk convenience rather than a required
step. A new row is genuinely row-shaped when every cell is known at once (an import, a fan-out
skeleton), and pipe rows are byte-identical to the document's markdown mirror, so the agent
reads and appends in one syntax. Leading `_id` mandatory, `-` mints one. Cell count is
validated against the schema and a mismatch is a hard error naming the expected columns.
- **Columns are always declared and always headed — a deliberate divergence from Sheets.** The
Sheets API has no column metadata at all: columns are anonymous letters, row 1 is ordinary data
that humans agree to treat as a header, and there is no type system beyond display formats and
optional data validation. That is correct for a *general* grid, where the schema lives in the
user's head. It is wrong here for three reasons:
- **Typed reference cells need a machine-readable schema.** `[[conversation: 9f2e…]]` has to
render as a live chip, resolve its label, and export as a hyperlink. Something has to know
that this column holds conversations. Sheets never needs this because every cell is a scalar.
- **Fan-out needs somewhere to hang the instruction.** A subagent filling one row is told
"fill `outreach` according to this instruction." With anonymous columns there is no stable
thing to attach that to. "Create a table and find these details" *is* per-column instructions;
`fillInstruction` is the column's real payload.
- **Anonymous grids push the structure problem onto the reader, and it is expensive.**
SpreadsheetLLM's entire contribution is recovering structure from Sheets-shaped grids —
structural-anchor compression, header detection, format-aware aggregation — precisely because
the format declares none of it. Declaring the schema is choosing not to pay that tax.
The comparison class is Airtable and Notion, not Sheets: every product with typed cells forces
named, typed fields. The cost is real and worth naming — a pasted CSV has to have its columns
inferred (Phase 9), and there is no such thing as a quick untyped scratch column.
- `label` is required, non-empty, and unique; `key` is a stable readable slug, immutable once
minted. Renaming a column changes `label` only, so every `r_07.outreach` reference — in the
document, in agent memory, in a fan-out already in flight — keeps resolving.
- **The verb set is backend-agnostic on purpose.** `read` / `add_rows` / `set` / `delete_rows`
map cleanly onto the Sheets API — `values.get`, `values.append`, `values.batchUpdate`,
`batchUpdate.deleteDimension` — so a `TableBackend` seam ships in Phase 2 with `CedarTableBackend`
as its only implementation. Driving a real Google Sheet is then one added file rather than a
redesign, and is deliberately not planned now.
- Note what is *not* being adopted: Sheets' **addressing model**. Matching its API shape is
useful; making A1 primary would reintroduce the fragility from the previous section for no
gain. The model does not need us to look like Sheets — SheetCopilot's result is that it needs a
small coherent verb set, which is what it gets either way.
- **`read` closes a gap.** Without it the agent's only way to see a table is the whole markdown
mirror, which is untenable at 500 rows — the SpreadsheetLLM result is that naive full-grid
serialization is exactly what blows the context budget. `read` takes an optional column subset,
row-id list, or `where` filter and returns pipe rows plus the current `revision`.
- Data per streamed call:
```json
{
"verb": "set",
"ifRevision": 43,
"assignments": "r_07.outreach = [[draft: r-3821]]\nr_08.outreach = [[draft: r-3822]]"
}
```
- Every bulk argument stays one record per line, so the streaming property from the previous
section is unchanged and gets finer: `set` streams *cell by cell* rather than row by row, so a
column fills top-to-bottom in front of the user. Streaming has two granularities — per tool call
(each verb publishes its own Y.js delta) and per line within a call (Phase 4's split-on-newline
emitter).
6. **The table as the general parallel-work primitive** — the framing the rest of this section
implements
- A table is not only a file type. **It is the control surface for any task with N independent
units of work.** Rows are the units. Materializing the work list as rows *is* the fan-out plan;
filling it *is* the execution; reading it afterwards *is* the audit trail. Nothing else in
Cedar makes a parallel agent run inspectable while it runs.
- **A fan-out needs no column taxonomy.** A subagent handed three things — its row's cells, the
schema (every column's `label` and `fillInstruction`), and the set of columns it may write —
has everything required to do its job and write back. A column labelled "Instruction" carrying
a `fillInstruction` is already unambiguous to a model; classifying it further would be a second
source of truth that can disagree with the first. Spawn an agent on the row, tell it what it
may write, let it work.
- Three workflows this has to serve, used as the acceptance criteria in Phase 9:
- **Post-release outreach.** "I just shipped a feature — look at all my customers." The agent
resolves the deal set, lays down one row per conversation, writes a per-row `instruction`,
and fans out. `output` holds `[[draft: id]]` or an explicit skip.
```text
| _id | Conversation | Instruction | Status | Output | Reasoning |
| r_01 | [[conversation: 9f2e…]] | Pitch inline-tables to Acme | ✅ drafted | [[draft: r-3821]] | Asked for this in Q2 |
| r_02 | [[conversation: 44a1…]] | Skip — churned last month | ⏭️ skipped | | Closed-lost 2026-06-14 |
```
- **Pipeline-wide analysis.** "Pull every deal closing this quarter into a table with CRM
fields X, Y, Z plus their ICP fit." Columns are the fields; the fan-out fills each row from
conversation history. The result is a durable artifact, not a chat message that scrolls away.
- **Multi-step outbound.** "Take this list, find their LinkedIn, connect, track who accepts,
then message them." Each row is a **long-lived state machine**, not a one-shot fill — the
distinguishing feature of this third case, and why Phase 10 exists.
- **Progress needs no new machinery, which is the point.** A subagent reporting
`set r_07.status = 🔍 researching` is an ordinary `set`: one `Y.Map.set`, one Y.js delta, one
`docEventBus` publish, and the cell updates in the open grid. Status is data in a column, not a
side channel — so it persists, it exports to Excel, it is greppable in the markdown mirror, and
a human can overwrite it. The reserved `_status` from step 7 stays, but only because retry and
cost-control need one machine-readable state; everything a human reads can live in an ordinary
`status` column the agent defines.
- **Rows come from a query, not from the agent typing them.** Asking a model to hand-write 40
rows of conversation refs is slow, lossy, and wrong as often as it is right. `create` therefore
takes a `from` clause that runs a real query server-side and materializes one row per result,
setting the ref cell and every bound column in the same pass:
```json
{
"verb": "create",
"schema": { "columns": ["…"] },
"from": { "source": "conversations", "filter": { "stage": "negotiation", "closingBefore": "2026-09-30" } }
}
```
`source: 'conversations'` reuses the same filter surface as
[findCRMConversationsTool.ts](apps/server/src/mastra/tools/conversation/findCRMConversationsTool.ts)
rather than a second query language. Workflow 1 becomes "create from this filter, then fan out"
— two calls, no row transcription.
- **Bound columns keep Cedar data live in the grid, and Excel's own storage model is the right
one to copy.** A `.xlsx` cell stores *both* the formula and the last computed value —
`<c r="B2"><f>SUM(A1:A5)</f><v>15</v></c>` — so the file opens instantly without recalculating,
and the formula is what gets re-evaluated when something changes. Sheets exposes the same
duality (`valueRenderOption: FORMATTED_VALUE | FORMULA`).
- Cedar copies the split but hoists the expression from the cell to the **column**, which is
Airtable's and Notion's model rather than Excel's: our columns are typed and uniform, so a
per-cell formula would be a thousand copies of one string. So `TableColumn.binding` holds
`conversation.nextStepDate`, and the cell holds the cached value. The binding resolves against
the row's ref cells — a row containing `[[conversation: 9f2e…]]` resolves `conversation.*`
against `crm_conversations` for that id.
- Both halves survive every round trip, because both already have a home: the binding lives in
the frontmatter column declaration and the cached value lives in the pipe cell, so the markdown
mirror stays readable and greppable and the agent reads a real value rather than an expression
it would have to evaluate. Excel export writes the value to the cell and the binding to the
hidden `_cedar` sheet — exactly `<v>` and `<f>`, split across the two places the workbook
already carries.
- **Keeping them correct is a recompute-trigger question, and there are three.** On `create`
(initial population), on open (`read` refreshes bound columns for the rows it returns, bounded
by the row cap), and on entity change — `crm_conversations` writes already append to
`crm_conversation_updates`, which is the natural invalidation signal: on a conversation update,
find tables holding that ref and recompute only the bound cells of only the affected rows. Each
recompute is an ordinary `set`, so it rides the same Y.js delta and SSE path as everything else
and the open grid updates live.
- **A bound cell is either LINKED or DERIVED, and it is the invalidation that decides.** A
path whose write invalidates the cell (`crm_conversations`, `crm_conversation_field_values`)
is edited through the binding: the grid writes the cell and sends the same
`crm.updateConversation` the deal header sends, and the recompute the CRM write already
triggers confirms it. Everything else — an enrichment field, a joined `owner.name`, a row
referencing no deal — stays read-only, because a write with no invalidation behind it would
leave the cell showing a value nothing keeps true. Severing a column's link converts it to a
plain column and keeps the last computed values, which is the escape hatch for a value that
should stop tracking at all.
- **This is worth putting in the prompt, not just the skill.** The agent will not reach for a
table unless told to, and the failure mode without it is the current one: N sequential tool
calls, no visible progress, and a chat message summarizing work the user cannot inspect.
Phase 9 adds the guidance to
[cedar-docs-awareness.ts](apps/server/src/mastra/prompts/cedar-docs-awareness.ts), which is
always-on preamble rather than a skill the agent must choose to load.
7. **Row-scoped subagent fan-out** — `runTableFill` in
`apps/server/src/services/documents/table/table-fill.ts` (new), driven by the existing
`spawn-subagent` tool ([spawnSubagentTool.ts](apps/server/src/mastra/tools/skills/spawnSubagentTool.ts))
- **Claim-then-fill.** The parent agent first calls `add_rows` to lay down the skeleton — every
row gets its `rowId` and whatever seed cells the parent already knows (the company name, the
conversation ref). Only then does it fan out, one subagent per `rowId`. Row identity therefore
exists *before* any concurrency does, which is the whole reason the fan-out is safe.
- Each subagent's brief is narrow: `{ documentId, rowId, columns: [...], instruction }`. Its
allowlist contains the `table` tool restricted to `set` on its own `rowId`, plus research tools.
It cannot `add_rows`, cannot `delete_rows`, and cannot touch a sibling's row.
- **Identity and scope.** Tables default to `user/tables/{name}` — private to the spawning user,
shared through the existing share-link path if wanted. Subagent writes carry that user's
identity with `lastEditedBy: 'agent'`, exactly as agent document writes already do, and each
`document_updates` row records `actorLabel: 'table-fill:{rowId}'` so a 40-row fan-out is
attributable row by row in history.
- **Why this composes rather than fighting the storage model.** Two subagents writing
`r_07.outreach` and `r_08.outreach` produce Y.js updates against disjoint `Y.Map` entries, so the CRDT merges them with no conflict resolution and no last-write-wins loss —
the same property that lets two humans type in two cells at once. Ordinal addressing would
break this completely (every concurrent insert shifts every other agent's coordinates), which
is the second, stronger reason the design addresses rows by `rowId`.
- **Write contention is the real constraint, and it is a queue problem, not a correctness one.**
`applyUpdate` already serializes per document with `pg_advisory_xact_lock(hashtext('cedar:doc:'||id))`
under an 8s `lock_timeout` ([applyUpdate.ts:66-72](apps/server/src/services/document-saving/applyUpdate.ts)),
so concurrent writes cannot be lost — but 40 subagents each firing their own transaction will
queue on that lock and the tail will hit the timeout. `runTableFill` therefore puts a
per-document coalescing queue in front of `writeTableAsYjs`: `set` calls arriving within a short
window merge into one `applyUpdate`, and a lock timeout retries with backoff instead of
surfacing as a failed row.
- **Progress is visible because it is data.** The schema reserves a `_status` column
(`pending | running | done | failed`) and `_error`, written by the orchestrator rather than the
subagent. The grid renders it as a per-row state chip, so a 40-row fan-out lights up row by row
— the same `docEventBus` delta path as any other write, no bespoke progress channel.
- **Failure is per row, not per run.** A subagent that dies marks its row `failed` with the error
text; the run completes with the other 39 rows filled. Re-running targets only `failed` rows.
- Concurrency is capped (default 8 in flight) and the fan-out requires confirmation above a row
threshold, because one subagent per row over a 500-row table is a runaway cost shape.
- Data after a fan-out step:
```json
{
"documentId": "b3f1…",
"spawned": 40,
"inFlight": 8,
"done": 31,
"failed": ["r_12"],
"coalescedWrites": 9
}
```
8. **Size limits** — enforced in `writeTableAsYjs`
- Cedar's save path is whole-document by construction, and a table makes that visible in a way
prose never did. Every `applyUpdate`
([applyUpdate.ts:221-231](apps/server/src/services/document-saving/applyUpdate.ts)) re-encodes
the full Y state, re-serializes the full markdown mirror, and rewrites both the entire
`content_yjs` bytea and the entire `content` text — so a one-cell `set` costs O(whole table).
- **A hard cap of 1,000 rows makes that a non-issue**, which is why it is the design rather than
a limitation to work around. MEASURED (`table-scale.test.ts`, 6 columns, realistic content,
interleaved single-cell `set`s):
| rows | `content_yjs` | mirror | rewritten per write | delta per cell |
| ---: | ---: | ---: | ---: | ---: |
| 100 | 41.3 KB | 29.6 KB | 70.9 KB | **71 B** |
| 500 | 205.8 KB | 146.0 KB | 351.8 KB | **71 B** |
| 1000 | 411.4 KB | 291.5 KB | **702.8 KB** | **71 B** |
The original estimate ("mirror ~200 KB, `content_yjs` ~600 KB") was inverted in both
directions but right in its conclusion: a cell write at the cap rewrites **~703 KB — under a
megabyte**. The number the whole design rests on is the last column: the Y delta for one cell
is **71 bytes at every size**, flat across a 10× table, because it is one `Y.Map.set`. What
scales is only the whole-blob rewrite `applyUpdate` performs, which is why Phase 5's
coalescing queue merges a fan-out's writes into a handful of transactions on top of it.
(Latency is deliberately not asserted — it swung 2–3× between runs on a shared machine, so a
wall-clock assertion would be a flaky test that gets deleted rather than a guarantee.)
The cap matches both the intended use — a conference list, a pipeline segment, a customer
set — and the category, since Airtable's free tier caps at 1,000 records and Notion databases
degrade in the low thousands. Warn at 500, reject above 1,000 with an actionable error.
- **The one thing the cap does not bound is edit count — but MEASUREMENT narrowed this
considerably from the original claim.** The assumption was that every cell rewrite costs bytes,
so a static 200-row table whose `status` churns `pending → running → done` daily would grow
indefinitely. Measured on real Y.Docs (200 rows × 3 columns, 17.3 KB baseline):
- Rewriting **one cell** 100,000 times grew the document by **33 bytes**. Not 33 KB — 33 bytes.
Y.js merges consecutive same-key writes from the same client and `gc` reclaims their content,
so single-cell churn is simply not a problem.
- Rewriting cells **round-robin across rows** costs ~9.7 bytes per write, because interleaving
different keys breaks that merge: 2,000 writes → 2.1× stored size, 10,000 → 6.3×,
50,000 → 29×.
So the churn axis is real, but only for *interleaved* writes, and it accrues over thousands of
them. The daily-schedule table does ~600 interleaved writes/day (~6 KB), so it takes **months**,
not days, to warrant compaction. That is what makes Phase 12's flag-then-compact-on-next-write
shape right: the event is rare and never urgent. A row cap still cannot bound it.
- Storing rows as a Y.Array rather than ProseMirror nodes (step 9) also removes the
`ydocToProsemirrorJson` walk on every save and makes a cell write one `Y.Map.set` item instead
of a cell → paragraph → text subtree.
- If rows ever needed to live outside the Y.Doc, `TableBackend` (step 5) is the seam — a
`postgres` backend would be a second implementation alongside `cedar`, and
`useYTable` / `table-ydoc.ts` are the only two modules that know the Y layout. Not planned.
9. **Rendering: a real data grid, not a ProseMirror node view** — `TableGrid` in
`apps/mail/modules/documents/table/TableGrid.tsx` (new)
- **Y.js and ProseMirror are separable, and this is the seam.** The `Collaboration` extension
binds to exactly one field — `field: 'prosemirror'`
([document.tsx:339](apps/mail/modules/documents/document.tsx)) — and every Y access in the
codebase today is `getXmlFragment('prosemirror')`. Sibling top-level Y types in the same Y.Doc
are untouched by it. So rows can live in `ydoc.getArray('tableRows')` while the entire
server-side apparatus (`applyUpdate`, the advisory lock, `document_updates` /
`document_snapshots`, `docEventBus`, share tokens) and the entire client transport
(`CedarYjsProvider`, IndexedDB warm load, `useDocEvents`) keep working byte-for-byte — none of
them knows or cares what shape the Y.Doc holds.
- **This is the shape Notion converged on**, and it is the strongest argument for the split.
Notion's database rows *are* blocks — the same atomic unit as a paragraph — but collections and
views sit on top as a separate layer, where "the views are rendering logic on top of the
flexible store." One store, two renderers: the block editor draws prose, the collection view
draws the grid. Cedar's equivalent is one Y.Doc, two renderers: TipTap for prose documents,
`TableGrid` for tables.
- **And Notion's editor is not a document tree at all — editability is per region.** Notion uses
no editor framework (not ProseMirror, not Slate); each block is an *independent contenteditable
DOM element*, and rich text is a **value format** — an array of rich-text objects carrying
annotations, where a mention is simply an object of `type: 'mention'` rather than a node in a
document tree. Database property values are arrays of that same format.
- That is precisely why you can `@` inside a Notion table cell: **not** because the cell lives
inside a document editor, but because the cell holds the same rich-text value a paragraph holds
and is edited by the same per-region editable component. The lesson to take is the inversion —
rich cells need a rich *value format* plus a per-region editor, not a document tree that
everything must live inside. Cedar's cell value (text plus `[[type: id]]` tokens) is the
analogue of Notion's rich-text array, and step 9 supplies the per-region editor.
- **ProseMirror genuinely has no table virtualization, and it is architectural rather than a
gap.** PM's view-descriptor tree is load-bearing: selection management and `view.coordsAtPos`
assume a complete descriptor tree, and each descriptor writes its own DOM immediately, so
removing off-screen nodes breaks selection, coordinate mapping, and the browser's native
selection (which expects a complete DOM). The people who needed it did not write a plugin —
they replaced the view layer wholesale (a React renderer, a canvas renderer). "Add an
extension" is not on the table; "fork the renderer" is.
- **But virtualization is not the load-bearing reason, and it would be dishonest to pretend
otherwise.** At the 1,000-row cap from step 8, a ProseMirror grid would render acceptably. The
decisive reasons are the other two: every real grid affordance — column resize and reorder,
sort, filter, frozen panes, range selection, fill-handle drag, pasting a block of cells — is
fighting a document model rather than using one; and a PM cell is a subtree (cell → paragraph →
text, each carrying a nodeId) which inflates every delta and forces a full
`ydocToProsemirrorJson` walk on each save. Virtualization is what buys headroom if the cap
later rises.
- `@tanstack/react-table` and `@tanstack/react-virtual` are already dependencies of `@zero/mail`,
so the headless grid model and the virtualizer cost zero new packages.
- The grid subscribes to the Y.Array with `observeDeep` and re-renders only the affected rows, so
an agent appending row 400 does not re-render rows 1-399. Virtualization means only the visible
window is in the DOM.
- Cells are strings; `[[type: id]]` tokens are parsed at render time into live chips
(see step 9). This is strictly simpler than storing them as ProseMirror inline atoms, and it
makes the markdown mirror a direct projection of the row rather than a tree walk.
- **The unbounded grid stays a rendering concern.** The grid draws every materialized row, then a
small fixed run of trailing blank rows (default 3) as a typing affordance — the feel of the
empty rows below the data in Sheets. Those blanks hold no `rowId`, exist nowhere in the Y.Doc
or the markdown mirror, and materialize into a real row on the first keystroke.
- **What this costs, stated plainly.** Four things currently key off the ProseMirror tree and
therefore need a table-specific branch or are absent in v1: Y.Doc comments
([use-ydoc-comments.ts](apps/mail/modules/conversations/lib/use-ydoc-comments.ts)), the
history / version-restore viewer, the public share page, and "Copy as Markdown". The first two
are deferred; the last two are cheap because both can read the markdown mirror directly.
- One transport detail this changes: `replaceWithServerState`
([CedarYjsProvider.ts:311](apps/mail/modules/documents/yjs/CedarYjsProvider.ts)) clears the
`prosemirror` fragment before applying server state, so it must also clear `tableRows` — or a
hard server-state replacement would merge into local rows instead of replacing them.
10. **Ref chips — the same chips the editor already renders, not a parallel set**
- This is the one real cost of leaving ProseMirror, and it is smaller than it looks because the
shared thing was never the TipTap node. Take `ConversationNodeView`
([ConversationNode.tsx](apps/mail/modules/agentCanvas/extensions/ConversationNode.tsx)): the
value in it is `useConversationNodeData(conversationId)` for live name/logo and
`openConversationFromAgenda(id)` for the click — both plain functions with no ProseMirror
dependency. The only PM-specific parts are `NodeViewWrapper` and reading `node.attrs`.
- **The codebase has already established this split.** `FileLinkChip.tsx` exports
`FileLinkChipContent` — a standalone chip explicitly documented as "used outside the editor
(sidebars, link previews)" — alongside `FileLinkChip`, the `NodeViewProps` wrapper that mounts
it, with a comment explaining that the in-editor variant must live directly inside
`NodeViewWrapper` to avoid PM DOM-reconciliation loops. The grid uses exactly the pattern the
file-link chip already uses.
- So the plan is extraction, not duplication: pull `ConversationChipContent` and
`EventChipContent` out of their node views the way `FileLinkChipContent` already is, have both
the PM node view and the grid render the same component, and behaviour stays identical by
construction — same data hooks, same click handlers, same artifact-open path. A chip renders
and behaves the same whether it is in a prose doc or a table cell because it *is* the same
component.
- The grid parses each cell value for `[[type: id]]` tokens and renders the spans between them as
text and the tokens as chips. No editor node, no schema registration.
**Editing a cell — a one-line TipTap instance, mounted on focus.** Display and edit are separate
modes, which is how Notion behaves and how every real grid behaves:
- Nothing requires an entire document to be one editor. TipTap is a per-region editor too, so the
focused cell — and only the focused cell — mounts a single-line editor configured with
StarterKit-minimal plus the **existing** suggestion extensions from `useRichTextExtensions`
([use-rich-text-extensions.ts](apps/mail/modules/documents/use-rich-text-extensions.ts)):
`createConversationMention`, `createEventMention`, `createFileLinkSuggestionExtension`. On blur
the content serializes back to the token string and lands as one `Y.Map.set`.
- This is the synthesis of the two architectures: Notion's per-region editability, but with
Cedar's existing `@` infrastructure instead of a hand-rolled contenteditable — which is the
part Notion engineers describe as learning "the internal state machines and undocumented
behaviors of every browser you support," including IME and CJK handling we would otherwise own.
- It also answers the reuse question completely. Typing `@` in a table cell runs the *same*
suggestion extension, the same search query, and inserts the same node as typing `@` in a prose
document — not a parallel implementation that drifts.
- Exactly one editor instance is alive at a time, so the virtualization and descriptor-tree costs
from step 8 never apply: they scale with rows rendered as editors, and that number is one.
- The costs worth naming: instantiating a TipTap editor on focus is a few milliseconds, so the
display cell must be pixel-identical to the edit cell or focus will visibly shift; and
`Escape`/`Tab`/`Enter` need explicit handling to return focus to the grid's keyboard model
rather than being swallowed by the editor.
- Each chip reads the referenced object through the existing query surfaces
(`crm.searchConversationsMinimal` for conversations, `userTasks` for tasks, the draft record
for drafts) so status is current, not frozen at write time.
- Clicking calls `useCedarStore.getState().setSelectedArtifact({ kind, id })` — the same entry
point `MentionChip` uses
([MentionChip.tsx:49](apps/mail/modules/cedar-os/src/cedar-os-components/chatMessages/MentionChip.tsx))
— so a draft cell opens the draft and a conversation cell opens the full `ConversationView`.
11. **FileEditor branch** — [CompanyExplorer.tsx:1677](apps/mail/modules/company/components/CompanyExplorer.tsx)
- Adds a `documentType === 'table'` branch that mounts `TableGrid` full-width — acquiring the
same refcounted provider `<Document />` uses, but binding the grid to the Y.Array instead of
mounting a TipTap editor. Toolbar: add row / add column / export. "Download as Markdown" keeps
working because it can read the mirror; "Copy as Markdown" switches to the mirror for tables.
12. **Artifact panel branch** — [DisplayArtifactPanel.tsx:176](apps/mail/modules/home/components/DisplayArtifactPanel.tsx)
- `ArtifactBody`'s `file` case fetches the doc (it already does, at
[DisplayArtifactPanel.tsx:199](apps/mail/modules/home/components/DisplayArtifactPanel.tsx)) and,
when `documentType === 'table'`, renders the same `TableDocumentView` the file editor uses
instead of the `<pre>` markdown dump. No new `ContextKind` — a table is a file.
13. **Stats hook** — `tableStatsHook` appended to
[registry.ts:15](apps/server/src/services/document-saving/registry.ts)
- Scoped to `documentType === 'table'`. Reads the final `tableRows` array and writes
`{ kind: 'table', schemaVersion, columnCount, rowCount }` into `documents.metadata` so list
views, the file tree, and `list-documents` can show "38 rows × 6 columns" without loading
content.
14. **Excel export** — `documents.exportTable` in
[documents.ts](apps/server/src/trpc/routes/documents.ts) → `tableToWorkbook` in
`apps/server/src/services/documents/table/table-excel.ts` (new)
- `exceljs@4.4.0` is already a dependency of both `@zero/server` and `@zero/mail`.
- Sheet `Data`: header row from `schema.columns[].label`, a hidden first column holding `_id`,
one row per `Y.Map`. Ref cells write the resolved label as the display value plus a
hyperlink to the Cedar deep link, and the raw `[[type: id]]` token into the cell's note, so a
human sees "Acme — Demo scheduled" and a re-import still recovers the reference.
- Sheet `_cedar` (hidden): a single cell holding `JSON.stringify(schema)`.
- Column types map to real Excel formats: `number`/`currency`/`percent` → numFmt, `date` →
date format, `checkbox` → boolean, `select` → a data-validation list from `column.options`.
15. **Excel import** — `documents.importTable` → `workbookToTableMarkdown` in the same module
- Source is an existing `attachment` document (spreadsheets are already in the upload allowlist
at [allowlist.ts:39](apps/server/src/services/file-system/uploads/allowlist.ts)), so import is
"upload a file, then convert it", not a second upload path.
- If the `_cedar` sheet is present the schema is taken verbatim and `_id`s are preserved, making
export → edit in Excel → import a lossless round trip that updates existing rows in place. If
absent, columns are inferred from the header row plus a value scan (all-numeric → `number`,
all-parseable-date → `date`, ≤ 12 distinct values in > 20 rows → `select`, else `text`) and
fresh `rowId`s are minted.
- Emits the canonical markdown mirror and calls the `create` path, producing an ordinary `table`
document.
### 3.3 Schema
No database migration is required. `documents.document_type` is a `text` column with all
enforcement in TypeScript ([documents-schema.ts:70](apps/server/src/db/documents-schema.ts)), so
`table` is a pure type-level addition; per-table structure lives in the document's own Y.Doc, and
the only DB-visible addition is a documented shape inside the existing `documents.metadata` jsonb.
Full schema:
```ts
// apps/server/src/services/documents/document-types.ts — CHANGED
export const DOCUMENT_TYPE = {
DOCUMENT: 'document',
HTML: 'html',
AGENDA: 'agenda',
CONVERSATION_AGENDA: 'conversation_agenda',
ATTACHMENT: 'attachment',
FOLDER: 'folder',
PLAYBOOK: 'playbook',
AGENT: 'agent',
TABLE: 'table', // NEW — Y.js-backed spreadsheet document
} as const satisfies Record<string, string>;
// apps/server/src/services/documents/table/table-types.ts — NEW (shared server + client)
/** What a column holds. Drives the cell editor, validation, and the Excel cell format. */
export const TABLE_COLUMN_TYPE = {
TEXT: 'text',
LONG_TEXT: 'long_text',
NUMBER: 'number',
CURRENCY: 'currency',
PERCENT: 'percent',
DATE: 'date',
CHECKBOX: 'checkbox',
SELECT: 'select',
MULTI_SELECT: 'multi_select',
URL: 'url',
EMAIL: 'email',
// Reference columns — cells hold `[[type: id]]` tokens, rendered as chips.
CONVERSATION: 'conversation',
TASK: 'task',
DRAFT: 'draft',
DOC: 'doc',
PERSON: 'person',
COMPANY: 'company',
} as const satisfies Record<string, string>;
export type TableColumnType = (typeof TABLE_COLUMN_TYPE)[keyof typeof TABLE_COLUMN_TYPE];
/** Reference kinds a cell token may carry. Subset of CEDAR_DOC_REFERENCE_TYPE. */
export const TABLE_REF_TYPE = {
CONVERSATION: 'conversation',
TASK: 'task',
DRAFT: 'draft',
DOC: 'doc',
PERSON: 'person',
COMPANY: 'company',
} as const satisfies Record<string, string>;
export type TableRefType = (typeof TABLE_REF_TYPE)[keyof typeof TABLE_REF_TYPE];
/**
* A column is always declared and always headed — there are no anonymous or
* positional columns. `key` is the stable handle (the column analogue of `rowId`);
* `label` is the human header. Renaming a column changes `label` only, so every
* `r_07.outreach` reference in the doc, in agent memory, and in a running fan-out
* keeps resolving.
*/
export interface TableColumn {
/**
* Stable, readable, lower-snake slug. Unique within the table and IMMUTABLE once
* created — `add_columns` mints it from the initial label, and rename never touches
* it. Readable rather than opaque (`outreach`, not `c_7a2f`) because the agent types
* it in every `set` line.
*/
key=[redacted];
/** Required, non-empty, unique within the table. The rendered header. */
label: string;
type: TableColumnType;
/** One-line human description of the column, shown on header hover. */
description?: string;
/**
* Agent-facing instruction: how to derive this cell's value. This is the column's
* real payload for a fan-out — each subagent gets the `fillInstruction` of every
* column it owns. Named to match the `fill_instructions` frontmatter key already
* used by agent documents.
*/
fillInstruction?: string;
/** Cells must be non-empty before a row counts as `done` in a fan-out. */
required?: boolean;
/**
* Bound column — the value is derived from the Cedar object model, not written by a
* human or an agent. The path is resolved against the row's ref cells: given a row
* holding `[[conversation: 9f2e…]]`, `conversation.nextStepDate` reads
* `crm_conversations.next_step_date` for that id. A bound cell is never typed OVER —
* it is either edited through the binding (a writable conversation field) or read-only.
* See §3.2 step 6.
*/
binding?: string;
/** SELECT / MULTI_SELECT only. */
options?: string[];
/** CURRENCY only. ISO 4217, e.g. 'USD'. */
currency?: string;
/** DATE only. date-fns format string; defaults to 'yyyy-MM-dd'. */
format?: string;
/** Rendered width in px. Null = auto. */
width?: number | null;
/** Hidden in the grid but present in the mirror and in exports. */
hidden?: boolean;
}
export interface TableSchema {
version: 1;
columns: TableColumn[];
/**
* Column key whose value labels a row in chips, exports, and agent prose. REQUIRED —
* the vivification guard in `set` refuses to create a row unless this column is
* assigned in the same call, so an optional titleColumn would leave that guard
* unenforceable. Defaults to the first non-reserved column on `create`.
*/
titleColumn: string;
/** Why the table exists, written on create. Shown as a subtitle; fed to filling agents. */
purpose?: string;
}
/**
* Reserved column keys. Present on every table, never declared in `schema.columns`,
* never editable by a filling subagent — only by the orchestrator.
*
* _id — the rowId. Serialized as the first pipe column; hidden in the grid.
* _status — per-row fan-out state, rendered as a state chip.
* _error — failure text when _status = 'failed'.
*/
export const RESERVED_COLUMN = {
ID: '_id',
STATUS: '_status',
ERROR: '_error',
} as const;
export type TableRowStatus = 'pending' | 'running' | 'done' | 'failed';
// ── Backend seam ──
//
// The verb set is deliberately backend-agnostic. `cedar` is the only implementation
// planned; the seam exists so an external-store backend (a Google Sheet, or a Postgres
// row table if the size limits in step 8 ever bind) is an added implementation rather
// than a reopened design.
export type TableBackendKind = 'cedar' | 'sheets' | 'postgres';
export interface TableBackend {
read(req: Extract<TableVerb, { verb: 'read' }>): Promise<TableReadResult>;
addColumns(cols: TableColumn[], after?: string): Promise<void>;
addRows(rows: TableRowInput[]): Promise<string[]>;
setCells(cells: TableCellWrite[]): Promise<void>;
deleteRows(rowIds: string[]): Promise<void>;
/** Monotonic version for the ifRevision guard. yjs_revision here; a Sheet's revisionId there. */
revision(): Promise<number | string>;
}
export interface TableReadResult {
schema: TableSchema;
/** Pipe rows, `_id` first — the same grammar `add_rows` accepts. */
rows: string;
revision: number | string;
/** Set when the backend cannot represent something (e.g. Sheets has no ref chips). */
degradations?: string[];
}
/**
* Set on a table document whose rows live in an external sheet rather than in this
* document's Y.Doc. Stored in `documents.metadata`.
*/
export interface TableBackendBinding {
kind: TableBackendKind;
/** External-store identifiers, when the backend is not `cedar`. */
externalId?: string;
externalName?: string;
}
// ── Row-scoped subagent fan-out ──
export interface TableFillRequest {
documentId: string;
/** Rows to fill. Omit to target every row whose `_status` is 'pending' or 'failed'. */
rowIds?: string[];
/** Column keys each subagent is allowed to write. Everything else is rejected. */
columns: string[];
/** The per-row brief. `{{rowId}}` and `{{<columnKey>}}` interpolate from the seed row. */
instruction: string;
/** Max subagents in flight. Default 8. */
concurrency?: number;
/** Above this row count the fan-out requires explicit user confirmation. Default 25. */
confirmAbove?: number;
}
export interface TableFillResult {
documentId: string;
spawned: number;
done: string[];
failed: Array<{ rowId: string; error: string }>;
/** How many subagent `set` calls were merged into a single applyUpdate transaction. */
coalescedWrites: number;
}
/** Written into `documents.metadata` by tableStatsHook on every save. */
export interface TableDocumentMetadata {
kind: 'table';
schemaVersion: 1;
columnCount: number;
rowCount: number;
generatedBy?: 'agent' | 'human' | 'import';
/** Set when the table came from `documents.importTable`. */
importedFrom?: {
attachmentDocumentId: string; // → documents.id (documentType = 'attachment')
sheetName: string;
};
/** Which backend holds the rows. Absent means `cedar`. Read by `tableTool` per call. */
backend?: TableBackendBinding;
/** Raised by the bloat detector; cleared by the next write that compacts. See Phase 12. */
compactionDue?: boolean;
}
// ── Y.js shapes (the authoritative structure, stored in content_yjs) ──
//
// Rows do NOT live in the `prosemirror` XmlFragment. They live in sibling top-level
// Y types in the SAME Y.Doc, so every server path (applyUpdate, the advisory lock,
// document_updates / document_snapshots, docEventBus, share tokens) and every client
// path (CedarYjsProvider, IndexedDB warm load, useDocEvents) is unchanged, while
// rendering is free to be a real data grid instead of a ProseMirror node view.
// See §3.2 step 8 for why.
//
// ydoc.getMap('tableSchema') → TableSchema, one key per field
// ydoc.getArray('tableRows') → Y.Array<Y.Map> , one Y.Map per row
// ydoc.getXmlFragment('prosemirror') → empty for a standalone table document
//
// A row is a flat Y.Map: `_id` plus one entry per column key. `Y.Map.set` is a single
// Y item, so a one-cell write is a handful of bytes on the wire — against a
// ProseMirror cell, which is a subtree of cell → paragraph → text each carrying a
// nodeId, and which forces a full `ydocToProsemirrorJson` walk on every save.
export const Y_TABLE_SCHEMA = 'tableSchema';
export const Y_TABLE_ROWS = 'tableRows';
/**
* One row. Values are cell markdown — plain text and/or `[[type: id]]` tokens, parsed
* into chips at render time rather than stored as nodes.
*
* { _id: 'acme', company: 'Acme Corp', deal: '[[conversation: 9f2e…]]',
* outreach: '[[draft: r-3821]]', _status: 'done' }
*
* Concurrency is cell-level last-write-wins, which is what Sheets and Airtable do and
* what a spreadsheet should do: two agents writing different columns of the same row
* both survive, and two writers racing the same cell resolve deterministically.
* LONG_TEXT columns may hold a `Y.Text` instead of a string when character-level merge
* inside one cell is wanted; every other type stores a plain string.
*/
export type TableRowMap = Y.Map<string | Y.Text>;
/** Materialized shape the grid and the serializer both consume. */
export interface TableRow {
rowId: string;
cells: Record<string, string>;
}
/**
* A parsed `[[type: id]]` token found in a cell value. Produced at render time by the
* grid and at export time by the workbook writer — never stored.
*/
export interface CellRef {
refType: TableRefType;
/** crm_conversations.id | user_tasks.id | draft id | documents.id | person/company id. */
refId: string;
/** Character range within the cell value, so the renderer can interleave text and chips. */
start: number;
end: number;
}
/**
* Anchor node placed in the `prosemirror` fragment ONLY when a table is embedded in a
* prose document. It carries the id of the table document whose Y.Array holds the rows;
* a standalone table document has no such node. Not needed for v1 — recorded here so
* embedding is an addition rather than a re-design.
*/
export interface TableEmbedNodeJson {
type: 'tableEmbed';
attrs: { nodeId: string; tableDocumentId: string };
}
// ── Agent tool surface: the `table` verb set ──
//
// Structure (the schema) is JSON. Bulk (rows, assignments) is line-oriented text in a
// single string arg, so a partially-arrived tool call can be split on '\n' and applied
// line by line. See §3.2 step 5.
export type TableVerb =
/** Returns pipe rows + the current revision. The agent's only cheap way to see a big table. */
| {
verb: 'read';
/** Omit for all columns. */
columns?: string[];
/** Omit for all rows. Mutually exclusive with `where`. */
rowIds?: string[];
/** `columnKey op value` per line, ANDed. Ops: =, !=, contains, empty, notempty. */
where?: string;
limit?: number;
offset?: number;
}
/**
* `rows` writes literal rows; `from` runs a query server-side and materializes one row
* per result, filling the ref cell and every bound column in the same pass. Use `from`
* whenever the row set comes from Cedar data — never make the model transcribe it.
*/
| { verb: 'create'; schema: TableSchema; rows?: string; from?: TableRowSource }
| { verb: 'add_columns'; columns: TableColumn[]; after?: string }
/**
* Bulk convenience for the dense case, NOT a required step — `set` vivifies rows on
* its own. Use this when every cell of a new row is known at once (importing a list,
* laying down a fan-out skeleton); use `set` for everything else.
*
* One GFM pipe row per line — identical syntax to the document's markdown mirror.
* Leading cell is the rowId; `-` mints a fresh one. Cell count must equal
* schema.columns.length or the whole call errors with the expected column list.
*
* - | Globex | [[conversation: 44a1…]] | |
* r_07 | Initech | | [[draft: r-99]] |
*/
| { verb: 'add_rows'; rows: string }
/**
* The workhorse, and an UPSERT — writing to an unknown rowId materializes that row.
* There is no "the table only has 12 rows" error, exactly as writing to A1000 in
* Sheets just works. One `<rowRef>.<columnKey> = <value>` assignment per line.
*
* acme.company = Acme Corp ← unknown rowId → row is created
* acme.outreach = [[draft: r-3821]] ← same row, now existing
* r_07.outreach = [[draft: r-3822]] ← stable rowId, always safe
* 4.outreach = Sent ← 1-based data-row ordinal, needs ifRevision
* D4 = Sent ← A1-style, needs ifRevision
*
* Vivification is guarded: a `set` that creates a row must also assign the schema's
* `titleColumn` in the same call, so a mistyped rowId errors instead of silently
* spawning a junk row. Ordinals never vivify — an out-of-range ordinal is an error.
*
* Values are cell markdown (plain text and/or `[[type: id]]` tokens). A `\|` escapes
* a literal pipe; a trailing `\` continues the value onto the next line.
*/
| { verb: 'set'; assignments: string; ifRevision?: number }
/** Same addressing as `set`; empties the cell without deleting the row. */
| { verb: 'clear'; targets: string; ifRevision?: number }
/** Newline- or comma-separated rowIds (ordinals allowed with ifRevision). */
| { verb: 'delete_rows'; rowIds: string; ifRevision?: number };
/**
* Where a table's rows come from. Reuses the existing CRM filter surface rather than
* inventing a second query language — `conversations` resolves through the same path as
* `findCRMConversationsTool`.
*/
export interface TableRowSource {
source: 'conversations' | 'tasks' | 'people' | 'companies' | 'documents';
/** Passed through to the source's existing filter surface. */
filter?: Record<string, unknown>;
limit?: number;
/** Column key that receives the `[[<source>: id]]` ref for each result. Defaults to the first ref column. */
intoColumn?: string;
}
// ── Bound columns (live values derived from the Cedar object model) ──
/**
* A resolvable binding path. The leading segment names the entity the row refs; the
* remainder is a field on it.
*
* conversation.nextStepDate | conversation.stage | conversation.owner.name
* conversation.fields.<customFieldKey> — user-defined CRM fields
* task.dueDate | task.status
* company.domain | company.headcount
*/
export type BindingPath = string;
export interface BindingResolution {
rowId: string;
columnKey=[redacted];
/** Cached value written into the cell. Null clears it. */
value: string | null;
/** Set when the path did not resolve — surfaced in the tool result, not written to the cell. */
error?: string;
}
/** Recompute triggers. See §3.2 step 6. */
export const BINDING_REFRESH = {
/** Initial population during `create`. */
ON_CREATE: 'on_create',
/** `read` refreshes bound columns for the rows it returns. */
ON_READ: 'on_read',
/** A `crm_conversation_updates` row for a referenced entity invalidates its bound cells. */
ON_ENTITY_CHANGE: 'on_entity_change',
} as const;
/** One parsed line of an `add_rows` payload. */
export interface TableRowInput {
/** Null when the line's leading cell was `-`; the writer mints `r_<nanoid>`. */
rowId: string | null;
/** columnKey → cell markdown. */
cells: Record<string, string>;
}
/** One parsed line of a `set` / `clear` payload, after ordinal resolution. */
export interface TableCellWrite {
/** Always a resolved rowId — ordinals are converted at parse time under ifRevision. */
rowId: string;
columnKey=[redacted];
value: string;
}
/**
* How a `set` line addressed its target, retained for error messages so a rejected
* ordinal write can tell the agent what it actually pointed at.
*/
export type CellAddressForm = 'rowId' | 'ordinal' | 'a1';
/** Raised when an ordinal-addressed call carries a stale `ifRevision`. */
export interface TableRevisionConflict {
code: 'REVISION_CONFLICT';
expected: number;
current: number;
/** Current row ids in order, so the agent can retarget without a second read. */
rowIds: string[];
}
/**
* Incremental parse used by BOTH the tool (whole payload) and the chat loop
* (partial `input_json_delta` accumulation). Returns only lines terminated by a
* newline; the trailing partial line is handed back for the next call.
*/
export function parseRowLines(
chunk: string,
schema: TableSchema,
): { rows: TableRowInput[]; remainder: string; errors: string[] };
export function parseAssignmentLines(
chunk: string,
schema: TableSchema,
rowIdsInOrder: string[],
): { cells: TableCellWrite[]; remainder: string; errors: string[] };
```
Relationship diagram:
```text
┌──────────────────────────────┐
│ documents │
│──────────────────────────────│
│ id uuid PK │
│ org_id uuid ──FK──► organizations.id
│ user_id text ──FK──► user.id
│ document_type text = 'table'│ ◄── NEW value, no migration (text column)
│ parent_id uuid ──FK──► documents.id (self, folder tree)
│ path text │
│ title, emoji text │
│ content text │ ── the markdown mirror: frontmatter + pipe table
│ content_json jsonb │ ── null for tables (rows live in content_yjs)
│ content_yjs bytea │ ── AUTHORITATIVE Y.Doc
│ metadata jsonb │
└───────┬──────────────────────┘
│ ▼ contains (jsonb)
│ ┌──────────────────────────┐
│ │ TableDocumentMetadata │
│ │ kind = 'table' │
│ │ columnCount, rowCount │
│ │ importedFrom.attachment │──FK(soft)──► documents.id (documentType='attachment')
│ └──────────────────────────┘
│
│ ▼ contains (content_yjs / content_json — NOT relational)
┌───────┴────────────────────────────────────────────────────────────┐
│ Y.Map 'tableSchema' → TableSchema │
│ Y.Array 'tableRows' │
│ ▼ 1:N │
│ Y.Map row { _id, <columnKey>: value, _status, _error } │
│ └─► columnKey ──► TableSchema.columns[].key │
│ ▼ 0:N (parsed from the value string at render time) │
│ CellRef { refType, refId } │
└───────┬────────────────────────────────────────────────────────────┘
│ refId resolves by refType — soft references, no DB FK
│
├── refType='conversation' ──►┌────────────────────┐
│ │ crm_conversations │
│ │ id uuid PK │
│ └────────────────────┘
├── refType='task' ──────────►┌────────────────────┐
│ │ user_tasks │
│ │ id uuid PK │
│ │ task_output jsonb │──► draft id lives here
│ │ conversation_id ──FK──► crm_conversations.id
│ └────────────────────┘
├── refType='draft' ─────────►┌────────────────────┐
│ │ user_tasks │
│ │ .task_output │ kind='email' → draftId
│ │ (or the Gmail │ the chip resolves via the
│ │ draft record) │ owning task when present
│ └────────────────────┘
└── refType='doc' ───────────►┌────────────────────┐
│ documents │
│ id uuid PK │
└────────────────────┘
Y.Doc history reuses the existing tables unchanged:
documents.id ──1:N──► document_updates (sessions)
documents.id ──1:N──► document_snapshots (checkpoints)
documents.id ──1:N──► document_share_views
```
## 4) Implementation phases
### Phase 1 — `table` document type + markdown round-trip (server)
**Goal:** A `table` document persists, parses, and re-serializes losslessly through the existing
save pipeline.
- [x] Add `TABLE: 'table'` to `DOCUMENT_TYPE` in `apps/server/src/services/documents/document-types.ts` and update the doc comment
- [x] Create `apps/server/src/services/documents/table/table-types.ts` with `TABLE_COLUMN_TYPE`, `TABLE_REF_TYPE`, `TableColumn`, `TableSchema`, `TableDocumentMetadata`, `TableRow`, `CellRef`, and the `Y_TABLE_SCHEMA` / `Y_TABLE_ROWS` key constants
- [x] Create `apps/server/src/services/documents/table/table-ydoc.ts` with `readTable(ydoc): { schema, rows }` and `writeTable(ydoc, { schema, rows })` over `ydoc.getMap('tableSchema')` and `ydoc.getArray('tableRows')` — the single place that knows the Y layout
- [x] Create `apps/server/src/services/documents/table/table-markdown.ts` exporting `parseTableMarkdown(md): { schema, rows }`
- [x] Implement schema frontmatter parsing in `table-markdown.ts` (reuse `extractFrontmatter` from `apps/server/src/services/document-saving/frontmatter.ts`)
- [x] Implement pipe-table row parsing with the reserved leading `_id` column mapped to `TableRow.rowId`, minting `r_<nanoid>` when the cell is blank
- [x] Export `serializeTableToMarkdown({ schema, rows }): string`, emitting frontmatter + pipe table with `_id` first; cell values pass through verbatim, so `[[type: id]]` tokens need no special handling
- [x] Export `parseCellRefs(value): CellRef[]` for the grid and the workbook writer — parse only, never store
- [x] Branch `getMarkdownSerializer` at `apps/server/src/services/document-saving/serialize.ts` so a `table` document serializes from the Y.Array via `readTable` instead of from the (empty) ProseMirror JSON
- [x] Branch `getMarkdownParser` at `apps/server/src/services/document-saving/markdown.ts` so a `table` document's markdown hydrates the Y.Array rather than the `prosemirror` fragment
- [x] Audit `reconstructYDoc` at `apps/server/src/services/document-saving/hydrate.ts` for `table` rows — a legacy/markdown-only row must rebuild into `tableRows`, not into the XML fragment
- [x] Add `tableStatsHook` in `apps/server/src/services/document-saving/hooks/table-stats.ts` writing `TableDocumentMetadata` into `documents.metadata`, and register it in `apps/server/src/services/document-saving/registry.ts`
- [x] Accept `'table'` in the `type` enum of `WriteDocumentInputSchema` at `apps/server/src/mastra/tools/document/writeDocumentTool.ts`
- [x] Hydrate the Y.Array on a `table` `upsert` inside `writeFileAsYjs`, and reject `append` / `patch` for a table in `writeDocument` — see "Implementation notes" below
**Tests:**
- [x] `apps/server/src/services/documents/table/__tests__/table-markdown.test.ts` — markdown → Y.Doc → markdown is byte-identical for a table with every column type
- [x] Same file — `_id` column round-trips into `rowId` and is never rendered as a data column
- [x] Same file — `parseCellRefs` finds every `[[type: id]]` token with correct offsets and ignores malformed ones
- [x] `apps/server/src/services/documents/table/__tests__/table-ydoc.test.ts` — `writeTable` then `readTable` is an identity round trip, and a table Y.Doc leaves the `prosemirror` fragment empty
- [x] `table-markdown.test.ts` — a row with fewer cells than columns is padded, and a cell containing a literal `|` is escaped and recovered. The mirror parser is deliberately tolerant because it reads already-persisted content; the *tool input* parser in Phase 2 is strict about cell count instead
- [x] `apps/server/src/services/document-saving/__tests__/table-stats.test.ts` — the hook writes correct `rowCount` / `columnCount`
- [x] `timeout 300 pnpm --filter @zero/server exec vitest run src/services/documents/table` — 38 tests; with `src/services/document-saving` too: 124 passing
**Headless verification (run as <email> against a worktree API on :8797):**
```bash
pnpm --filter @zero/server run cedar-cli document write --user <uid> \
--path user/tables/phase1-smoke --mode upsert --type table --content-file phase1.md
pnpm --filter @zero/server run cedar-cli document read --user <uid> \
--path user/tables/phase1-smoke # → byte-identical mirror, schema + _ids + refs intact
pnpm --filter @zero/server run cedar-cli document write … --mode append # → rejected with the table-tool error
```
Row state after the run: `document_type='table'`, `metadata={kind:table,rowCount:2,columnCount:4,schemaVersion:1}`,
`content` 536 B, `content_yjs` 758 B.
**Implementation notes — where the build diverged from the plan, and why:**
- **`getMarkdownSerializer` now returns `(json, ydoc?) => string`.** The plan assumed the
serializer could be branched on document type alone, but its whole input was ProseMirror JSON —
which for a table is empty by construction. The seam had to become Y.Doc-aware. `applyUpdate` is
the only site that *persists* `documents.content`, so it always passes the ydoc; read-only callers
(history reconstruct, verify-playbook) may omit it and get an empty mirror for a table.
- **`HookContext` gained `ydoc` (the post-merge Y.Doc).** `HookContext.document` is the *pre*-update
row, so a `tableStatsHook` reading `document.contentYjs` reported counts one save stale, forever.
`afterJson` could not substitute because it describes the fragment only.
- **The `append` / `patch` rejection lives in `writeDocument`** (`services/documents/index.ts`), not
only in `writeFileAsYjs` as planned. `writeDocument` folds the mode into `finalContent` itself and
then calls `writeFileAsYjs` with `UPSERT`, so a downstream-only guard never fires. Verified: the
guard was silently bypassed until moved. The `writeFileAsYjs` check is kept as defence-in-depth
for any caller that passes a partial mode directly.
- **`writeFileAsYjs` needed a table hydration branch, not just a mode guard.** Its `upsert` path fed
the parsed markdown into `prosemirrorJsonToYDoc`, leaving `tableSchema` / `tableRows` empty — and
because `applyUpdate` serializes a table's mirror *from those*, the same save then wrote an empty
mirror over the markdown just supplied. The first headless run reproduced exactly this: a created
table read back as an empty schema and zero rows.
- **`admin.documents.writeGuarded` gained `documentType`, `readByPath` now returns
`documentType` / `yjsRevision` / `version`.** There was no headless path to create a non-default
document type at all, and `yjsRevision` is the value the `ifRevision` guard needs. `--type` is
wired through `cedar-cli document write`.
- **The schema frontmatter uses a purpose-built mini-YAML reader/writer.** `@zero/server` has no YAML
dependency, and the block's shape is fixed (scalars plus a `columns:` list of single-line flow
maps). The emitter JSON-quotes any value containing `:,{}[]"'#|` so a `fillInstruction` sentence
cannot break the flow-map grammar; the reader is deliberately tolerant.
- **`apps/mail` tests run under Jest, not vitest.** Every later phase's `pnpm --filter @zero/mail
exec vitest run …` command should read `pnpm --filter @zero/mail run test <path>`, and new tests
must live under `apps/mail/tests/…` (a fresh `__tests__` dir outside `jest.config.cjs`'s
hand-listed roots is not collected).
### Phase 2 — Row-granular Y.js writes + the `table` verb set
**Goal:** An agent can append rows and set individual cells with a delta proportional to the change,
and each call broadcasts live.
- [x] Create `apps/server/src/services/document-saving/writeTableAsYjs.ts` implementing `create` / `add_columns` / `add_rows` / `set` / `clear` / `delete_rows` against `tableSchema` / `tableRows` via `table-ydoc.ts`
- [x] Reject a `table` documentType inside `writeFileAsYjs` for `append` and `patch` modes with a clear error pointing at the `table` tool (`upsert` stays legal for full rewrites) — landed in Phase 1, and the guard had to move up into `writeDocument`; see Phase 1's notes
- [x] Publish on `docEventBus` from `writeTableAsYjs`, mirroring `writeFileAsYjs.ts:232`
- [x] Add `createStructuredLog('info', '[writeTableAsYjs] applied verb', …)` with verb, affected row/cell counts, `updateByteLength`, `contentYjsBytes`, and `rowCount`
- [x] Add `parseRowLines(chunk, schema)` and `parseAssignmentLines(chunk, schema, rowIdsInOrder)` to `apps/server/src/services/documents/table/table-lines.ts` — split on `\n`, return complete records plus the trailing remainder, never throw on a partial line
- [x] Validate cell count per row against `schema.columns.length` in `parseRowLines` and return a per-line error naming the expected columns, so a collapsed empty cell fails loudly instead of shifting the row
- [x] Implement address resolution in `parseAssignmentLines` for all three forms (`rowId`, 1-based ordinal, A1) against `rowIdsInOrder`, tagging each parsed cell with its `CellAddressForm`
- [x] Implement row vivification in `set`: an unknown `rowId` pushes a new row `Y.Map`, gated on the same call also assigning `schema.titleColumn`; an out-of-range ordinal is an error and never vivifies
- [x] Enforce column invariants in `add_columns`: `label` required, non-empty and unique; `key` minted as a lower-snake slug from the first label, de-duplicated on collision, and rejected if it would change an existing column's key
- [x] Define column deletion semantics: removing a column drops it from the schema AND deletes that key from every row `Y.Map`, in one transaction — a schema-only removal would leave orphaned cell values that reappear if the column is ever re-added under the same key
- [x] Require `titleColumn` on `create`, defaulting it to the first non-reserved column, since the vivification guard is unenforceable without it
- [x] Make column rename a `label`-only operation and add a test proving existing `set` lines keyed on the old slug still resolve
- [x] Enforce the revision guard in `writeTableAsYjs`: any call containing an ordinal- or A1-addressed line requires `ifRevision`, and a mismatch against `documents.yjs_revision` rejects the whole call with a `TableRevisionConflict` carrying the current revision and row ids
- [x] Enforce the row cap in `writeTableAsYjs`: reject writes that would exceed 1,000 rows with an actionable error, and return a warning in the tool result above 500
- [x] Define the `TableBackend` interface in `apps/server/src/services/documents/table/backend.ts` and implement `CedarTableBackend` against it, so `tableTool` talks to the interface rather than to `writeTableAsYjs` directly
- [x] Create `apps/server/src/mastra/tools/document/tableTool.ts` exposing the verb set, resolving its backend from `documents.metadata.backend`, and register it in `FAMILY_TOOLS` + `scope-map.ts`
- [x] Implement the `read` verb with `columns` / `rowIds` / `where` / `limit` / `offset`, returning pipe rows plus the current `revision`
- [x] Emit the `docUpdate` SSE event from the table tool the same way `writeDocumentTool` does at line 520
- [x] Document the verb set in the `documents` skill at `apps/server/.claude/skills/documents/SKILL.md` — read-then-set as the default loop, `rowId` addressing preferred over ordinals, and when to reach for `add_rows` vs `set`
- [x] Add the headless driver: `admin.tables.*` (`apps/server/src/trpc/routes/admin-tables.ts`) and `cedar-cli table` (`apps/server/src/table-admin/cli.ts`), mirroring the agent verb set one-for-one over the same parsers, backend and guards
**Tests:**
- [x] `apps/server/src/services/document-saving/__tests__/writeTableAsYjs.test.ts` — `add_rows` leaves every existing row's `Y.Map` and `rowId` untouched (asserted on Y *item identity*, not just on values)
- [x] Same file — a concurrent edit to row A's cell survives an `add_rows` that adds row B (apply both deltas to a third Y.Doc, assert both present)
- [x] Same file — `add_columns` adds the column and every existing row reads it as empty
- [x] Same file — `set` on an unknown `rowId` that assigns the title column creates exactly one row; the same call without the title column errors and creates nothing
- [x] Same file — an out-of-range ordinal errors rather than vivifying, and the row count is unchanged
- [x] Same file — renaming a column's `label` leaves its `key` and every existing cell reference intact
- [x] Same file — an ordinal-addressed `set` with a stale `ifRevision` writes nothing and returns `REVISION_CONFLICT` with the current row ids
- [x] Same file — the same `set` addressed by `rowId` succeeds regardless of revision drift, including after rows were reordered
- [x] Same file — 20 simulated agents each writing their own row merge to 20 filled rows with no lost writes
- [x] `apps/server/src/services/documents/table/__tests__/table-lines.test.ts` — feeding a payload one character at a time yields exactly the same records as parsing it whole, and never emits a partial line
- [x] Same file — a row with the wrong cell count produces a named error and does not silently shift columns
- [x] Same file — all three address forms resolve to the same `rowId` for an unmodified table
- [x] `timeout 300 pnpm --filter @zero/server exec vitest run src/services/document-saving/__tests__/writeTableAsYjs.test.ts src/services/documents/table` — 37 + 28 + 38 + hook/mirror tests; 606 green across the affected server suites
**Headless verification (as <email>, worktree API on :8797):**
| check | result |
| --- | --- |
| `table create --schema-file … --rows-file …` | 3 rows × 5 columns |
| `table read` | pipe rows + `revision=1` |
| `table set` — 3 sparse cells by rowId | `cellsWritten: 3`, `updateByteLength: 107` |
| `table set` with an ordinal and no `ifRevision` | refused, names the current revision |
| `table set` with a stale `ifRevision` | `REVISION_CONFLICT` + current rowIds |
| `table set '3.status = …'` + `'D1 = …'` with the right revision | applied; A1 letters index `[_id, …columns]` |
| `table set` vivifying without the titleColumn | refused, names the guard and lists live rows |
| `table set` vivifying with it | exactly one row created |
| `table clear` / `delete-rows` / `rename-column` | applied |
| `set` on a renamed column, keyed by its original slug | resolves — label moved, key did not |
| `table read --columns company,status --where 'status notempty'` | projected 2 columns |
**Implementation notes — deviations and why:**
- **The wire schema cannot be the doc's `TableVerb` discriminated union.** MCP family-tool registration requires `inputSchema` to be a real `z.object` with a `.shape` (see `hasRealShape` in `mcp/external/server.ts`), so a `z.discriminatedUnion` is rejected outright. The tool uses the repo's established shape instead: a flat object with an `action` enum plus one optional, action-named sub-object per action, with `path` hoisted to the top level. `TableVerb` remains the accurate description of the *semantics*; it is not the wire format.
- **`TableBackend` needed a `rowIds()` accessor.** The parsers take `rowIdsInOrder` as their ordinal→rowId resolution table, and the interface as designed could only produce it by serializing rows to pipe text through `read` and parsing them back.
- **`TableWriteResult` gained `broadcast?: { broadcastUpdate, fullUpdate, version }`.** The `docUpdate` chat event needs the Y.js delta bytes, which `CedarTableBackend` otherwise discards along with the whole `applyResult`. Only the three fields that event uses are re-exposed, so the seam stays honest and a Sheets backend simply omits it. The `admin.tables.*` HTTP envelope strips it — superjson renders a `Uint8Array` as `{"0":1,"1":13,…}`, which buried the actual result in the CLI output until stripped.
- **`applyTableOps(ydoc, ops)` is exported separately from `writeTableAsYjs`.** Every interesting invariant (vivification guard, column rules, row cap, delete-order safety, the CRDT merge properties) is pure Y.Doc mechanics, so it is tested without a database; the DB round trip is proven headlessly instead.
- **Atomicity is structural, not transactional.** `Y.transact` does not roll back, so a mid-batch throw leaves earlier ops applied *to the in-memory reconstruction* — but that doc is local to `writeTableAsYjs`, and throwing means `applyUpdate` is never reached, so nothing is encoded, persisted or broadcast and `yjs_revision` does not advance. The row cap is checked before the transaction so it cannot even half-apply in memory. Tested both ways.
- **A partially-bad payload returns `success: false`.** Since nothing is applied when *any* line fails, reporting success for a call that wrote nothing would mislead the agent.
- **`create`'s `from` clause is Phase 10**, not Phase 2 — `create` takes literal rows only for now, matching the doc's own phase split.
- **No `legacy-tool-name-map.ts` entry.** Those maps rename a family+action onto a pre-existing *legacy flat tool name*; `table` has no predecessor, so `toLegacyToolName` correctly falls through to the family id. An empty entry would be dead code.
### Phase 3 — Grid rendering
**Goal:** A `table` document opens as an editable grid in the file editor and grows live as the
agent appends.
- [x] Add the scale measurement — landed as `apps/server/src/services/documents/table/__tests__/table-scale.test.ts`, **not** `.bench.ts`: this package has no `benchmark` block in its vitest config and `include` is `src/**/*.test.ts`, so a `.bench.ts` would never run under the verification command and would rot silently. It measures bytes (deterministic) and asserts the invariant that matters — a single-cell delta at 1,000 rows within a small constant factor of one at 100 — while printing latency for information only. The measured numbers have replaced the estimates in §3.2 step 8. Read them with `--reporter=verbose`; the default reporter swallows a passing test's stdout.
- [x] Create `apps/mail/modules/documents/table/useYTable.ts` — acquires the refcounted provider for the documentId (same registry `<Document />` uses), binds `ydoc.getMap('tableSchema')` and `ydoc.getArray('tableRows')` via `observeDeep`, and exposes `{ schema, schemaError, rows, revision, isLoading, setCell, addRow, addRows, deleteRow, moveRow, setSchema, removeColumn, undo, redo }`
- [x] Create `apps/mail/modules/documents/table/TableGrid.tsx` on `@tanstack/react-virtual` (already a dependency), rendering only the visible window. NOT on `@tanstack/react-table`: its row model is derived from a `data` array, and this grid deliberately has none — cell values live in per-row `Y.Map`s that each row subscribes to itself, so a materialized `data` array would rebuild the row model on every keystroke. Column order and width live in the Y schema rather than in table state, and no sort/filter/group/paginate is in v1, so nothing else it offers applies. Follows the hand-rolled `modules/crm/components/crm-table.tsx` precedent over the same virtualizer
- [x] Make row re-render granular: a change to one row's `Y.Map` must not re-render sibling rows (memoize per `rowId`)
- [x] Add a sticky header with a column menu (rename label, change type, edit `fillInstruction`, insert left/right, delete, hide) writing back to the schema map — rename must alter `label` only, never `key`
- [x] Add column resize and reorder, persisting `width` and column order into the schema
- [x] Render a fixed run of trailing blank rows (default 3) below the last materialized row, holding no `rowId` and absent from the Y.Doc until the first keystroke materializes one
- [x] Extend the blank run on scroll-to-bottom rather than allocating rows
- [x] Add row affordances (add row at end, delete row, drag to reorder) operating on the Y.Array
- [x] Add grid-scoped undo: a `Y.UndoManager` bound to `tableRows` + `tableSchema` with `trackedOrigins` covering local edits and `'agent'`, wired to Cmd+Z. Without it the grid has no undo at all — the TipTap `Collaboration` extension supplied one for prose documents and there is no editor here to inherit it from
- [x] Scope undo to the local client's own actions plus agent writes, matching the `yUndoOptions` already used at `apps/mail/modules/documents/document.tsx:341`, so one user's Cmd+Z never reverts another's edit
- [x] Create `apps/mail/modules/documents/table/TableDocumentView.tsx` — `TableGrid` plus the toolbar, and `useDocEvents(documentId)` for live agent writes
- [x] Add the `documentType === 'table'` branch in `FileEditor` at `apps/mail/modules/company/components/CompanyExplorer.tsx:1677`, defaulting to full width
- [x] Extend `replaceWithServerState` at `apps/mail/modules/documents/yjs/CedarYjsProvider.ts:311` to clear `tableRows` alongside the `prosemirror` fragment, so a hard server-state replacement replaces rather than merges
- [x] Point "Copy as Markdown" at the markdown mirror for `table` documents, since there is no editor instance to read from
- [x] Add a `parseTableSchema` guard that renders an error card (mirroring `DashboardFenceView`) rather than crashing on a malformed schema
**Tests:**
- [x] `apps/mail/tests/modules/documents/table/TableGrid.test.tsx` — renders header labels from the schema and one row per `Y.Map` in the array
- [x] Same directory — applying an agent `add_rows` delta through the provider adds a row without remounting existing rows
- [x] Same directory — a `set` on row 3 re-renders row 3 only
- [x] Same directory — a malformed schema renders the error card, not a thrown component
- [x] Same directory — trailing blank rows render but add nothing to the Y.Doc, and typing into the first one materializes exactly one row with a fresh `rowId`
- [x] Same directory — with 1,000 rows only the visible window is in the DOM
- [x] Same directory — Cmd+Z reverts a local cell edit and an agent write, and does not revert a remote user's edit
- [x] Same directory — deleting a column removes it from the schema and from every row, and re-adding the same key yields empty cells
- [x] `timeout 300 pnpm --filter @zero/mail run test tests/modules/documents/table` (apps/mail runs JEST, not vitest — config at `apps/mail/jest.config.cjs`)
**Implementation notes — deviations and why:**
- **`rows` is a list of Y.Map HANDLES, not materialized `TableRow`s.** `useYTable` returns `{ rowId, map }` per row and each row component subscribes to its OWN map through `useYRowCells`. That is what makes the re-render granular: a materialized `TableRow[]` would be a new array on every cell write, re-rendering all 1,000 rows. The row-list state is only replaced when the row order or membership changes, so a `set` produces no parent render at all — hence `revision` counts structural changes only, and would defeat the whole scheme if it ticked per keystroke.
- **The grid does not re-implement the Y layout.** Every read and write goes through `@zero/server/table/ydoc` (exported as a new package subpath), the module that already declares itself the only one that knows the layout — so the client cannot drift from the agent writer on invariants like "an empty cell is absent, not `''`". `parseTableSchema` is the one deliberate exception: the server's `readTableSchema` is tolerant because it runs inside the save pipeline where a throw would fail the user's write, which is the wrong contract for a renderer.
- **`replaceWithServerState` clears `tableSchema` as well as `tableRows`.** A stale local schema would otherwise merge with the server's, and a merged column list is as wrong as merged rows.
- **Cells are plain inputs, committed on blur/Enter.** The one-line TipTap instance mounted on focus, and ref chips, are step 10's and Phase 6's job; a per-keystroke `Y.Map.set` would also leave a discarded Y item per character. Blank rows are the exception — they commit on the first keystroke, because the row has to exist before the character has anywhere to go.
- **Cmd+Z is bound to the grid root, not the window.** A table document has no competing editor, but a window listener would still reach the chat composer and every other input on the page.
- **`moveRow` rebuilds the row's `Y.Map`.** `Y.Array` has no move and a Y type cannot be re-inserted once it belongs to a document, so an explicit drag-reorder costs that row's CRDT identity. The agent writer never reorders, so this is bounded to a deliberate user gesture.
- **"Download as PDF" prints a table's markdown mirror in a `<pre>`.** It read the live editor DOM, which a table has none of, so it would have produced a blank page. A grid-shaped export lands with Excel export in Phase 8.
### Phase 4 — Mid-call row streaming
**Goal:** Rows land in the open grid while the model is still writing the tool call, not when the
call completes.
- [x] Extend the tool-call accumulator at [run-chat-agent-sdk.ts:267](apps/server/src/mastra/workflows/chat/run-chat-agent-sdk.ts) so a `table` `set` call scans `activeInputAccum` for newly-completed lines on each `input_json_delta` instead of waiting for `content_block_stop`
- [x] Extract the in-flight `rows` string value from the partial args with a narrow scan (locate the `"rows":"` key and read to the current end) — no general partial-JSON parser
- [x] Apply each completed line through `writeTableAsYjs` as it arrives, coalescing lines that land within one debounce window into a single Y.js delta
- [x] Make row application idempotent by `rowId` so the final `content_block_stop` pass re-applying the full payload is a no-op for already-applied rows
- [x] Fall back to whole-payload application when the partial scan finds no usable `rows` value, so a malformed or reordered arg stream degrades to today's behaviour rather than dropping rows
- [x] Gate the mid-call path behind a flag checked at tool dispatch so it can be disabled without reverting the tool
**Tests:**
- [x] `apps/server/src/mastra/workflows/chat/__tests__/table-stream.test.ts` — a simulated `input_json_delta` sequence delivered in arbitrary chunk boundaries applies each row exactly once
- [x] Same file — the terminal `content_block_stop` full-payload pass adds no duplicate rows
- [x] Same file — a truncated stream (no `content_block_stop`) leaves every completed row applied and the partial row absent
- [x] `timeout 300 pnpm --filter @zero/server exec vitest run src/mastra/workflows/chat/__tests__/table-stream.test.ts`
**Implementation notes — deviations and why:**
- **The state machine is its own module, `services/documents/table/table-stream.ts`**, with the
DB-backed writer split into `table-stream-apply.ts` and run-chat-agent-sdk.ts reduced to three
lines of wiring (start on a `table` `content_block_start`, `push` each delta, `end` on
`content_block_stop`). The accumulator sits deep inside a 60-branch event loop, so testing it in
place would have meant standing up a whole `query()` session per assertion.
- **The narrow scan is string-aware, not `indexOf('"rows":"')`.** It tracks quote/escape/brace
state so a cell value containing the text `"path":"` cannot redirect the scan into the middle of
a record. It is still only a lexer — it finds `action`, `path`, and the offset where the bulk
string's content begins, then stops. It never materializes a value, so a truncated number or a
missing brace is not its problem.
- **Escape decoding is the subtle part.** The buffer holds a JSON-*encoded* string, so a record
boundary arrives as the two characters `\` `n`. A chunk that ends mid-escape is HELD BACK rather
than decoded: a lone trailing `\` could still become `\n` (a record boundary) or `\\` (a literal
backslash, which the line grammar reads as a continuation), and guessing splits or joins records.
Same for a `\u` with fewer than four hex digits. A split surrogate pair needs nothing special —
decoded text is concatenated onto the previous line-parse remainder before the `\n` split, and a
pair can never straddle a newline.
- **`add_rows` now MERGES a pipe row whose explicit rowId already exists**
([writeTableAsYjs.ts](apps/server/src/services/document-saving/writeTableAsYjs.ts)) instead of
appending a second row with the same `_id`. That is what makes the terminal pass a no-op, and it
is a correctness fix in its own right: two rows sharing an `_id` are unaddressable, since
`findRowMap` only ever returns the first.
- **Anonymous (`-`) rows are never streamed.** A minted rowId is not stable across the two passes,
so streaming one would land the row twice under two ids. They land at the end, as they do today.
- **Ordinal / A1 payloads are not streamed at all.** They resolve against a row order the session
is itself changing, and the terminal pass guards them with `ifRevision` — which our own writes
advance. The residual case is a payload that MIXES rowId lines with a later ordinal line: the
early writes bump the revision and the terminal pass returns `REVISION_CONFLICT`, which is the
self-healing retry the design already specifies rather than a lost write.
- **Cells for a row that does not exist yet are held** until the same call assigns the schema's
`titleColumn` for that rowId, so a streamed batch cannot trip the writer's vivification guard
mid-payload. A row whose title never arrives is simply left to the terminal pass.
- **A bad line stops the session but keeps the well-formed lines that parsed.** The terminal pass
rejects the whole payload and lists every bad line, so the agent resends — and every streamed
write is idempotent, so the resend converges. Discarding the good lines would buy nothing.
- **The flag is `TABLE_STREAM_DISABLED`**, an env kill-switch read through `env` at tool dispatch,
matching `CRM_ACTIVE_DEAL_RECONCILE_DISABLED`. Streaming is ON by default; setting it restores
apply-at-`content_block_stop` without reverting the tool.
### Phase 5 — Row-scoped subagent fan-out
**Goal:** One subagent per row can fill its own row concurrently, without colliding, with per-row
progress visible in the grid and failures isolated to their row.
- [x] Add the reserved `_status` / `_error` columns to `apps/server/src/services/documents/table/table-types.ts` and teach `parseTableMarkdown` / `serializeTableToMarkdown` to round-trip them without declaring them in `schema.columns` (landed in Phase 1)
- [x] Create `apps/server/src/services/documents/table/table-fill.ts` implementing `runTableFill(TableFillRequest): Promise<TableFillResult>` — claim rows, run one row-scoped agent per `rowId`, cap in-flight at `concurrency`
- [x] Build the per-row brief by interpolating `{{rowId}}` / `{{<columnKey>}}` from the seed row so each subagent gets only its own context, not the whole table
- [x] Attribute every fan-out write with `actorLabel: 'table-fill:{rowId}'`, so `document_updates` shows which row produced which edit — a *merged* batch is labelled `table-fill:N rows` rather than falsely credited to one row
- [x] Enforce row scope: `assertRowScope` rejects any write whose target `rowId` is not the assigned one, and any column outside the writable allowlist
- [x] Reject ordinal and A1 addressing for fan-out subagents — `runRowAgent` writes go through ops carrying `form: 'rowId'` only, and under concurrent inserts a coordinate is meaningless
- [x] Add a per-document coalescing write queue (`table-write-queue.ts`) in front of `writeTableAsYjs` so `set` calls arriving within a short window merge into one `applyUpdate` transaction rather than each taking `pg_advisory_xact_lock` separately
- [x] Retry with backoff + jitter on the 8s `lock_timeout` from [applyUpdate.ts:66](apps/server/src/services/document-saving/applyUpdate.ts) instead of surfacing it as a failed row
- [x] Write `_status` transitions (`pending` → `running` → `done` / `failed`) from the orchestrator, never the subagent, so a crashed subagent still leaves an accurate row state
- [x] Record the failure text in `_error` and make a re-run target only rows whose `_status` is `failed` (or absent / `pending`)
- [x] Require explicit user confirmation above `confirmAbove` rows (default 25) before spawning, since one subagent per row is a superlinear cost shape
- [x] Add `createStructuredLog('info', '[runTableFill] complete', …)` with spawned / done / failed / `coalescedWrites` counts
- [x] Add the headless driver: `admin.tables.fill` + `cedar-cli table fill --simulate`
- [x] Render `_status` as a per-row state chip in `TableGrid` — landed with Phase 6, which owns cell rendering (`RowStatusChip`, in the row gutter)
- [x] Restrict the spawned subagent's allowlist in `apps/server/src/mastra/tools/subagent-tool-allowlists.ts` — deferred with the live spawn path (see below)
**Tests:**
- [x] `apps/server/src/services/documents/table/__tests__/table-fill.test.ts` — 21 tests covering row selection, brief construction, scope enforcement, status transitions, failure isolation, re-run targeting, the concurrency cap, the confirmation gate, and coalescing arithmetic
- [x] `writeTableAsYjs.test.ts` — 20 simulated agents writing 20 distinct rows merge to 20 filled rows with no lost writes (the CRDT property lives with the writer, where it belongs)
- [x] Same file — two agents writing different columns of the SAME row both survive the merge
- [x] `table-fill.test.ts` — an agent attempting to write a `rowId` other than its assigned one is rejected
- [x] Same file — one agent throwing leaves its row `failed` with the error text and every other row `done`
- [x] Same file — a re-run with no `rowIds` targets only the `failed` row and does not re-spawn for `done` rows
- [x] Same file — the coalescing queue turns N rapid writes into fewer than N `applyUpdate` invocations while preserving every value
- [x] `timeout 300 pnpm --filter @zero/server exec vitest run src/services/documents/table/__tests__/table-fill.test.ts`
**Headless verification (12-row fan-out, concurrency 4):**
```
table create → 12 rows × 4 columns (claim-then-fill: identity exists before concurrency)
table fill → spawned 12, done 12, failed 0, coalescedWrites 14
table read → every row filled with ITS OWN interpolated brief, _status = done
(done order r_01,r_02,r_04,r_03,… — genuine concurrency, not a loop)
set r_03._status = failed; table fill (no rowIds) → spawned 1, done [r_03]
```
**Implementation notes — deviations and why:**
- **The live subagent spawn path is deferred, and the headless driver simulates it.**
`executeAnalyzerSubagent` requires a Mastra `RequestContext` that a tRPC route cannot
cheaply build, and the spawned agent needs the `table` tool in its allowlist. So
`admin.tables.fill` takes `simulate: true` and swaps a deterministic stub for the model
call. That is not a toy: the stub writes through the *same* `TableBackend`, so the real
Y.Doc, the real save pipeline, the real coalescing queue and the real SSE broadcast are
all exercised — which is where the risk actually lives. What remains unproven is only
that a model, handed a `RowBrief`, writes sensible values; that is Phase 11's eval work.
`runTableFill` takes its runner as a `RowAgentRunner` dependency precisely so the real
spawn is an injected function rather than a rewrite.
- **`_status` is written by the orchestrator, and claimed for the whole run up front.**
Marking every target `pending` before any worker starts means the grid shows the run's
full shape immediately rather than revealing rows as workers pick them up.
- **Merged-batch attribution is explicit.** A coalesced batch carries
`table-fill:N rows`, never one row's label — `document_updates` must not claim a 12-row
batch came from one row's agent.
- **A run stopped for confirmation writes nothing at all**, not even `pending` statuses.
Leaving 30 rows marked `pending` for a run that never happened would be worse than the
refusal itself. Tested.
### Phase 6 — Typed cells and live ref chips
**Goal:** Reference columns render live Cedar objects that open as artifacts, and typed columns get
real editors.
- [x] Extract `ConversationChipContent` from `ConversationNodeView` at `apps/mail/modules/agentCanvas/extensions/ConversationNode.tsx`, mirroring the existing `FileLinkChipContent` / `FileLinkChip` split, and have the node view render the extracted component
- [x] Extract `EventChipContent` from `EventNode` at `apps/mail/modules/documents/mention/EventNode.tsx` the same way
- [x] Create `apps/mail/modules/documents/table/CellRefChip.tsx` that dispatches on `refType` to the extracted chip components — conversation, task, draft, doc, person, company — so a cell chip and a prose chip are literally the same component
- [x] Implement live resolution per `refType` through the existing query surfaces, with a skeleton while pending and a dimmed "missing" state on 404
- [x] Wire chip click to `setSelectedArtifact({ kind, id })`, following `MentionChip.tsx:49`
- [x] Create `apps/mail/modules/documents/table/CellEditor.tsx` — a single-line TipTap instance mounted only for the focused cell, configured with StarterKit-minimal plus `createConversationMention`, `createEventMention`, and `createFileLinkSuggestionExtension` from `apps/mail/modules/documents/use-rich-text-extensions.ts`
- [x] Serialize the editor's content back to the token string on blur and commit it as a single `Y.Map.set`
- [x] Make the display cell and the edit cell pixel-identical (same padding, font, line height) so mounting the editor on focus produces no visible shift
- [x] Handle `Escape` (revert), `Enter` (commit + move down), `Tab` (commit + move right) explicitly so grid keyboard navigation is not swallowed by the editor, and ensure the suggestion popover takes priority while open
- [x] Add typed cell editors for `date`, `select`, `multi_select`, `checkbox`, `number`, `currency`, `percent`, `url`, `email` — these replace `CellEditor` for their columns rather than wrapping it
- [x] Add a per-row overflow action "Create task for this row" that creates a `user_tasks` row and writes `[[task: id]]` into the row's first `task`-typed column
- [x] Verify copy-of-a-cell yields its raw value including any `[[type: id]]` token, so pasting between tables preserves references
**Tests:**
- [x] `apps/mail/tests/modules/documents/table/CellRefChip.test.tsx` — a conversation chip renders the resolved name and calls `setSelectedArtifact` on click
- [x] Same directory — the extracted `ConversationChipContent` renders identically inside a TipTap node view and inside a grid cell (same component, both mount paths)
- [x] Same directory — an unresolvable `refId` renders the missing state without throwing
- [x] Same directory — selecting a date in a `date` cell writes the configured format into the cell text
- [x] Same directory — typing `@` in a focused text cell opens the same conversation-suggestion popover as a prose document and inserts a resolvable `[[conversation: id]]` token
- [x] Same directory — exactly one TipTap instance exists while a cell is focused, and zero after blur
- [x] Same directory — `Escape` reverts the cell, `Enter` commits and moves down, `Tab` commits and moves right
- [x] `timeout 300 pnpm --filter @zero/mail run test tests/modules/documents/table` (apps/mail runs JEST, not vitest — config at `apps/mail/jest.config.cjs`)
**Implementation notes — deviations and why:**
- **The extraction is a polymorphic container, not a wrapper.** `FileLinkChip` re-implements
`FileLinkChipContent` because its comment's constraint is that the chip's OUTER element must BE
the `NodeViewWrapper` (carrying `data-conversation-node` so the live DOM matches `parseHTML`'s
selector and `renderHTML`'s output; anything else and PM's MutationObserver reconciles in a
loop). `ConversationChipContent` / `EventChipContent` satisfy that constraint without the second
copy: the node view passes `container={NodeViewWrapper}` and the chip renders AS the wrapper, so
the DOM ProseMirror sees is unchanged by the extraction and the grid and the editor are provably
the same component (`CellRefChip.test.tsx` compares the two mounts' class sets).
- **`CellEditor` takes the whole `useRichTextExtensions` hook**, not the three factories
individually. The requirement is that `@` in a cell runs the SAME extension as `@` in prose;
re-wiring the factories would satisfy it today and be a thing that can drift tomorrow. Cost: a
cell editor also mounts the `[[` file-link and `{{` event suggestions, which is what the design
asked for anyway.
- **Client-side `parseCellRefs`.** The canonical one lives in `table-markdown.ts` over
`CEDAR_DOC_REFERENCE_REGEX`, and neither is reachable — `@zero/server` publishes only `./table`
and `./table/ydoc`. `apps/mail/modules/documents/table/cell-refs.ts` restates the grammar once,
and `cell-refs.test.ts` reads the server regex literal off disk and pins the two together, so a
divergence fails a test instead of silently rendering tokens as text. Replace it with a shared
export (see the server-export note under Phase 7).
- **The chip set is `TABLE_REF_TYPE` plus `event`.** `event` is not a column type and is never
validated server-side, but `{{` inserts an `eventNode` and the cell serializer writes
`[[event: id]]`, so the grid has to render it back. It resolves to a label, not a link — no
surface takes a bare event id.
- **A cell's display is an `<input>` until it holds a chip.** Three renderings, chosen by the DATA:
a value with tokens renders text runs interleaved with chips, anything else renders the input
(which is what makes display and edit pixel-identical for the common case), and focus swaps in
the editor. The input keeps its own blur-commit because focus → setState → remount is
asynchronous: a keystroke in the same task as the focus lands on the input, and dropping it would
be a lost character.
- **`Escape` returns focus to the grid root, not to the cell.** Focusing the display is what OPENS
the editor, so restoring focus there would immediately reopen it.
- **Cell geometry is one constant.** `CELL_CLASS` in `constants.ts` is the only place padding, font
size, height and the cell rule are written down — the pixel-identity requirement is not something
two similar-looking class strings can hold.
- **A `select` column with no `options` falls back to the scalar input** rather than rendering a
dead trigger, so an incompletely-declared schema stays editable.
- **`draft` chips open the draft's THREAD.** There is no `draft` ContextKind; `email_thread` is what
every other draft-review surface already uses.
### Phase 7 — First-class file and artifact
**Goal:** Tables are creatable from the UI, look right in the file tree, and render properly when
selected as an artifact.
- [x] Add "New table" to the file-creation menu in `apps/mail/modules/company/components/CompanyExplorer.tsx`, creating at `user/tables/{slug}` with a 3-column starter schema and `titleColumn` set to the first column
- [x] Add a `table` icon branch to `nodeKindFor` at `apps/mail/modules/files/graph/transform.ts:17` and a `'table'` member to `GraphNodeKind` in `apps/mail/modules/files/graph/types.ts`
- [x] Show "N rows × M columns" from `TableDocumentMetadata` in the file list and doc-link results
- [x] Render `TableDocumentView` for `documentType === 'table'` in `ArtifactBody` at `apps/mail/modules/home/components/DisplayArtifactPanel.tsx:176`
- [x] Render tables on the public share page at `apps/mail/app/(full-width)/share/[token]/page.tsx` (read-only grid)
- [x] Add a `table` case to `resolveOpenDoc` at `apps/mail/modules/conversations/components/files/resolveOpenDoc.ts` so a conversation-scoped table opens in the Files tab
**Tests:**
- [x] `apps/mail/tests/modules/home/DisplayArtifactPanel.test.tsx` — a `file` artifact whose doc is `documentType: 'table'` renders the grid, not the `<pre>` fallback
- [x] `apps/mail/tests/modules/files/graph/transform.test.ts` — a `table` document maps to the `table` node kind
- [x] `timeout 400 pnpm --filter @zero/mail run test tests/modules/documents/table tests/modules/home tests/modules/files` (apps/mail runs JEST, not vitest)
**Implementation notes — deviations and why:**
- **"New table" is create-then-seed, not create-with-markdown.** `files.createFile` provisions the
row with `documentType: 'table'` and empty content; the starter schema then arrives as an ordinary
Y.js update through `files.applyUpdate` (`starter-table.ts`). Writing the frontmatter + pipe header
from the client instead would put a second copy of the mirror grammar in `apps/mail`, which is
exactly what `table-markdown.ts` exists to own. `getDoc`'s lazy `content_yjs` backfill means the
ordering is safe either way.
- **The new table lands where the user asked, not at `user/tables/{slug}`.** `CreateNodePopover` is
the file tree's create affordance and is always invoked against a scope and a parent folder;
forcing every table into one directory would ignore the folder the user opened the menu in.
`user/tables/{name}` remains the AGENT's default (§3.2 step 7), which is where it matters — that
is the path a fan-out addresses.
- **`resolveOpenDoc` needed no `table` case; the RENDERER did.** A conversation-scoped table is an
ordinary entry in `nonAgentDocs` and already resolves by id. What is wrong is routing it through
`OverviewDocTab`, which re-resolves its document by `{ documentType, path }` — a find-or-create —
and a table whose rows live in the Y.Doc must never be re-provisioned by a read. `FilesTab` now
mounts `TableDocumentView` on the id it already has, gated by an exported `isTableDoc` predicate.
- **The share page renders the markdown mirror, not the grid.** It is unauthenticated: no session,
no tRPC client, no Y.js provider, and no way to resolve `[[conversation: id]]` into a name. So
`TableSharePreview` parses the mirror's pipe table (escaped `\|` included) and degrades every
reference to its KIND rather than leaking an internal id as content — the "read the mirror
directly" escape hatch §3.2 step 9 names for exactly this surface.
- **"N rows × M columns" lands in the file tree but not yet in doc-link results.**
`files.searchForLink` does not select `documents.metadata`, so `FileLinkSuggestionItem.metadata` is
declared and read (`formatTableSummary`) but always absent; the popover shows the table icon today
and the shape as soon as the route returns the column.
- **Server exports still needed.** Moving `parseCellRefs` / `formatCellRef` (and the
`CEDAR_DOC_REFERENCE_REGEX` they use) into `table-types.ts` — already the shared client+server
surface behind `@zero/server/table` — would delete `cell-refs.ts`'s grammar restatement outright.
Adding `metadata` to `files.searchForLink` closes the doc-link gap above.
- **The graph node caption is one constant.** Four node components repeated the same label class
string; `GRAPH_NODE_LABEL` in `scope-styles.ts` holds it, and `TableNode` (squared off, so a table
is findable at graph zoom where the caption is not) is the fifth consumer rather than the fifth
copy.
### Phase 8 — Excel export
**Goal:** Any table downloads as a well-formatted `.xlsx` that carries enough metadata to be
re-imported losslessly.
- [x] Create `apps/server/src/services/documents/table/table-excel.ts` exporting `tableToWorkbook(…)` (+ `tableToCsv`)
- [x] Map column types to Excel formats (numFmt for number/currency/percent, date format, boolean, data validation from `column.options`)
- [x] Write the hidden `_id` first column and the hidden `_cedar` schema sheet
- [x] Resolve ref labels server-side (batch by `refType`, in `table-ref-labels.ts`) and write display text + hyperlink + the raw `[[type: id]]` token as a cell note
- [x] Add `documents.exportTable` to `apps/server/src/trpc/routes/documents.ts` returning a base64 workbook, plus the admin-facing `exportTableDocument` used by the CLI
- [x] Add the headless driver: `cedar-cli table export --path <p> [--format xlsx|csv] --out <file>`
- [x] Add "Download as Excel" to the `FileEditor` dropdown, shown only for `documentType === 'table'` — landed with Phase 7, alongside a CSV item and the same pair on the table's own toolbar (`download-table.ts`)
- [x] Add CSV export using the same serializer path with refs flattened to their labels
**Tests:**
- [x] `apps/server/src/services/documents/table/__tests__/table-excel.test.ts` — exported workbook has the hidden `_id` column, the `_cedar` sheet, and one data row per table row
- [x] Same file — a `currency` column carries its numFmt and a `select` column carries its validation list
- [x] Same file — an unresolvable ref falls back to its raw token without throwing
- [x] `timeout 300 pnpm --filter @zero/server exec vitest run src/services/documents/table/__tests__/table-excel.test.ts`
**Headless verification:** `cedar-cli table export --out out.xlsx` produced a real 9,671-byte
`Microsoft Excel 2007+` file. Read back with exceljs: sheets `['Data', '_cedar (hidden)']`,
column A hidden and holding the `_id`s, one data row per table row, the raw
`[[conversation: …]]` / `[[draft: …]]` tokens preserved in cell notes, and the schema JSON
recovered from `_cedar` with all five columns and their types.
**Implementation note — a latent bug this surfaced.** The first export failed with
`Workbook is not a constructor`. exceljs is CommonJS, and under native ESM (how `tsx` runs
the dev server) `await import('exceljs')` yields a namespace whose only key is `default` —
so the `const { Workbook } = await import('exceljs')` idiom binds `undefined`. The same
idiom was already in use at
[extraction.ts](apps/server/src/services/file-system/uploads/extraction.ts) for
spreadsheet upload text extraction, so that path was broken under ESM too; both now accept
`mod.Workbook ?? mod.default?.Workbook`. A bundled build synthesizes the named export,
which is why this only ever failed in dev.
### Phase 9 — Excel/CSV import
**Goal:** An uploaded spreadsheet converts into a live table document, and a Cedar-exported workbook
re-imports without losing rows or references.
- [x] Add `workbookToTableData(buffer, opts)` — kept in `table-excel.ts` alongside the writer on purpose: the two must agree on three conventions (`_id` is column 1, schema at `_cedar!A1`, a ref's token lives in the cell NOTE), and a disagreement between them *is* the bug that silently drops rowIds. One module makes the agreement structural
- [x] Read the `_cedar` sheet when present and preserve `_id`s; otherwise infer columns from the header row plus a value scan
- [x] Recover `[[type: id]]` tokens from cell notes when present, falling back to plain text — and only when the note actually contains a token, so a human's comment never becomes the cell value
- [x] Add `documents.importTable` taking `{ attachmentDocumentId, sheetName?, targetTableDocumentId?, path?, deleteMissingRows? }`, plus `admin.documents.importTableFromBase64` for the CLI (both call one `importTableDocument`)
- [x] Implement update-in-place: match rows by `_id` and emit `set` / `add_rows` / `delete_rows` through the backend (`table-diff.ts`)
- [x] Expose import as an agent capability — an `import` action on the `table` tool, classified as mutating in `scope-map.ts`
- [x] Add the headless driver: `cedar-cli table import --file <f> [--update-in-place] [--keep-missing-rows]`
- [x] Add "Convert to table" to the attachment viewer for `spreadsheet`-category uploads (gated on the shared upload allowlist's `spreadsheet` category; opens the created table on success)
**Tests:**
- [x] `apps/server/src/services/documents/table/__tests__/table-import.test.ts` — 20 tests, including export → import identity (schema `toEqual`, both rowIds, `[[conversation:]]` *and* `[[draft:]]` recovered from notes)
- [x] Same file — a plain spreadsheet with no `_cedar` sheet infers `number` / `date` / `select` / `text` correctly, and three-values-in-four-rows is NOT a select
- [x] Same file — re-importing an edited export into an existing table updates matched rows and appends only genuinely new ones, asserted on **Y.Map identity** (`findRowMap(ydoc,'r_01') === before`), which a wholesale replace would fail while still passing every value check
- [x] Same file — renamed-header round trip, CSV round trip with quoted commas / escaped quotes / CRLF, real Excel number/Date/boolean cells, `_status`/`_error` round trip, and the zero-op no-write case
- [x] `timeout 300 pnpm --filter @zero/server exec vitest run src/services/documents/table`
**Headless verification:**
```
table export → rt.xlsx; table import --path <new> → 3 rows; `read` diff vs the original is
IDENTICAL but for the header line
edit the workbook (change 1 cell, delete 1 row, add 1),
then table import --update-in-place → updated [r_01], deleted [r_02],
added [imported_1], cellsChanged: 1
```
The untouched row kept its `[[conversation:]]` ref and instruction, and `cellsChanged: 1` shows
the update was surgical rather than a rewrite.
**Implementation notes:**
- **Missing-row semantics.** A row in the table but absent from the workbook is deleted **only when
the workbook carried `_id`s** — a Cedar export coming home is a real snapshot of the row set,
whereas a plain sheet has no opinion about which existing row is which. `--keep-missing-rows`
opts out either way. Two further safety rules: a column the workbook did not show is never
cleared, and a column the table has but the workbook lacks stays declared.
- **Inference needed stricter predicates than the export's coercers.** `parseNumeric('Company 0')`
returns `0` (it strips a leading symbol on purpose, for declared currency columns), which typed a
text column as `number`. Inference therefore uses whole-string `looksNumeric` and a shape-checked
`looksLikeDate`.
- **Duplicate `_id`s in a sheet** (an Excel row copy-paste) get a fresh id plus a warning, rather
than making later `set`s ambiguous.
- `coerceColumn` is exported from `table-ydoc.ts` and reused: the `_cedar` cell is untrusted JSON a
human can edit, so it needs exactly the same field-by-field validation, not a second copy.
### Phase 10 — Query-materialized rows and bound columns
**Goal:** Rows come from a real query rather than the model transcribing them, and columns can hold
live Cedar values that stay correct without anyone rewriting them.
- [x] Add `TableRowSource` and the `from` clause to `create` in `apps/server/src/services/documents/table/table-types.ts`
- [x] Implement `materializeRowsFromSource` in `apps/server/src/services/documents/table/table-source.ts` — resolve the query, mint one row per result, and write the `[[<source>: id]]` ref into `intoColumn` (defaulting to the first ref column)
- [x] Route `source: 'conversations'` through the same filter surface as `apps/server/src/mastra/tools/conversation/findCRMConversationsTool.ts` rather than a second query language
- [x] Enforce the row cap at materialization with a clear error naming the result count, so an over-cap filter fails fast instead of half-building a table
- [x] Add `binding` to `TableColumn` and implement `resolveBindings(rows, schema)` in `apps/server/src/services/documents/table/table-bindings.ts`, resolving each path against the row's ref cells
- [x] Support `conversation.*` paths including `conversation.fields.<customFieldKey>` for user-defined CRM fields, batching one query per entity type rather than per row
- [x] Write resolved values as ordinary cell values so bound cells round-trip through the markdown mirror, Excel export, and the SSE path with no special casing
- [x] Serialize the binding in the frontmatter column declaration (the `<f>` half) while the cell holds the cached value (the `<v>` half), mirroring how a `.xlsx` cell stores both
- [x] Refresh bindings on `create` and on `read` for the rows returned, bounded by the row cap
- [x] Add entity-change invalidation: on a `crm_conversation_updates` insert, find tables holding that ref and recompute only the bound cells of only the affected rows
- [x] Render bound cells read-only in `TableGrid` with a subtle derived-value affordance, and add an "unbind column" action that keeps the last computed values — `BoundCell` is the single read-only rendering for every column type (bound columns are intercepted in `TableCell` before the typed editors), and it stays a tab stop so the derived value is reachable by keyboard without opening an editor; `unbindColumn` in `schema-edits.ts` drops only the `binding`, so the cells Cedar already computed stay put and become editable
- [x] **Linked columns** — a bound cell over a writable conversation field is edited THROUGH the binding rather than being read-only: `resolveBoundWrite` (`bound-writes.ts`) resolves the row's deal and the field, `useBoundCellWrite` coerces the cell string and sends `crm.updateConversation` / `crm.upsertWorkingMemory`, and the invalidation those already fire recomputes the cell. `BoundCellEditors` picks the editor off the FIELD (the AOP stage/priority picker, the natural-language date picker) rather than off the column type, because a stage column is declared `text`. Every gesture obeys it — double-click, Enter, a Delete over a range, a paste — since "linked" is a property of the column. `parseBindingPath` moved to `table-binding-paths.ts` and is published as `@zero/server/table/bindings`, so the grid resolves the path with the server's own parser instead of a second copy of the grammar.
- [x] Move the link affordance from the cell to the COLUMN HEADER — one glyph per bound column that severs the link (confirmed), instead of a glyph in every cell of it; a read-only derived cell stays muted, a linked one is drawn like any other
- [x] Export bindings to the hidden `_cedar` sheet in `table-excel.ts` so an export → edit → import round trip restores them
**Tests:**
- [x] `apps/server/src/services/documents/table/__tests__/table-source.test.ts` — a conversation filter materializes one row per result with the ref cell populated, and a result set over the cap errors without creating a partial table
- [x] `apps/server/src/services/documents/table/__tests__/table-bindings.test.ts` — `conversation.nextStepDate` resolves from the row's ref, and an unresolvable path reports an error without writing to the cell
- [x] Same file — resolution batches one query per entity type regardless of row count
- [x] Same file — updating the underlying conversation recomputes only that row's bound cells and leaves every other row's Y.Map untouched
- [x] Same file — a bound column round-trips through markdown mirror and Excel export → import with both the binding and the cached value intact
- [x] `apps/mail/tests/modules/documents/table/BoundColumns.test.tsx` — a bound cell with nowhere to write renders the derived affordance and never becomes an editor on click, double-click or focus while a non-bound cell still does; it remains a tab stop; "Sever link" removes the binding, leaves the computed value intact, and the cell then commits a hand-typed value
- [x] `apps/mail/tests/modules/documents/table/bound-writes.test.ts` — the resolution: a `stage` binding maps to the `status` column it aliases, a column-form path reads its named ref, a `fields.<key>` path goes to working memory, and a joined field, a non-conversation entity and a row with no deal all resolve to nothing; the writable set is a subset of the server's read whitelist
- [x] `apps/mail/tests/modules/documents/table/LinkedColumns.test.tsx` — a linked cell writes the Y cell AND sends `crm.updateConversation` for the row's deal (typed, toggled, date-picked, and Delete-cleared), a rejected write puts the previous value back, and a derived cell in the same row still refuses to open
- [x] `timeout 300 pnpm --filter @zero/server exec vitest run src/services/documents/table`
**Headless verification (against real CRM data):**
```
table create --from-file '{ "source": "conversations", "limit": 5, "intoColumn": "deal" }'
→ matched 1745, materialized 5, boundCells 1, bindingErrors []
table read → 5 rows carrying REAL company names and real [[conversation: uuid]] refs;
`Next step date` = 2026-01-01 for the one conversation that has one
```
Cross-checked against `crm_conversations`: `status` is genuinely NULL for all five rows and
`next_step_date` is set only for the one that shows a date — so `boundCells: 1` is accurate
and the empty cells are absent data, not a resolution failure. A null value writes an empty
cell; only an *unresolvable path* is reported as a `bindingError`.
### Phase 11 — The table as the default parallel-work primitive
**Goal:** The agent reaches for a table whenever a task has N independent units of work, and the
three canonical workflows are verified end to end.
- [x] Build the per-row brief in `runTableFill` from the row's cells, the schema (each column's `label` + `fillInstruction`), and the writable column set — no role taxonomy, no per-table configuration (landed in Phase 5)
- [x] Add `TABLE_TEMPLATES` (work-queue, pipeline-analysis, outbound-sequence) as suggested column sets — labels, types, and `fillInstruction`s only — in `table-templates.ts`
- [x] Add a `<parallel_work>` section to `apps/server/src/mastra/prompts/cedar-docs-awareness.ts`: when a task has more than ~5 independent units, create a table with a row per unit and fan out, rather than looping tool calls
- [x] State the progress convention in that prompt — the assigned agent writes its own `status` cell as it goes (`🔍 researching` → `✍️ drafting` → `✅`), and an explicit skip is a written value with a `reasoning`, never a blank. The prompt also states rows-come-from-a-query, rowId-over-ordinal addressing, and the do-NOT-use-for-trivial-work threshold
- [x] Document the pattern in the `documents` skill at `apps/server/.claude/skills/documents/SKILL.md` with the three worked examples (post-release outreach, pipeline analysis, multi-step outbound), the non-obvious rules, and a pointer at `TABLE_TEMPLATES` as the known-good starting column sets
- [x] Make the `write-document` result surface a table as a clickable card showing "N rows × M columns", so a fan-out the user did not explicitly open still lands in the transcript
- [x] Auto-open the table as the selected artifact when a fan-out starts. No single stream event carries both facts — the `fill` tool-call knows a fan-out began but holds only the model-supplied path (pre-resolution), while the `docUpdate`s that follow carry the documentId but are emitted by EVERY table write. So the fill's tool-call arms a latch and the first `docUpdate` while armed opens that document, disarmed only by the fill's own `toolCallId` (a row agent reads before it writes, so disarming on any `table` result would kill the latch one event early). Follow-up worth doing: have the `fill` branch emit an explicit `tableFillStarted { documentId }` before `runTableFill`, which removes the correlation and opens the grid the instant the fan-out starts rather than on its first row's write
**Tests:**
- [x] `apps/server/src/services/documents/table/__tests__/table-brief.test.ts` — the per-row brief carries the row's cells, every column's label and `fillInstruction`, and only the writable column keys
- [x] `apps/server/src/mastra/__tests__/table-fanout-eval.test.ts` — **workflow 1**: given "I just released a feature, look at all my customers", the agent creates the table via `from` (not by transcribing rows), then fans out, leaving a `[[draft: id]]` or an explicit skip with reasoning in every row
- [x] Same file — **workflow 2**: "pull every deal closing this quarter with CRM fields X/Y/Z plus ICP fit" produces one row per deal and no empty required cells
- [x] Same file — **workflow 3**: "take this list, find their LinkedIn, connect" produces one row per person with a populated `status`
- [x] Same file — a task with 2 units of work does NOT create a table (the primitive must not fire on trivial work)
- [x] Run these as fire-rate evals over N trials via the `playbook-instruction-eval` skill, not as single-shot assertions
**Eval results — the design's own prediction was right, the prompt as first written did not
prevent it, and the fix is measured.** Gated on `EVAL_TABLE_FANOUT=1`; claude-sonnet-5,
6 trials/scenario. Note the eval needs `ANTHROPIC_API_KEY` exported (vitest does not read
`.env`) and `--disable-console-intercept`, or the fire-rate summary — the actual product of an
eval — never prints.
Authoritative numbers are the **12-trial** run; the 6-trial figures are kept only to show what
the prompt edits moved, because n=6 turned out to mislead in BOTH directions (see the limitation
note below).
| metric | as first written (n=6) | shipped now (n=12) | needs |
| --- | --- | --- | --- |
| workflow 1 — rows via `from` | 4/6 (67%) | **12/12 (100%)** | ≥ 80% ✅ |
| workflow 1 — transcribed rows (lower is better) | 3/6 (50%) | **2/12 (17%)** | ≤ 20% ✅ |
| workflow 1 — fanned out | 6/6 | **12/12 (100%)** | ≥ 60% ✅ |
| workflow 2 — rows via `from` | 2/6 (33%) | **12/12 (100%)** | ≥ 80% ✅ |
| workflow 2 — every requested field declared | 6/6 | **12/12 (100%)** | ≥ 80% ✅ |
| workflow 2 — fill covers the derived column | 4/6 | **10/12 (83%)** | ≥ 60% ✅ |
| workflow 3 — one row per person | 6/6 | **12/12 (100%)** | ≥ 80% ✅ |
| workflow 3 — `status` owned by the row agent | 2/6 (33%) | **9/12 (75%)** | ≥ 60% ✅ |
| a column marked `required` | 1/6 (17%) | 7/12 (58%) | informational |
| negative control — any table call | 0/6 | **0/12** | 0 ✅ |
All five scenarios pass at n=12.
The strongest result is the negative control: across every variant and run, the primitive
**never once** fired on a 2-unit task. It is not an over-firing feature.
The failure mechanism was visible in every early trace: the agent called
`find-crm-conversations` FIRST, and once 34 rows sat in its context, transcribing them and then
doing the per-row work itself was cheaper than delegating. The prompt stated the provenance
rule but forbade neither the fetch-then-copy ordering nor the serial `set` loop. Three
prohibitions now in `<parallel_work>`, each earning its place in the numbers above:
*do not fetch the rows yourself*; *filling is `fill`, not a loop of `set`* (put `status` in
`fill.columns`); and *pass `from` OR literal rows, never both* — which took transcription from
2/6 to 0/6, since the agent had been passing `from` and pasting rows on top of it, duplicating
every row. A fourth line — *mark the columns a row is not done without as `required`* — made
"no empty required cells" expressible at all, moving from 1/6 to 5–6/6.
**Six trials misled in both directions, which is why the default is now 12.** A 6-trial run of
the shipped prompt reported workflow 2's fan-out at 3/6 (50%) — under its bar — while n=12 puts
it at 10/12 (83%). The prompt edit in between concerned duplicate rows and had no plausible
mechanism for touching fan-out, which is what flagged it as noise; n=12 confirmed that. The same
run flattered two metrics: transcription read 0/6 when the truth is ~17%, and `required` read
5–6/6 when the truth is ~58%. So n=6 was not merely imprecise, it produced a number too good and
a number too bad in the same run. Treat any single small-n result as a hypothesis and re-measure
before acting on it.
`PROPOSED_PARALLEL_WORK_PATCH` is now empty on purpose — its contents shipped — but the
`EVAL_VARIANT` knob stays, so the next prompt change is arguable with numbers instead of taste.
**Row agents run on a selectable model tier.** Every subagent spawn was hardcoded to Sonnet —
`createOrchestratorAgent` passed `modelId: SONNET_MODEL` literally, and neither
`OrchestratorParams` nor `spawn-subagent` exposed a choice. That is a cost bug specific to a
fan-out, because the model is paid for PER ROW: a 500-row fill runs 500 subagents, so the tier is
multiplied by the row count rather than paid once, and "write `[[draft: id]]` into a cell from a
supplied instruction" is not Sonnet work.
`fill.modelTier: 'fast' | 'standard'` now threads through
`RowAgentSpawnContext → OrchestratorParams → createOrchestratorAgent → resolveModel`, defaulting
to `standard` so every pre-existing spawn is byte-identical.
Deliberately a TIER, not a model id. `services/llm/models.ts` describes itself as "the ONE place
model versions are defined", and a caller — least of all a model writing a tool call — passing
`claude-haiku-4-5` would hardcode a version outside it, surviving neither the sonnet-5 → 4.6
rollback that file records nor a future Haiku bump. A tier names the *intent* ("this row's job is
mechanical") and lets the mapping move underneath it. `modelForSubagentTier` degrades an unknown
tier to `standard`, because such a value can arrive from a model's tool call or a persisted row
and resolving to `undefined` would surface as an opaque provider error at request time.
Two adjacent gaps found while doing this, NOT fixed here:
- **`aop_agents.model` is carried and dropped.** `aop-agents.ts:114` synthesizes `model` from a
subagent doc's frontmatter, but nothing reads it at spawn time — so a playbook subagent's
declared model has no effect. The plumbing added here is what it would hang off: map that
frontmatter value to a tier and pass it as `modelTier`.
- **The analyzer path is untouched.** Only the orchestrator (the one a fan-out uses) takes a tier;
`createAnalyzerAgent` still hardcodes its own model. Symmetric change, no caller needs it yet.
**Unmeasured, and worth saying plainly:** the eval harness measures the MAIN agent's tool
selection, not a row agent's output quality, so nothing here shows `fast` is *good enough* for a
mechanical fill. That needs a different eval — same trials-over-N shape, scoring filled cells
rather than tool calls. Until it exists, `standard` remaining the default is the safe posture and
`fast` is an informed opt-in.
**A claim is a LEASE, because a fan-out can outlive its parent.** The relevant timeouts: the ALB
idles a connection at **300 s** (bytes, not wall clock — the chat stream heartbeats, so a long turn
survives) and the chat agent caps at **maxTurns: 60**. A background task gets its declared
`timeoutMinutes` (5–60). A 40-row fan-out with research per row can exceed any of those.
`_status` transitions are written by the orchestrator, so a dead SUBAGENT still lands `failed` —
that part held. But if the ORCHESTRATOR died (turn timeout, deploy, `maxTurns`), every row it had
marked `running` stayed `running`, and `selectRowsToFill` reclaimed only `''` / `pending` /
`failed`. Those rows were **stranded permanently**, recoverable only by a human editing a cell.
The design's claim that "a crashed subagent still leaves an accurate row state" was true of the
subagent and false of the orchestrator.
Fixed with a reserved `_claimed_at` timestamp written alongside `running` and cleared on a
terminal state: a `running` row whose claim is older than `STALE_CLAIM_MS` (1 h) is reclaimable, so
an interrupted fan-out resumes on the next re-run. A missing or unparseable stamp counts as stale —
rows claimed before the field existed would otherwise be unreclaimable forever, and a duplicate
fill is the cheaper mistake than a lost row, since every write is keyed by `rowId`.
Verified headlessly: with `r_01` unstarted, `r_02` `running` on a 2-hour-old claim and `r_03`
`running` on a fresh one, a re-run spawned exactly 2 — `r_01` and `r_02` — leaving the live claim
alone. Before the fix `r_02` was unrecoverable.
**What makes the long workflow worth it is precisely this**: progress lives in cells, not in a
chat turn. A timeout loses the *narration*, never the work — which is the property a chat message
summarizing 40 results does not have.
### Phase 12 — Long-lived rows, and keeping a churned table small
**Goal:** A row can be a durable state machine — "connect on LinkedIn, wait, then message once they
accept" — rather than a one-shot fill. The user re-prompts to advance it; there is deliberately no
scheduler (see below).
- [x] Extend `TableFillRequest` with a `where` filter so a run targets rows in a given `status` rather than the whole table
- [x] Add `runTableFill` resumability: a run over a `where` filter is idempotent and safe to repeat, since every write is keyed by `rowId`
- [ ] ~~Register a scheduled re-run~~ — **deliberately not built.** A schedule is a lot of machinery for the value: a cron pass, a KV registration and its drain, a run lease, two stop conditions, four control procedures, a toolbar that polls, and an unattended loop spawning subagents with nobody watching. All of it exists to save the user from re-prompting. Re-prompting is cheap, and it keeps a human in front of every fan-out — which for a feature whose unit of work is "spawn N agents that write to the CRM and draft outbound" is a feature, not a limitation. A long-lived row still works exactly as designed: the row's own `status` column holds where it is, and the user says "advance the ones that are connected" the next day, which runs the same `where`-filtered fill a schedule would have. Revisit if users actually ask for it, and if a durable agent-work queue lands that the fan-out can ride rather than owning its own scheduler.
- [ ] ~~Drive LinkedIn steps through the existing outbound engine~~ — **deliberately not built, and this is a design change rather than an omission.** Outbound's send seam takes `enrollmentId` + `stepId` + `contactId` + a chosen seat, dispatched over enrollments in a sequence. Handing it a table row would mean minting records, lists, enrollments and seat selection FROM rows — a new integration surface, not a dispatch. So the step stays generic: the row's own `status` column plus its `fillInstruction` is enough for a filling agent to drive the channel, and a later `where`-filtered fill reads that column back to pick up rows ready for their next step. Revisit only if a second channel wants the same treatment, which is when the abstraction would earn itself.
- [x] **Detect** bloat inline and for free: `tableStatsHook` computes `bloatRatio = contentYjsBytes / expectedTableBytes(data)` and sets `metadata.compactionDue` past 4×, logging at `warn`. Detection does no work — it only raises a flag
- [x] Rebuild from a fresh `Y.Doc` containing only the current schema and rows — `compactTableYDoc` in `apps/server/src/services/documents/table/table-compaction.ts`
- [x] **Broadcast a replace, not a delta.** Added `replace?: boolean` to `DocUpdatePayload` in `apps/server/src/services/documents/doc-event-bus.ts`, documented with the reason: a compacted doc has an entirely new clientID and clock space, so a merging client converges on BOTH copies of every row. A test demonstrates the wrong outcome (3 rows → 6) to pin it. The flag has to ride BOTH transports: the in-process `docEventBus` publish almost never reaches the task holding the user's doc-events connection (which is why `TableWriteResult.broadcast` exists at all), so `broadcast.replace` carries it on the chat stream and `docUpdateResponseProcessor` applies it through `applyServerReplace`. Setting it only on the bus meant a compaction reached an open grid as an ordinary delta and the client merged it
- [x] Skip compaction when `docEventBus.listenerCount(documentId) > 0` (`canCompactNow`) and defer to the next eligible write, so the common case never risks discarding a viewer's unflushed local edits. Worth knowing how weak this gate is on its own: the count is per Node process, and in prod the process holding the user's connection is almost never the one doing the write — so it is `replaceStateFromRevision` and the `replace` flag, not this, that make compaction safe when someone IS watching
- [x] **Execute** at the start of the next write when the flag is set, and write a `reason: 'compaction'` row to `document_snapshots` first so the pre-compaction state stays restorable. The trigger is wired into `writeTableOnce`, and the replacement is guarded by `replaceStateFromRevision` — a compare-and-swap checked under the advisory lock, because everything the rebuild was computed from is read BEFORE that lock exists, so a cell edit committing in the window was inside the state being replaced and not inside the replacement. On a mismatch the write retries once with compaction suppressed
- [x] Client side: apply `fullUpdate` through `CedarYjsProvider.replaceWithServerState` when the SSE frame carries `replace: true`
- [x] Add a manual "Compact" action to the table toolbar — the UI has landed in `TableDocumentView` (behind the toolbar's maintenance menu, reporting the bytes reclaimed) and calls `documents.compactTable({ documentId })` → `{ success, bytesBefore, bytesAfter, reclaimedBytes }` through the narrowed client interface in `apps/mail/modules/documents/table/compact-table.ts`. `admin.tables.compact` is staff-only (`cedarAdminProcedure`) and addresses the table by `path`, so the toolbar has its own `privateProcedure` wrapper over the same `compactTableDocument` service, keyed by `documentId` and forcing past the threshold. `compacted` absent maps to `success: false`, which the toolbar reports as deferred rather than done
- [ ] Alert on tables whose `contentYjsBytes` grows while `rowCount` is flat — **the code side is done and the signal is emitted**: `tableStatsHook` logs `[tableStats] compaction due` at `warn` with `bloatRatio`, `contentYjsBytes`, `expectedBytes`, `rowCount` and `columnCount`, which is exactly the shape an Axiom monitor needs. Creating the monitor itself is live monitoring configuration with an owner and a notification target, so it is left as an explicit operator step rather than something this branch creates.
**Tests:**
- [x] `apps/server/src/services/documents/table/__tests__/table-compaction.test.ts` — 16 tests: a rebuilt doc shrinks back toward its materialized size with every current value intact; the source doc is untouched; the merge-instead-of-replace bug is demonstrated (3 rows become 6); an empty table compacts without throwing; `canCompactNow` defers while a subscriber is attached
- [x] Same file — **single-cell churn is proven NOT to be a bloat problem**, and interleaved churn is proven to be one; a normally-edited table is not flagged; the estimated denominator is checked against a real compacted encode
- [x] `apps/mail/tests/modules/documents/table/TableCompact.test.tsx` — the toolbar action calls the procedure for the open document and reports the bytes reclaimed; "already compact", a server-side deferral and a failure each surface as their own message
- [x] `timeout 300 pnpm --filter @zero/server exec vitest run src/services/documents/table/__tests__/table-compaction.test.ts`
**Implementation note — why the denominator is an estimate.** A true ratio needs the compacted
size, which means an actual rebuild: measured at 20–180 ms for 200–1,000 rows, far too slow to run
on every save. So `expectedTableBytes` estimates it, calibrated against those measurements
(~10 bytes of framing per map entry, ~12 per row), landing within a few percent of a real compacted
encode at 1,000 rows and erring LOW — which delays a compaction rather than triggering a spurious
one. A test asserts the estimate stays within 40% of a real rebuild so the calibration cannot rot.
**Remaining for this phase (not started):**
- [x] `table-fill.test.ts` — a `where`-filtered run touches only matching rows and re-running it changes nothing, which is what makes a user-driven re-prompt the next day safe; a malformed filter is refused rather than matching every row
- [x] A `reason: 'compaction'` snapshot written before the rebuild, with tests that the captured bytes RESTORE the pre-compaction document (deep-equal rows/schema, `byteLength === bytesBefore`), that it stays a valid point-in-time restore after later writes land on the compacted doc, and that a restore must REPLACE — merging the snapshot back yields 8 rows from 4, the mirror image of the broadcast-direction hazard
- [x] An end-to-end test that a compacting save's broadcast carries `replace: true`
### Phase 13 — Output cells for LinkedIn and WhatsApp
**Goal:** A fan-out over people can draft the message it would actually send them. `linkedin` and
`whatsapp` were already in `TABLE_OUTPUT_KIND` — a column could declare them, the header menu
offered them, the badge rendered — but the payload behind both fell into `GenericCellOutput`
(`{ kind, message?, sentAt? }`), which holds the words and nothing that says who they are for. So
the cell could not name a recipient, the pill was never clickable, and the Send button was
permanently disabled: `canSendCellOutput` narrowed to `SlackCellOutput` and nothing else.
**Two addresses per channel, because the providers genuinely have two.** `chatId` names a
conversation that exists; `attendeeProviderId` (LinkedIn) and `phoneE164` (WhatsApp) name a
PERSON. Requiring a chat would make the column fillable only for people you have already spoken
to, which is the opposite of the case a lead-list fan-out is for — and both tRPC routes already
accept either. The pill is the one place the two differ: a cell with no chat yet can be SENT (the
provider opens the conversation on the way) but there is nowhere to navigate to, so it renders as
a plain pill rather than one that would land on a chat id that does not exist.
- [x] `LinkedinCellOutput` / `WhatsappCellOutput` in `cell-output.ts`, mirroring `LinkedinTaskOutput` / `WhatsappTaskOutput` field for field plus a `recipientName` — the precedent is Slack's `channelName`, and without it a column of DMs is a column of Unipile chat ids
- [x] Both removed from `GenericCellOutput`'s `Exclude`, and `ChatCellOutput` added as the union of the three kinds whose payload is a message to a person — `summarizeCellOutput`, `describeCellOutputTarget` and the Send affordance all dispatch on it rather than on three separate equality checks
- [x] `canSendCellOutput` (a type guard onto a payload whose every field is still optional) replaced by `sendableCellOutput`, which RESOLVES the cell into a `SendableCellOutput` whose fields are required. That is what removed the three `!` assertions from the send site — narrowing to `SlackCellOutput` and then asserting `workspaceId!` is exactly the shape that ships `undefined` to a provider API
- [x] `describeUnsendable` names the missing piece per channel, so a disabled Send says "Needs a chat or a phone number" rather than being inert
- [x] `OutputCellEditor` gained `linkedin.messaging.sendDm` and `outbound.whatsapp.sendMessage` beside the Slack mutation, with one `stampSent` success path — `sentTs` is Slack's alone, since Unipile's id arrives later on the ingest path rather than in the response
- [x] `open-output-in-channel.ts` generalized: `openableOutput` resolves any cell to `{ param, containerKey, message, sent }` and the hook navigates `?slack=` / `?linkedin=` / `?whatsapp=`, all three of which `LayoutUrlSync` already projects and restores
- [x] `mail.tsx` adopts the feed filter for all three deep links, not just `?slack=` — without it a LinkedIn link landed on the email list, set the artifact, and opened nothing
- [x] `ChannelThreadView` takes its composer handoff key from the container id on every channel (`ref.slackChannelId` or `ref.chatId`), where it used to pass `null` for anything but Slack — which is why a LinkedIn draft handed over from a table arrived at an empty composer
- [x] `table-row-agent.ts` and the `tables` skill state both payloads, and that a `sentAt` an agent writes would be a row claiming a human sent it
**Not done, and worth naming:** the persisted `linkedin_chats.draft` / `whatsapp_chats.draft` is
still write-only — `saveChannelDraft` writes it and the feed surfaces `hasDraft`, but nothing reads
the body back into the composer. That predates this work and is why the handoff still goes through
the in-memory seed rather than the durable draft.
**Tests:**
- [x] `apps/mail/tests/modules/documents/table/cell-output.test.ts` — 14 tests: each channel's minimum for a send, the two addresses, a seat with nowhere to land, a sent cell never offering a second send, and the untrusted-JSON field filter (a numeric `attendeeProviderId` is dropped rather than forwarded)
- [x] `apps/mail/tests/modules/documents/table/OutputTargetPill.test.tsx` — the LinkedIn/WhatsApp deep links and their seeded composer, the recipient name over the raw address, and the DM addressed only by a person rendering as a plain pill
- [x] `cd apps/mail && npx jest tests/modules/documents/table`
### Phase 14 — Sorting a table by a column
**Goal:** Order the rows by any column, in the UI and through `read`, the way that column's TYPE
orders — because the naive string sort is wrong in exactly the cases people sort for: `$1,200`
lands before `$300`, and a `select` column of pipeline stages comes back Closed, Demo, Discovery.
**The sort is a VIEW, and that is the load-bearing decision.** `schema.sort` orders the handle
list `useYTable` hands the grid; `tableRows` keeps document order. So `_id` addressing, the
ordinal/A1 forms, `moveRow` and every row's CRDT identity are exactly what they were, and a sort
costs one Y.Map entry rather than a rewrite of the row array. Two consequences fall straight out
of it: the row drag handle is withdrawn while sorted (a drag writes a position in document order,
and the position dropped on is a position in the view), and `read` does NOT silently adopt the
declared sort — an ordinal resolves against document order on every backend, so a read that
quietly reordered itself would point `4.outreach` at a row the agent never looked at. It reports
the table's sort in `degradations` instead and leaves the choice explicit.
- [x] `TableSort` + `TableSchema.sort` + `coerceTableSort` in `table-types.ts`, and an arm on all four surfaces a table schema is CLOSED on — `readTableSchema`/`writeTableSchema`, the markdown mirror's frontmatter (as one JSON line, like `triggers`), the client's `parseTableSchema`, and the workbook's `_cedar` sheet. A key with no arm on one of them is dropped silently on the way through, which is how a sort ends up "not sticking" for reasons nobody can see from the grid
- [x] `table-sort.ts` (exported as `@zero/server/table/sort`) — one comparator, shared by the grid and by `read`, dispatching on `column.type`: numeric for number/currency/percent, the instant for a date, declared-option order for select/multi-select, the fan-out lifecycle for `_status`, artifact state for `output`, visible-label-then-id for the reference columns. Empty is last in BOTH directions, handled before the direction is applied — negating a comparator that had already ranked empties floats them to the top on `desc`
- [x] The module also owns the tolerant readers the comparator needs (`readCellNumber` / `readCellDate` / `readCellBoolean`). Those existed twice before — the Excel writer's and the grid's — and had already drifted; both now delegate, which is what makes `cell-values.ts`'s claim to be the only place a cell's string form is interpreted true
- [x] `readCellDate` takes the column's own `format`: a `dd/MM/yyyy` column holds `03/04/2026` meaning the 3rd of April, and `new Date` reads it as the 4th of March — a comparator that disagreed with the cell's renderer about which day a value is would sort a quarter into a visibly wrong order
- [x] `useYTable` sorts the handle list, and its `observeDeep` bailout learned one more question: a write to the SORTED column is the one cell write whose effect is structural, so it is allowed through while every other one still bails on the first line. A `long_text` cell is a Y.Text, so the check reads the event's `path` as well as a map event's `keys`
- [x] The schema signature includes the sorted COLUMN's declaration, not just the sort — reordering a select's options changes the row order without a cell changing
- [x] Header menu: two direct rows named in the column's own vocabulary ("Sort oldest first", "Sort by option order"), the active one clearing the sort when clicked again; a glyph on the sorted header is the only thing on screen that explains the order, so it is not hover-revealed
- [x] Deleting or hiding the sorted column clears the sort (`dropColumn` / `withoutSortOn`) — rows in an arrangement with nothing on screen to explain it is the state that avoids
- [x] `read` gained `orderBy`, applied after `where` and before paging so `orderBy` + `limit` is a real top-N, plumbed through `ReadProjectionRequest` → `TableReadRequest` → the `table` tool's `read` params and `admin.tables.read`. An unknown column is a degradation, not a silent no-op — a typo'd `orderBy` that returns document order looks exactly like a sort that changed nothing
- [x] Export and print stay in DOCUMENT order, deliberately: the workbook and the mirror are the document, the sort is a view of it, and both carry `_id` so nothing is lost
**Tests:**
- [x] `apps/server/src/services/documents/table/__tests__/table-sort.test.ts` — 33 tests. Each comparator case pins the naive string order alongside the typed one, because those two differing is the whole reason the module exists; plus the read's ordering-before-paging, its four degradations, and the schema round trip through the Y.Map (including that an unchanged sort writes no Y item) and the markdown mirror
- [x] `apps/mail/tests/modules/documents/table/TableSort.test.tsx` — 13 tests over a real Y.Doc: the typed order rendered, `readRowIdsInOrder` proving the row array is untouched, a re-sort when the sorted cell changes and NO re-order when another column's does, the withdrawn drag handle, and the menu writing/clearing the schema
- [x] `timeout 300 pnpm --filter @zero/server exec vitest run src/services/documents/table` · `cd apps/mail && npx jest tests/modules/documents/table`
### Phase 15 — Section rows
A row that is a heading rather than a record — the merged title row a spreadsheet gives you.
Designed and built separately: see [`table-section-rows.md`](./table-section-rows.md).
The one thing worth knowing from over here is that it interacts with Phase 14's sort: a section
row segments the row list, so a sort orders rows WITHIN each section and holds every heading in
place. `sortByColumn` delegates to `sortWithinSections` for exactly that reason.