Libra CodeHub

CedarCopilot/cedar-mail

Branch: staging

fix(documents): stop agenda.integrity freezing content_yjs forever

merged#2686CedarCopilot

CedarCopilot wants to merge 3 commits into staging from fix/agenda-integrity-abort-freezes-yjs

Live on prod, no production signal yetTimeline and evidence
  1. Opened
    Sep 4, 2026, 10:35 AM
  2. Sep 4, 2026, 11:05 AM
  3. Merged
    Sep 4, 2026, 2:55 PM
  4. Live on prod
    Sep 4, 2026, 2:55 PM
  5. Observed 3 days
    Sep 4, 2026, 2:55 PM
  6. Pipelines steady after this deploy
    Sep 4, 2026, 2:55 PM
  7. Unobserved

    Live on prod, no production signal yet

    Sep 7, 2026, 2:47 PM

Behaviors Libra is checking

The document save pipeline must explicitly report whether content_yjs was persisted and return all hook rejection reasons so callers can distinguish a committed save from an aborted one.Brokenlow confidence

Verification reached its budget before a verdict.

prod, checked Sep 7, 2026, 2:47 PM
Agent-originated writes refused by a save hook must raise a DocumentPersistAbortedError and reach the write-document caller as a recoverable success:false result instead of being reported as a successful write.Inconclusivelow confidence

Only 4 prod OTEL spans matching write-document reached Libra since the deploy, below the 20-span floor for calling a change verified. Nothing is failing; there is not yet enough traffic to confirm it.

prod, checked Sep 6, 2026, 11:47 PM
A document save that succeeds while refusing an associated side effect must return success while appending the refusal reasons to the agent-visible write-document message.Inconclusivelow confidence

Only 4 prod OTEL spans matching write-document reached Libra since the deploy, below the 20-span floor for calling a change verified. Nothing is failing; there is not yet enough traffic to confirm it.

prod, checked Sep 7, 2026, 12:47 AM
When playbook compilation aborts a save, the agent must receive the number and messages of the compilation errors that caused the refusal.Inconclusivelow confidence

Verification reached its budget before a verdict.

prod, checked Sep 7, 2026, 1:49 AM
An aborted document persist must not create a document history entry for an update that never reached documents.content_yjs.Inconclusivelow confidence

Verification reached its 120s wall-clock budget before a tied verdict. Libra is keeping this intent verifying until the next check.

prod, checked Sep 7, 2026, 2:49 AM
When agenda.integrity detects an open task removed from an agent-written agenda, it must reject only the task-side effect while still persisting the new agenda content_yjs revision.Inconclusivelow confidence

Verification reached its budget before a verdict.

prod, checked Sep 7, 2026, 3:49 AM

Libra has verdicts on 0 of 8 tracked behaviors on prod; 8 are still being checked. Libra has 1 low-confidence broken verdict it is still confirming. Libra checks hourly for 3 days after each deploy.

65 of 72 real daily agendas were not showing today's date. 29 were more than a fortnight behind. <email> was being served an agenda from Aug 10; <email> and <email> from Jul 30, 36 days. On the same morning, 42 of those users had a correct # 2026-09-04 agenda sitting in documents.content that nothing would ever show them.

There were no failed tool calls. Every agent was told it had written the agenda.


The mechanism

write-document persists in two non-atomic phases, and only the first one had happened.

All line numbers below are as of staging , the code that had the bug.

#SiteWhat it commits
1writeDocumentTool.ts:479calls writeDocumentGuarded({ skipYjsSync: true })
2services/documents/index.ts:620-638markdown only , content, version + 1, word_count, updated_at; closes its connection at :673
3writeDocumentTool.ts:578-592calls writeFileAsYjs as a separate later step
4writeFileAsYjs.ts:318-329hands the mutated Y.Doc to applyUpdate
5applyUpdate.ts:292-315the only site that writes content_yjs + yjs_revision

agendaIntegrityHook returned abortPersist: true. applyUpdate.ts:258-272 then returned early with no UPDATE documents at all.

So phase 2 had committed and phase 5 had not. content held today's agenda; content_yjs held one from weeks ago. Every reader takes content_yjs.

It is self-sustaining

The hook builds beforeJson from content_yjs. Frozen, its task set is frozen , so every later agenda, which legitimately rotates rows, appears to drop the same open tasks and is refused again. Xander's Sep 3 write carried 24 taskIds; 12 of the 20 frozen ones were absent and all 12 were still todoabortPersist. The refusal manufactures the condition for the next refusal.

Nothing self-heals

  • reconstructYDoc never re-reads markdown once bytes exist , hydrate.ts:264-269 returns early on row.contentYjs.
  • The lazy backfill is gated on their absence , get-doc.ts:529 is if (!row.contentYjs), so stale bytes skip it. get-doc.ts:508-511 already warns about exactly this shape for playbooks.
  • The agenda reconciler is subtractive only (tasks-to-agenda.ts:152-176; its own header says "REMOVES ONLY ... There is deliberately no INSERT") and returns mutated: false because all rows are still todo.

And it actively anti-repairs

applyUpdate.ts:289 derives markdownMirror from the Y.Doc and writes it to documents.content at :308; get-doc.ts:651-668 does the same on a mutating reconcile. So the next passing write, or a human opening the editor, reverts the good markdown to the stale state.

This happened to Xander mid-investigation. His content now reads # 2026-09-01, and the Sep 4 agenda survives only in agent_tool_calls. 12 of the 57 affected documents are in this state.


The data signature

An aborted persist still called recordHistoryAsync, with Buffer.alloc(0) from applyUpdate.ts:265. The normal path is Y.encodeStateAsUpdate, which is never 0 bytes , so a zero-byte document_updates row is unambiguous.

 zero_byte_rows | docs | document_type
----------------+------+---------------
            991 |   47 | agenda          ← this bug
            104 |   51 | playbook        ← playbook.compile, working as designed

Xander's last 8, consecutive, no successful write between them:

 started              from_seq  to_seq  bytes  origin
 09-03 16:53:10       298       298     0      agent
 09-03 17:18:05       299       299     0      agent
 09-03 17:48:02       300       300     0      agent
 09-03 19:35:12       301       301     0      agent
 09-03 22:30:51       302       302     0      agent
 09-04 14:41:24       303       303     0      agent
 09-04 15:10:13       304       304     0      agent
 09-04 15:38:30       305       305     0      agent

Two things this rules out

**The

Show production surfaces and changed-file mapping

Production surfaces

Libra has not measured any production surfaces for this change yet.

Changed files → surfaces

  • apps/server/scripts/repair-frozen-agenda-yjs.tsno production surface mapped
  • apps/server/scripts/run-repair-frozen-agenda-yjs.mjsno production surface mapped
  • apps/server/src/mastra/tools/document/writeDocumentTool.tsno production surface mapped
  • apps/server/src/services/document-saving/__tests__/applyUpdate-history-isolation.test.tsno production surface mapped
  • apps/server/src/services/document-saving/__tests__/persist-abort-contract.test.tsno production surface mapped
  • apps/server/src/services/document-saving/__tests__/writeFileAsYjs-abort.test.tsno production surface mapped
  • apps/server/src/services/document-saving/applyUpdate.tsno production surface mapped
  • apps/server/src/services/document-saving/hooks/__tests__/agenda-integrity.test.tsno production surface mapped
  • apps/server/src/services/document-saving/hooks/__tests__/agenda-to-tasks.test.tsno production surface mapped
  • apps/server/src/services/document-saving/hooks/agenda-integrity.tsno production surface mapped
  • apps/server/src/services/document-saving/hooks/agenda-to-tasks.tsno production surface mapped
  • apps/server/src/services/document-saving/hooks/playbook-compile.tsno production surface mapped
  • apps/server/src/services/document-saving/index.tsno production surface mapped
  • apps/server/src/services/document-saving/types.tsno production surface mapped
  • apps/server/src/services/document-saving/writeFileAsYjs.tsno production surface mapped
  • apps/server/src/services/documents/index.tsno production surface mapped