Libra CodeHub

CedarCopilot/cedar-mail

Branch: staging

fix(agents): live pre-output re-check + playbook reorder backfill

open#2776CedarCopilot

CedarCopilot wants to merge 2 commits into staging from fix/live-thread-revalidation-gate

Live on prod, watching, less than a day leftTimeline and evidence
  1. Opened
    Sep 13, 2026, 6:48 PM
  2. Sep 14, 2026, 5:57 AM
  3. Live on prod
    Sep 14, 2026, 6:17 AM
  4. Observed 2 days
    Sep 14, 2026, 6:17 AM
  5. Pipelines steady after this deploy
    Sep 14, 2026, 6:17 AM
  6. Watching

    Live on prod, watching, less than a day left

    Sep 16, 2026, 3:17 PM
  7. Merged
    Pending

Behaviors Libra is checking

Before drafting a new email thread, the agent searches live email for recent messages to the resolved recipients and skips the draft when another thread already covers the same request.Inconclusivelow confidence

Verification reached its 130s outer budget before the verifier returned. Libra is keeping this intent verifying until the next check.

prod, checked Sep 16, 2026, 9:14 AM
draft-email refuses to persist a new email draft when a final live re-check finds newer thread activity or an existing mailbox draft, while allowing an intentional rewrite of the draft it already found.Inconclusivelow confidence

Only 6 prod OTEL spans matching draft-email reached Libra since the deploy, below the 20-span floor for calling a change verified. Nothing is failing; there is not yet enough traffic to confirm it.

prod, checked Sep 16, 2026, 3:17 PM
draft-slack refuses to persist a Slack draft when an email-triggered thread has newer live activity or an existing draft immediately before persistence.Not checked
prod
The post-event executor receives a live_thread_check prompt block when the live mailbox shows newer activity or an existing draft, instructing it not to duplicate work against stale context.Not checked
prod
Email-triggered runs fetch the triggering thread directly from the mailbox immediately before output dispatch and detect non-draft messages newer than the event snapshot or drafts already present in the thread.Not checked
prod
Missing connections, invalid event timestamps, nonexistent threads, or mailbox fetch failures cause the live check to return no blocking signal rather than breaking the event pipeline.Not checked
prod

Libra has verdicts on 0 of 14 tracked behaviors on prod; 14 are still being checked. Libra checks hourly for 3 days after each deploy.

Summary

Second PR from the same latency/staleness investigation as #2775 (dispatch ordering). This one covers the two remaining pieces: making the playbook documents describe the order that's now actually true, and closing the actual stale-context/duplicate-output gap.

1. Playbook reorder migration , already applied

Moves strategist/next-steps (with their prose, correctly kept attached to whichever ref it precedes) to the tail of every account's <trigger type="any"> block, matching what the orchestrator dispatches at runtime after #2775. Already run for real against the full active cohort: 56/57 succeeded, 1 correctly skipped (an eligibility edge case, not guessed at) and fixed by hand in scripts/manual-fix-jacob-reorder.ts. Every account's before/after state was independently read back from Postgres via a separate DOM-based verifier (not the migration's own logic) , see commit message for full detail.

2. Live pre-output re-validation gate

Traced both incidents to the same underlying gap: a subagent decides to draft or alert from a conversation snapshot frozen at the start of its run, and nothing re-checks it before acting , even though the run can sit queued for minutes. Directly measured: a rep's own reply was sent 2m47s after the trigger and wasn't visible in Cedar's own synced conversation state for another 16m35s , so a re-check against Cedar's own database would have caught nothing. Only a live mailbox fetch, bypassing Cedar's sync pipeline entirely, can catch this in time.

checkThreadForNewerActivity (staleness-check.ts) fetches the live thread and flags a newer non-draft message (the rep already replied) or an existing draft (already asked Cedar via chat) , wired into both places a subagent actually produces output: the drafter dispatch and the generic custom-subagent path (covers inbound-email-notifier and similar). Degrades to null on any failure rather than ever blocking the pipeline. For brand-new threads, added a prompted search-live-emails check to the drafter's own instructions instead, since the deterministic check only covers the thread that triggered the run.

Test plan

  • New unit tests (staleness-check.test.ts) , newer-reply, existing-draft, nothing-changed, connection/API failure, unparseable-timestamp cases
  • Existing runSubagentTool/drafter-executor-prompt suites (54 tests) pass unchanged
  • pnpm --filter @zero/server run types clean
  • oxlint --deny-warnings clean on every changed file
  • Migration dry-run reviewed before the real --apply run; live DB read back independently afterward for multiple accounts across every distinct shape (simple, custom-subagent-spliced, the most complex multi-sequencer account, and the manually-fixed edge case)
  • Deploy to staging and watch a live execution's tool-call trace to confirm the live-thread check actually fires and reads correctly against a real mailbox (unit tests mock getThread)

Related

  • #2775 (dispatch ordering , merge first if not already)

šŸ¤– Generated with Claude Code

https://claude.ai/code/session_01WDwXHeMTJKLmcSjj1msiuS

Show production surfaces and changed-file mapping

Production surfaces

Libra has not measured any production surfaces for this change yet.

Changed files → surfaces

  • apps/server/scripts/manual-fix-jacob-reorder.tsno production surface mapped
  • apps/server/scripts/reorder-any-block-tail.tsno production surface mapped
  • apps/server/src/mastra/agents/on-event-execution-agent.tsno production surface mapped
  • apps/server/src/mastra/tools/event-execution/orchestrator-dispatch-tools.tsno production surface mapped
  • apps/server/src/mastra/tools/event-execution/runSubagentTool.tsno production surface mapped
  • apps/server/src/services/mail/threads/__tests__/staleness-check.test.tsno production surface mapped
  • apps/server/src/services/mail/threads/staleness-check.tsno production surface mapped
  • apps/server/src/services/playbook/backfill/reorder-any-block-tail.tsno production surface mapped