Libra CodeHub

CedarCopilot/cedar-mail

Branch: staging

fix(crm): resync deal fields from live CRM before crm-updater reasons about them

merged#2901CedarCopilot

CedarCopilot wants to merge 3 commits into staging from worktree-send-it-crm-updater-resync

Customer behavior verdict

Libra exercised 4 changed behaviors; none regressed.

Who this touches

  • Ambiently opened artifacts are not remembered for the active chat: Libra cannot measure who uses this yet; it has not mapped this to a production entry point yet; it will trace it through the import graph on a coming run.
  • Removing the active conversation chip clears its active context: Libra cannot measure who uses this yet; it has not mapped this to a production entry point yet; it will trace it through the import graph on a coming run.
  • Successful board creation opens the created board beside the transcript: Libra cannot measure who uses this yet; it has not mapped this to a production entry point yet; it will trace it through the import graph on a coming run.
  • Document editing reads remain scoped to the requested document owner: Libra cannot measure who uses this yet; it has not mapped this to a production entry point yet; it will trace it through the import graph on a coming run.
  • board: no production calls in the last 7 days.
  • document-read: 175 runs for 4 of 17 active accounts in the last 7 days (4 of them internal); 6 failed (3%).
  • Most common action: read (64%).
  • Most common action: list (27%).
  • Most common action: search (9%).
  • Pre-execution CRM refresh changes fields without changing ownership: Held on this PR; the scenario fails when this is broken on purpose.
  • A linked deal with a legacy null organization remains eligible for refresh: Held on this PR; the scenario fails when this is broken on purpose.
  • The updater reasons from refreshed core and custom field values: Held on this PR; the scenario fails when this is broken on purpose.
  • Unavailable pre-execution refresh preserves normal updater execution: Held on this PR; the scenario fails when this is broken on purpose.

What Libra verified

  • A linked deal with a legacy null organization remains eligible for refresh

    Libra ran this behavior against the change and confirmed the check detects when it breaks.

    Technical evidence
    • the live CRM refresh targets the conversation with a null organization ID and supplies its fresh field value instead of leaving the local baseline unchanged

    Changed code: orchestrator-dispatch-tools.ts.

  • Pre-execution CRM refresh changes fields without changing ownership

    Libra ran this behavior against the change and confirmed the check detects when it breaks.

    Technical evidence
    • the pre-execution CRM refresh makes freshly resynced linked-deal field values visible instead of stale conversation values

    Changed code: orchestrator-dispatch-tools.ts.

  • The updater reasons from refreshed core and custom field values

    Libra ran this behavior against the change and confirmed the check detects when it breaks.

    Technical evidence
    • the crm-updater current-field view uses refreshed values for all eight core metadata fields on the same run
    • the crm-updater current-field view represents an absent refreshed core value as null instead of retaining the stale value

    Changed code: conversation-context.ts.

  • Unavailable pre-execution refresh preserves normal updater execution

    Libra ran this behavior against the change and confirmed the check detects when it breaks.

    Technical evidence
    • an enabled non-CRM-sync CRM updater can continue with its pre-refresh local field view when the live CRM refresh fails
    • an enabled non-CRM-sync CRM updater can continue with its pre-refresh local field view when the post-refresh field re-read fails

    Changed code: orchestrator-dispatch-tools.ts.

Live on prod, watching, 1 day leftTimeline and evidence
  1. Opened
    Sep 23, 2026, 1:13 AM
  2. Sep 23, 2026, 2:32 AM
  3. Merged
    Sep 23, 2026, 12:59 PM
  4. Live on prod
    Sep 23, 2026, 12:59 PM
  5. Observed 0 hours
    Sep 23, 2026, 12:59 PM
  6. Watching

    Live on prod, watching, 1 day left

    Sep 23, 2026, 12:59 PM
  7. Pipelines steady after this deploy
    Sep 23, 2026, 12:59 PM

Behaviors Libra is checking

CRM pull-refresh and pre-execution resync outcomes, including failures, timeouts, and abandoned work, are emitted through structured logging for operational diagnosis.Not checked
prod
If the live CRM fetch outlives the pre-execution timeout, its eventual results are discarded before any field-mapping or orphan-metadata write can overwrite newer updater changes.Not checked
prod
The pre-execution resync uses the conversation row's own `organization_id`, including its legacy null value, instead of substituting the calling user's context organization.Not checked
prod
The pre-execution resync refreshes fields only and does not fetch owner directories, reassign conversation ownership, add members, or apply active-deal rule side effects.Not checked
prod
Before an enabled crm-updater reasons about a conversation, Cedar refreshes its linked deal fields from the live CRM and passes the freshly re-read values into the updater context.Not checked
prod
A CRM resync database or provider failure is treated as a soft failure so the crm-updater continues updating fields and reaches its normal successful completion.Not checked
prod

Libra has verdicts on 0 of 9 tracked behaviors on prod; 9 are still being checked. Libra checks hourly for 3 days after each deploy.

Summary

  • Cedar's CRM push is diff-based against its own locally stored field values, with nothing verifying those values still match what's actually committed on the CRM side. Confirmed incident: a Forecast Category value that had never actually been pushed to Salesforce was treated as "unchanged" during a stage advance, so it was silently excluded from the outbound push, and Salesforce's own validation then rejected the stage move because the live field was blank.
  • Adds resyncLinkedDealFromCrm, a timeout-bounded (9s), never-throwing wrapper around the existing pullRefreshCrmDealsForConversations, wired in as a pre-step right before the crm-updater agent starts reasoning about a linked deal , skipped when the subagent is disabled or the event is itself a CRM sync, and falling back to today's behavior (no resync) if the call fails or hangs.
  • Scoped with a new fieldsOnly flag so this pre-execution gate only refreshes field values, never the owner-reassignment / active-deal-marking side effects pullRefreshCrmDealsForConversations also performs for its other callers.
  • Freshly resynced fields are also merged into the LLM's own <current_fields> view (all 8 core metadata fields, not just a subset), so the agent reasons from live data on the same turn, not just the later push-diff. That re-read phase has its own, smaller timeout ceiling (3s) so it can't hang the run either.

Full incident writeup and design: docs/design/crm-updater-pre-execution-resync.md.

Test plan

  • pnpm --filter @zero/server run types , clean, no errors
  • pnpm deps:check , no dependency violations
  • Scoped vitest across every touched/related test file , 39 tests passing:
    • src/services/crm/__tests__/crm-pull-refresh.resync.test.ts , resyncLinkedDealFromCrm's hard no-throw/no-hang contract (real DB failure + genuine hang, not mocked away), and the fieldsOnly guarantee (proves zero owner-directory fetch / reassignment calls under fieldsOnly: true, plus a regression check that omitting it preserves the original behavior for the two pre-existing callers)
    • src/services/crm/__tests__/crm-resync-field-overrides.test.ts , direct coverage of fetchResyncedFieldValues's metadata/custom-field split and null-handling, plus a genuine-hang test proving the new 3s re-read timeout actually bounds the phase (not just typechecks)
    • src/mastra/tools/event-execution/__tests__/crm-updater-enabled-flag.test.ts , end-to-end proof through the real runCrmUpdaterTool.execute call site that a resync failure never affects the run, and that the resync correctly skips when the subagent is disabled or the event is external_crm
    • src/mastra/utils/context-formatting/__tests__/extract-conversation-fields-resync-overrides.test.ts , unmocked extractConversationFields, proving all 8 core metadata fields (not just custom fields) correctly reflect a resync override
    • src/services/crm/__tests__/conversation-field-comparison.test.ts , regression check on the shared type guard used to eliminate a cast in both this file and the new one
  • Pre-existing, unrelated failure noted and confirmed out of scope: apps/mail/tests/modules/chat-store/chatHistoryList.test.ts (chat-history date bucketing) , zero files under apps/mail are touched by this diff.

Verify in prod

Two structured log families to watch in Axiom (cedar-prod dataset) or CloudWatch (/aws/ecs/aws-prod-api/*-service):

  • crm_pre_execution_resync_complete (info) / crm_pre_execution_resync_failed (warn) , the live-CRM fetch + field-write phase, fields: user_id, conversation_id, org_id, updated_fields (complete) or error (failed, including "timed out after 9000ms" on the 9s ceiling).
  • crm_resync_refetch_timed_out / crm_resync_refetch_failed (warn) , the post-resync re-read phase that feeds the LLM's prompt, same user_id/conversation_id/org_id fields plus error.
['cedar-prod']
| where _time > ago(1h) and message in ("crm_pre_execution_r
Show production surfaces and changed-file mapping

Production surfaces

Libra has not measured any production surfaces for this change yet.

Changed files → surfaces

  • apps/server/src/mastra/tools/event-execution/__tests__/crm-updater-enabled-flag.test.tsno production surface mapped
  • apps/server/src/mastra/tools/event-execution/orchestrator-dispatch-tools.tsno production surface mapped
  • apps/server/src/mastra/utils/context-formatting/__tests__/extract-conversation-fields-resync-overrides.test.tsno production surface mapped
  • apps/server/src/mastra/utils/context-formatting/conversation-context.tsno production surface mapped
  • apps/server/src/services/crm/__tests__/crm-pull-refresh.resync.test.tsno production surface mapped
  • apps/server/src/services/crm/__tests__/crm-resync-field-overrides.test.tsno production surface mapped
  • apps/server/src/services/crm/conversation-field-comparison.tsno production surface mapped
  • apps/server/src/services/crm/crm-pull-refresh.tsno production surface mapped
  • apps/server/src/services/crm/crm-resync-field-overrides.tsno production surface mapped
  • docs/design/crm-updater-pre-execution-resync.mdno production surface mapped