Libra CodeHub

CedarCopilot/cedar-mail

Branch: staging

docs(bug-reports): ambiguity_pinged_at meeting-ingestion outage writeup

merged#2806CedarCopilot

CedarCopilot wants to merge 0 commits into staging from docs/ambiguity-pinged-at-outage-writeup

Live on prod, no production signal yetTimeline and evidence
  1. Opened
    Sep 17, 2026, 8:48 AM
  2. Merged
    Sep 17, 2026, 3:48 PM
  3. Live on prod
    Sep 17, 2026, 3:48 PM
  4. Observed 2 hours
    Sep 17, 2026, 3:48 PM
  5. Unobserved

    Live on prod, no production signal yet

    Sep 17, 2026, 3:48 PM
  6. Pipelines steady after this deploy
    Sep 17, 2026, 3:48 PM

Libra has no production signal for this change on prod yet. Libra checks hourly for 3 days after each deploy.

Summary

  • Documents the 2026-09-14/15 schema-drift incident: crm_meeting_events.ambiguity_pinged_at was added to the Drizzle schema (commit a87e2865e) without the corresponding migration ever being applied to prod, silently failing every meeting-ingestion attempt org-wide for ~23 hours.
  • Root-causes a second, still-open bug found while investigating: updateExecutionStep unconditionally writes status: 'completed' regardless of whether the run actually succeeded, which is why nobody was paged and why the true blast radius (91 executions / 84+ meetings / 30+ users) was undercounted on the first pass.
  • Captures every gotcha hit while backfilling the dropped meetings (stale-status blocking its own replay, a broken meeting-id regex in replay-execution.ts, a background batch that survived TaskStop, a CloudWatch search window that wasn't re-derived from the actual fix time).
  • Cross-referenced as gotcha #46 in SHERLOCK_SYSTEM_KNOWLEDGE.md so a future related investigation finds it via the sherlock skill.

Test plan

  • Docs-only change, no code touched.
  • Reviewed rendered markdown for accuracy against the live investigation (Sherlock investigation 5cf965ab-e9d1-4af9-b41c-d9d7dd07d020).

🤖 Generated with Claude Code

RetriggerConfidence Score: 4/5

The docs-only PR is safe to merge from a runtime perspective, but the incident record should be corrected to avoid misleading future responders.

Findings

  1. P2 Migration State Is Inaccurate ▶
  2. P2 Wrong Webhook Success Signal ▶
  3. P2 Backfill Scripts Are Missing ▶
Fix with agent prompt 
### Issue 1
apps/server/docs/bug-reports/ambiguity-pinged-at-meeting-ingestion-outage.md:34
The report says no migration file exists, but this PR’s head contains `apps/server/src/db/migrations/add_ambiguity_pinged_at_to_crm_meeting_events.sql` with the documented `ALTER TABLE`. The file remains outside the Drizzle journal and requires manual application. Please distinguish “the migration exists but is not applied automatically” from “no migration was created” so responders understand both the current repository state and the remaining drift risk. The same inaccurate claim appears in the evidence table at line 93.

### Issue 2
apps/server/docs/bug-reports/ambiguity-pinged-at-meeting-ingestion-outage.md:63-64
The write-up says the webhook checks the final step’s `success` field, but `handleExecuteMeeting` actually checks Mastra’s `workflowResult.status`. Because the upstream failures are returned instead of thrown, the workflow status remains successful and the route returns HTTP 200. Documenting the wrong signal directs future debugging toward the final payload rather than the non-throwing workflow boundary. The new Sherlock entry repeats the same inaccurate explanation.

### Issue 3
apps/server/docs/bug-reports/ambigu
Show production surfaces and changed-file mapping

Production surfaces

Libra has not measured any production surfaces for this change yet.

Changed files → surfaces

  • apps/server/.claude/skills/sherlock/SHERLOCK_SYSTEM_KNOWLEDGE.mdno production surface mapped
  • apps/server/docs/bug-reports/ambiguity-pinged-at-meeting-ingestion-outage.mdno production surface mapped