CedarCopilot wants to merge 2 commits into staging from fix/execution-stale-threshold
Live on prod, no production signal yet
Strict CloudWatch fallback found 26 recent prod failure log lines for Failed to create execution for task (Task already claimed or no longer eligible), but the same failure had 41 log lines in the comparable pre-deploy baseline. Libra is not blaming this PR intent without a.
Strict CloudWatch fallback found 103 recent prod failure log lines for [UpdateExternalCrmWorkflow] Driver update failed (Step updateOpportunityField failed: HTTP 400: [{\), but the same failure had 84 log lines in the comparable pre-deploy baseline. Libra is not blaming this PR.
Strict CloudWatch fallback found 8 recent prod failure log lines for [backfillSlackChannelHistory] Backfill failed (Failed query: insert into \), but the same failure had 3 log lines in the comparable pre-deploy baseline. Libra is not blaming this PR intent without a.
single_user
single_user
single_user
Libra has verdicts on 0 of 3 tracked behaviors on prod; 3 are still being checked. Libra attributed 3 failures to this change at medium confidence and is still confirming them. Libra checks hourly for 3 days after each deploy.
Zach (Concentrate) got two near-identical post-meeting recap drafts for the same Razorpay meeting, five minutes apart, on 2026-08-24.
validateExecutionStatus treated any executing run older than 10 minutes as abandoned, deleted its dedupe row, and let the caller start a fresh run. processStaleExecutionReaper made the same judgement with a 30-minute threshold measured from the last progress write. A run could therefore be alive to one and dead to the other.
Granola redelivers the same meeting payload every ~5 minutes (identical 65,226 bytes each time). CloudWatch, /aws/ecs/aws-prod-api/api-service:
| Delivery | Time (UTC) | Verdict |
|---|---|---|
| 1 | 12:40:40 | No execution found → starts d449c5d1-2284-4217-852a-209c2208cc49 |
| 2 | 12:41:41 | Skipping - execution in progress |
| 3 | 12:45:40 | is currently executing (started 300s ago), skipping |
| 4 | 12:50:41 | has been executing for 600s (>10min), deleting → [deleteAgentExecution] Successfully deleted → starts f05dbb80-029e-477a-80a4-bcd7fb07d2f4 |
Delivery 4 landed one second past the line. The deleted run had already drafted at 12:47:54 (draft-email in c459c93c-592f-4dd2-bf23-03c1ca3ac96d); the replacement drafted again at 12:52:53 (95be5a76-e707-4e67-b512-06ee266d4692).
The reaper's own header already documents why 10 minutes is too aggressive , its threshold is set high "to avoid false-positiving on legitimately long-running meeting-transcript field extraction."
Blast radius: 25 orchestrator rows deleted mid-flight across ~15 users in 14 days; 11 of those chains had already called draft-email/save-draft before deletion, across 7 users.
STALE_EXECUTING_THRESHOLD_MS, shared by the reactive check and the sweep, so they cannot drift apart again.GREATEST(createdAt, updatedAt)), mirroring the reaper , an actively-progressing run is never mistaken for a dead one. updatedAt is optional and falls back to createdAt, so a caller without a progress signal keeps the old behavior.updatedAt.Genuinely abandoned runs are still deleted, so the heal-by-rerun path that recovers from an api-service SIGTERM is unchanged. That delete is the mechanism that lets a killed run's meeting get processed at all , blocking it would trade a visible duplicate for a silent miss.
apps/server/src/services/agent-action-queue/__tests__/execution-staleness.test.ts , 7 cases, including the exact regression (a run at the old 10-minute mark stays alive) and the inverse (a 45-minute-silent run is still deleted).
Verified: pnpm --filter @zero/server run types clean · pnpm deps:check clean · scoped vitest green.
🤖 Generated with Claude Code
This PR aligns reactive execution validation with the stale-execution reaper by sharing a 30-minute threshold and measuring inactivity from the latest progress timestamp.
updatedAt through meeting, email, and Slack deduplication paths.The PR appears safe to merge after the non-blocking import-convention issue is cleaned up.
The execution-staleness behavior is consistently updated across the reactive callers and reaper, with focused regression coverage; the only accepted concern is the use of relative imports in two changed files.
Files Needing Attention: apps/server/src/cron/process-stale-execution-reaper.ts, apps/server/src/services/agent-action-queue/tests/execution-staleness.test.ts
| Filename | Overview |
|---|---|
| apps/server/src/services/age |
Libra has not measured any production surfaces for this change yet.