Summary

on-event-agent-execution-workflow.ts builds the email orchestrator's prompt as a single plain string sent to GPT-4o , no explicit cache_control blocks, so OpenAI's automatic prefix caching is the only lever, and it requires an exact byte match of everything before a given point in the request.

The old section order was: execution_context (carries a fresh run_id on literally every call) → conversation_context (the email/thread content, different every triggering event) → ai_inbox_options → playbook_section (a full rendered AOP playbook, static for a given org/AOP/stage/event type) → task. Putting the two genuinely reusable sections (playbook_section, ai_inbox_options) downstream of run_id and the ever-different conversation content meant they could never land in a matching prefix either , the same structural bug already fixed in conversation-field-updates.ts / strategist-updates.ts / updateNextStepsAndTasksTool.ts, just discovered in a fourth, GPT-4o-routed file with no explicit cache markers to make the problem visible in a cache_write/cache_read split.

The fix: reorder so the stable sections (playbook_section, ai_inbox_options, collaboratorContext, outputPolicyContext) come first, right after the preamble this string gets appended to; conversation_context and task stay next since they're unique per triggering event; execution_context (the one section guaranteed unique on every single call via run_id) moves to the very end, where it can't block anything upstream from being reused. Same text, same relative order within each section , only which section comes before which changes.

Estimated savings

Derived from real Understudy trace + billing data, not assumed:

• Pulled 40 real captures and measured actual composition: system prompt 14,242 chars (constant, cached), tools schema 11,176 chars (constant), preamble avg 10,808 chars, conversation_context avg 22,822 chars (32% of request, genuinely volatile), playbook_section avg 10,918 chars / up to 24,179 (15.3% of request), task avg 921 chars, execution_context avg 536 chars.
• Measured token-level cache hit rate: 80% (avg prompt_tokens 19,074, avg cached_tokens 15,264 across the sample).
• playbook_section was structurally almost always outside the cacheable prefix under the old order , it's ~15.3% of total request size and, at worst, could account for up to ~76% of the currently-uncached ~20% of tokens (15.3/20).
• OpenAI's cached-token discount is ~50% off list. If reordering lets same-org/AOP/stage repeats hit cache at a rate similar to the rest of the now-stable prefix (~80%, matching what system/tools/preamble already see): estimated $20-25/day ($600-750/month, ~$7-9k/year).
• This is a genuine estimate, not a guaranteed number , confirm with a real before/after cache-hit-rate comparison once deployed, the same way as the other three prompt-reorder fixes.

Test plan

• Typecheck clean (pnpm run types from apps/server , only pre-existing, unrelated better-auth plugin-export errors in the full build, not touching this file)
• Confirm in practice: compare cached_tokens/prompt_tokens ratio for this workload before/after deploy

🤖 Generated with Claude Code

https://claude.ai/code/session_01JfEdawy9Y7huYi1j59saZN

Greptile Summary

This PR reorganizes the event orchestrator’s plain-string model prompt to improve automatic prefix-cache reuse.

• Places the rendered playbook, AI inbox options, collaborator context, and output policy before event-specific conversation content.
• Moves per-run execution metadata, including the unique run ID, to the end.
• Preserves the contents of each section while introducing an ordering invariant that should be covered by a focused prompt-construction test.

Confidence Score: 4/5

The PR appears safe to merge, with a non-blocking recommendation to protect the