Libra CodeHub

CedarCopilot/cedar-mail

Branch: staging

fix(mcp): make tool annotations match real behavior (OpenAI app review)

merged#2922CedarCopilot

CedarCopilot wants to merge 2 commits into staging from fix/mcp-tool-annotations-match-behavior

Customer behavior verdict

Libra exercised 2 changed behaviors; none regressed.

Who this touches

  • SQL export calls are marked as external writes rather than read-only: Libra cannot measure who uses this yet; it has not mapped this to a production entry point yet; it will trace it through the import graph on a coming run.
  • Slack thread reads are marked as Cedar writes: Libra cannot measure who uses this yet; it has not mapped this to a production entry point yet; it will trace it through the import graph on a coming run.
  • Irreversible notifications and LinkedIn messages are marked destructive: Libra cannot measure who uses this yet; it has not mapped this to a production entry point yet; it will trace it through the import graph on a coming run.
  • Provider-backed drafts and workspace changes expose their external side effects: Libra cannot measure who uses this yet; it has not mapped this to a production entry point yet; it will trace it through the import graph on a coming run.
  • External table filling remains Cedar-only: Libra cannot measure who uses this yet; it has not mapped this to a production entry point yet; it will trace it through the import graph on a coming run.
  • SQL export advertises its externally accessible result: Libra cannot measure who uses this yet; it has not mapped this to a production entry point yet; it will trace it through the import graph on a coming run.
  • SQL export advertises its externally accessible result: Held on this PR; the scenario fails when this is broken on purpose.
  • External table filling remains Cedar-only: Held on this PR; the scenario fails when this is broken on purpose.

What Libra verified

  • External table filling remains Cedar-only

    Libra ran this behavior against the change and confirmed the check detects when it breaks.

    Technical evidence
    • external table fill keeps open-world false because it returns delegate briefs without external work

    Changed code: server.ts.

  • SQL export advertises its externally accessible result

    Libra ran this behavior against the change and confirmed the check detects when it breaks.

    Technical evidence
    • a SELECT-only run-sql call that exports results is marked non-read-only and open-world

    Changed code: server.ts.

Live on prod, watching, 1 day leftTimeline and evidence
  1. Opened
    Sep 23, 2026, 1:45 PM
  2. Sep 23, 2026, 2:07 PM
  3. Merged
    Sep 23, 2026, 5:12 PM
  4. Live on prod
    Sep 23, 2026, 5:12 PM
  5. Observed 17 hours
    Sep 23, 2026, 5:12 PM
  6. Watching

    Live on prod, watching, 1 day left

    Sep 23, 2026, 5:12 PM
  7. Pipelines steady after this deploy
    Sep 23, 2026, 5:12 PM

Behaviors Libra is checking

MCP clients now receive `openWorldHint: true` for analytics because monthly-growth queries the live PostHog API.Not checked
prod
MCP clients now receive `openWorldHint: true` for board operations because moving or setting a card can trigger an HTTP POST to a user-configured URL.Not checked
prod
MCP clients now receive `openWorldHint: true` for configuration writes because field creation and link mapping can query the connected live CRM.Not checked
prod
MCP clients now receive `destructiveHint: true` for conversation writes because merging conversations deletes the source and other actions can clear or overwrite stored data.Not checked
prod
MCP clients now receive `destructiveHint: true` for draft communications because drafting against an existing provider draft replaces its prior content in place.Not checked
prod
MCP clients now receive `openWorldHint: true` for run-sql because the `out` operation returns a presigned URL that can be fetched outside Cedar.Not checked
prod

Libra has verdicts on 0 of 12 tracked behaviors on prod; 12 are still being checked. Libra checks hourly for 3 days after each deploy.

Why

OpenAI rejected Cedar v1.0.0 in app review: "one or more of your tool's annotations do not appear to match the tool's behavior", and asked for explicit true/false hints with a justification per hint. Every tool already had all three hints set, so the gap was wrong values, not missing ones.

What

Audited every external tool's actions (source, not descriptions) against its hints and corrected 13 in EXTERNAL_TOOL_METADATA:

ToolChangeReason
tableopenWorld → truefill subagents use web search/fetch, enrichment, live Slack
boardopenWorld → truemove/set can POST to a user-configured URL
config-writeopenWorld → truecreate-field (sourceFromCrm) and link-mapping hit the live CRM
workspace-writedestructive → true, openWorld → truehard deletes; Gmail label rename/delete; webhook export config
debugdestructive → truetest-run with deliverForReal sends real Slack/email
run-sqlreadOnly → false, openWorld → trueout writes to S3 and returns a presigned URL; title now "Run a SQL query (SELECT only)"
analyticsopenWorld → truemonthly-growth calls PostHog
conversation-writedestructive → truemerge_conversations deletes source, clear_working_memory wipes, update-fields overwrites
slack-readreadOnly → falseread-thread ingests messages into the deal timeline
taskopenWorld → truedeletes Gmail drafts, relabels threads
notifydestructive → truereal sends cannot be unsent
draft-commsdestructive → truedraft-email replaces an existing provider draft in place
linkedin-writedestructive → truemessages/invitations/InMail cannot be unsent

Each table comment now states why the hint is set, which is the justification OpenAI asks for.

Judgment calls

  • notify and linkedin-write are marked destructive because sends are irreversible. Revert those two if "destructive" should mean delete/overwrite only.
  • run-sql: alternative is stripping out from the external schema and keeping the read-only hint.
  • send-user-feedback left as is (one post to Cedar's own feedback channel).
  • conversation-read, conversation-analytics, mail write ephemeral scratch buffers but stay read-only. First candidates if OpenAI rejects again.

Verify

  • pnpm --filter @zero/server exec vitest run src/mastra/mcp/external/__tests__/tool-hints-match-behavior.test.ts src/mastra/mcp/external/__tests__/server.test.ts passes (54 tests).
  • After deploy, tools/list on the external MCP should show the new values, e.g. linkedin-write with destructiveHint: true, slack-read with readOnlyHint: false. Then resubmit from the OpenAI Platform dashboard.

🤖 Generated with Claude Code

RetriggerConfidence Score: 2/5

The PR is not safe to merge until the table, run-sql, and additive-send annotations are reconciled with actual behavior and the metadata definitions.

Findings

  1. P1 Table hint mismatches behavior ▶
  2. P1 S3 is not open-world ▶
  3. <img alt="P1" src="h
Show production surfaces and changed-file mapping

Production surfaces

Libra has not measured any production surfaces for this change yet.

Changed files → surfaces

  • apps/server/src/mastra/mcp/external/__tests__/tool-hints-match-behavior.test.tsno production surface mapped
  • apps/server/src/mastra/mcp/external/server.tsno production surface mapped