Summary

• Adds the field-query skill (apps/server/.claude/skills/field-query/SKILL.md): mandatory discover-before-query order for crm_conversation_field_values (list-fields/taxonomy-fields before run-sql or conversation-read), a vocabulary-check step, and a standing rule to always state the search space used in the final answer. States plainly that extraction is a primitive capture layer with no notion of confidence or canonicalization , that judgment belongs to the querying agent, applied fresh every time.
• Heals the 74 candidate (org, field) pairs found with real casing/punctuation drift against their declared select/list options, using the exact coerceToCanonicalOptionValue algorithm from #2883 ported to Python (this worktree has no node_modules). 61 of 74 had real drift; applied with per-row verification, 16,382 rows healed across 16 orgs. Originals preserved in metadata.healedFrom; a post-hoc sweep confirms zero healed row differs from its original by anything beyond case/whitespace/underscore.
• Adds two eval harnesses under playbook-instruction-eval, reimplemented in Python, exercising the real chat agent against real DB reads (read-only run-sql/config-read):
  • field_query_corpus_eval.py , full agentic completions against real historical failure prompts.
  • field_query_fire_rate_eval.py , cheap targeted fire-rate measurement across the full 9-prompt real-customer corpus, isolating two specific behaviors instead of requiring full completions: discovery-before-query (22% baseline → 73% with skill) and search-space disclosure in the final answer (13% → 33%). Includes an explicit negative control that correctly stays at 0/5 in both conditions.

Honest status

• Discovery-before-query effect is strong and consistent. Search-space-disclosure improved meaningfully but is still far from universal (33% with the skill) , real remaining gap, not fully solved.
• Two of the 9 corpus prompts (Mihir/Invoice Butler's deal-source questions) stay weak even with the skill in the discovery-first test; plausibly because the compound question's first sub-answer (a plain count) doesn't need field discovery, so the measurement may be counting a legitimate non-field run-sql call as a miss. Flagged, not resolved.
• Both scripts require DATABASE_URL (and ANTHROPIC_API_KEY for the evals) in the environment , no secrets committed.

Test plan

• Healing script dry-run spot-checked across dozens of mappings (including all of Warp's affected fields) before apply , confirmed pure formatting changes only, never a merge of two distinct option values.
• Applied for real with per-row verification; post-hoc DB sweep confirms 16,380 healed rows are all case/whitespace/underscore-only diffs from their originals.
• Fire-rate eval run against the real chat-agent framing + real tools across all 9 corpus prompts, both variants, 5 trials each.
• Not yet run against the full chat-agent server process (dry Python reimplementation of the prompt/tool shape, not a live E2E call to the running server).

🤖 Generated with Claude Code

Confidence Score: 2/5

The PR is not safe to merge until live customer CRM data is removed and the outstanding SQL tenant-scope and healing-verification defects are addressed.

Findings

1.  <img alt="Security" src="h