Libra CodeHub

CedarCopilot/cedar-mail

Branch: staging

feat(taxonomy): continuous taxonomy extraction , GLM 5.3 Flash, event-driven + sync-time enqueue, two Jev quality gates

merged#2890CedarCopilot

CedarCopilot wants to merge 10 commits into staging from feat/continuous-taxonomy-extraction

What Libra verified

  • Continuous queue runs use GLM with bounded field coverage

    Libra ran this behavior against the change and confirmed the check detects when it breaks.

    Technical evidence
    • the continuous queue run explicitly selects GLM instead of allowing extraction to default to another model
    • the continuous queue run excludes meddpicc.decision_criteria from judgment extraction
    • a queue-run deadline aborts the underlying extraction signal instead of merely abandoning its result

    Changed code: process-taxonomy-extraction-queue.ts.

Live on prod, 0 of 1 surfaces working, 2 days leftTimeline and evidence
  1. Opened
    Sep 22, 2026, 6:19 PM
  2. Sep 23, 2026, 3:07 AM
  3. Merged
    Sep 23, 2026, 3:15 AM
  4. Live on prod
    Sep 23, 2026, 3:15 AM
  5. Observed 0 hours, 1 surface, 0 requests
    Sep 23, 2026, 3:15 AM
  6. Watching

    Live on prod, 0 of 1 surfaces working, 2 days left

    Sep 23, 2026, 3:15 AM
  7. Pipelines steady after this deploy
    Sep 23, 2026, 3:15 AM

Behaviors Libra is checking

Call-level and qualification-level taxonomy fields use aligned default values so missing extraction results have consistent behavior across both scopes.Not checked
prod
Taxonomy values are continuously refreshed as conversations are ingested or changed instead of being extracted only during a one-time processing pass.Not checked
prod
Conversation lifecycle events enqueue asynchronous taxonomy extraction so taxonomy processing starts automatically when relevant conversation data changes.Not checked
prod
Taxonomy extraction uses GLM 5.3 Flash as its production language model.Not checked
prod
The second Jev quality gate prevents taxonomy results that do not meet the configured extraction-quality requirement from replacing accepted production values.Not checked
prod
The first Jev quality gate prevents malformed or schema-invalid taxonomy model responses from being persisted as production field values.Not checked
prod

Libra found 1 production surface on prod but could not judge any of them yet. Libra checks hourly for 3 days after each deploy.

Summary

Takes the sub-event sales-taxonomy extraction pipeline out of "staff runs a CLI per audit" and makes it a standing, org-gated production capability, per docs/design/continuous-taxonomy-extraction-production.md.

Continuous execution , two active paths, the daily backfill sweep intentionally disabled (2026-09-23).

  • handleExecuteMeeting's success branch fires a fire-and-forget enqueue on each new meeting , lowest latency, live coverage.
  • The conversation-sync.extract-taxonomy step enqueues on every two-phase sync , covers a newly-opted-in org's INITIAL sync and routine re-syncs. Already flag-gated: enqueueTaxonomyExtraction re-checks organizations.settings.features.taxonomyExtractionEnabled internally, so this is a no-op for every org that hasn't opted in, and only ever fires in response to a real sync event for one that has.
  • The daily backstop-sweep cron (process-taxonomy-extraction-backfill.ts) is DISABLED , unscheduled from cron-task-registry.ts and aws/lib/stacks/app-stack.ts's cronTasksDaily (confirmed removed from the synthesized CloudFormation template). Unlike the two paths above, this one re-scans and backfills every never-covered conversation on a fixed daily schedule, independent of any single sync event or how long ago an org opted in , real backfill volume/cost/latency turned out to be large (see below), and that recurring, unbounded-lookback behavior is what's being held back for now, not ordinary forward coverage. Implementation + tests kept for later, just not wired.

Model: GLM 5.3 Flash by default, no cross-model fallback (explicit decision , if GLM can't do a plane reliably, the fix is prompt/schema engineering on that plane, not routing to a more expensive model). Retry-once-on-the-same-model on a structured-output failure, then skip that field/event without failing the whole run. Two specific instance-plane fields (demo.feature, product.pitch) are routed to the traditional (non-flash) glm-5.3 instead , see the recall-improvement work below for why, and the cost note below for the real tradeoff.

GLM ground-truth eval , done, not just planned, and iterated on same-day. Real comparison against stored Sonnet-4.6 extractions (not a fresh in-run baseline): 16 conversations, 464 instance-plane + 333 judgment-plane comparisons across Pirros/Aspireiq/Vooma. Full writeup, including every subsequent fix and its validated numbers: docs/design/taxonomy-model-switching-eval.md.

  • Found and fixed a real bug: GLM 5.3 Flash hangs past any timeout on real transcripts unless reasoning is explicitly bounded (reasoningEffort: 'low').
  • Found and fixed a second real bug: the queue consumer never explicitly requested GLM, so it would have silently defaulted to Sonnet on first real use , caught before anything was deployed.
  • Judgment plane: 0 of 333 comparisons + 30 stability re-runs hit a schema failure. 7 of 8 fields show real semantic agreement with Sonnet on manual read. meddpicc.decision_criteria is genuinely broken (39% non-prose garbage) , held back via a new excludeFieldIds denylist.
  • Instance plane: batched 5 fields/call, recall was 26.3% overall, as low as 10-17% on the highest-volume fields.

Recall-improvement pass (same day, small-sample-validated at every step, per an explicit working agreement not to scale anything unproven):

  1. Single-field-per-call for GLM's instance plane (the same fix Gemini/Haiku already needed for an identical "fill the first key(s), empty the rest" weakness). Re-verified with a fresh eval: 26.3% → 35.3% overall , real gains on moderate-volume fields (rep.question 44%→63%, prospect.question 41%→55%), but the worst fields (demo.feature, product.pitch) were essentially untouched (~9-12%), and it introduced a new failure mode: 82 (event, field) pairs came back completely empty.
  2. Verbatim-guard retry now carries an explicit correction hint instead of resending the byte-identical prompt (root cause of the 82 complet
Show production surfaces and changed-file mapping

Production surfaces

SurfaceRequestsErrorsp95UsersVerdict
taxonomy.extractno baseline → 0no baseline → 0 (0%)not measurednot countedUnavailable
ClickHouse reads are unavailable, so Libra could not measure this surface.

Changed files → surfaces

  • apps/server/docs/usage-reports/taxonomy-glm-ground-truth-eval.jsonno production surface mapped
  • apps/server/scripts/jev-taxonomy-quality-gates-prototype.tsno production surface mapped
  • apps/server/scripts/run-jev-taxonomy-quality-gates-prototype.mjsno production surface mapped
  • apps/server/scripts/run-taxonomy-glm-ground-truth-eval.mjsno production surface mapped
  • apps/server/scripts/taxonomy-glm-ground-truth-eval.tsno production surface mapped
  • apps/server/src/cron/__tests__/process-taxonomy-extraction-queue.test.tsno production surface mapped
  • apps/server/src/cron/process-taxonomy-extraction-backfill.tsno production surface mapped
  • apps/server/src/cron/process-taxonomy-extraction-queue.tsno production surface mapped
  • apps/server/src/db/schema.tsno production surface mapped
  • apps/server/src/mastra/routeHandlers/event-execution/handleExecuteMeeting.tsno production surface mapped
  • apps/server/src/mastra/tools/agenda/__tests__/reviewConversationFanOut.test.tsno production surface mapped
  • apps/server/src/runtime/worker-entry.tsno production surface mapped
  • apps/server/src/services/agent-action-queue/types.tsno production surface mapped
  • apps/server/src/services/crm/__tests__/jev-canonicalization.test.tsno production surface mapped
  • apps/server/src/services/crm/__tests__/jev-judgment-grounding.test.tsno production surface mapped
  • apps/server/src/services/crm/__tests__/taxonomy-chunking.test.tsno production surface mapped
  • apps/server/src/services/crm/__tests__/taxonomy-extraction-run.test.tsno production surface mapped
  • apps/server/src/services/crm/__tests__/taxonomy-judgments.test.tsno production surface mapped
  • apps/server/src/services/crm/__tests__/taxonomy-model.test.tsno production surface mapped
  • apps/server/src/services/crm/jev-canonicalization.tsno production surface mapped
  • apps/server/src/services/crm/jev-judgment-grounding.tsno production surface mapped
  • apps/server/src/services/crm/taxonomy-chunking.tsno production surface mapped
  • apps/server/src/services/crm/taxonomy-extraction-queue.tsno production surface mapped
  • apps/server/src/services/crm/taxonomy-extraction-step.tsno production surface mapped
  • apps/server/src/services/crm/taxonomy-extraction-stream.tsno production surface mapped
  • apps/server/src/services/crm/taxonomy-judgments.tsno production surface mapped
  • apps/server/src/services/crm/taxonomy-model.tsno production surface mapped
  • apps/server/src/services/field-values/__tests__/defaults-equivalence.test.tsno production surface mapped
  • apps/server/src/services/field-values/__tests__/instance-batching.test.tsno production surface mapped
  • apps/server/src/services/field-values/__tests__/jev-canonicalization-wiring.test.tsno production surface mapped
  • apps/server/src/services/field-values/crm-custom-field-values.tsno production surface mapped
  • apps/server/src/services/field-values/defaults/call-level.tsno production surface mapped
  • apps/server/src/services/field-values/defaults/qualification.tsno production surface mapped
  • apps/server/src/trpc/routes/org-admin.tsno production surface mapped
  • apps/server/src/workflows/step-registry/conversation-sync/__tests__/taxonomy-step.test.tsno production surface mapped
  • apps/server/src/workflows/step-registry/conversation-sync/taxonomy-step.tsno production surface mapped
  • aws/lib/runtime-contract.tsno production surface mapped
  • aws/lib/stacks/app-stack.tsno production surface mapped
  • aws/lib/stacks/data-stack.tsno production surface mapped
  • aws/lib/stacks/workflow-state-machines.tsno production surface mapped
  • docs/design/jev-taxonomy-quality-gates.mdno production surface mapped
  • docs/design/taxonomy-model-switching-eval.mdno production surface mapped