Libra CodeHub

CedarCopilot/cedar-mail

Branch: staging

fix(retrieval): keep link URLs in agent context, steer to whole-transcript reads, drop the phantom hybrid mode

merged#2599CedarCopilot

CedarCopilot wants to merge 2 commits into staging from fix/preserve-email-links

Live on prod since Wed, 1 of 4 behaviors workingTimeline and evidence
  1. Opened
    Aug 26, 2026, 6:59 PM
  2. Aug 26, 2026, 7:23 PM
  3. Merged
    Aug 26, 2026, 7:25 PM
  4. Live on staging
    Aug 26, 2026, 7:25 PM
  5. Live on prod
    Aug 26, 2026, 7:25 PM
  6. Observed 2 days
    Aug 26, 2026, 7:25 PM
  7. Pipelines steady after this deploy
    Aug 26, 2026, 7:25 PM
  8. Pipelines steady after this deploy
    Aug 26, 2026, 7:25 PM
  9. Verified

    Live on prod since Wed, 1 of 4 behaviors working

    Aug 29, 2026, 11:23 AM

Behaviors Libra is checking

Agents are guided to use FTS search for literal strings such as URLs, order numbers, names, and attachment markers, while reserving vector search for semantic similarity.Brokenmedium confidence

The deployed search-semantic path produced 33 concrete post-deploy search-turbopuffer executions with successful result logs (Found 8–30 results), but every observed input explicitly used searchType=\"vector\" and there were 0 fts calls. Thus the required exact-token/URL FTS.

staging, checked Aug 29, 2026, 6:24 PM
Agents are instructed to resolve a named call or email to its event ID and read the complete event with get/expandEventId rather than answering from partial semantic-search snippets.Workingmedium confidence

Post-deploy production telemetry shows positive runtime evidence for the conversation-read behavior: 33 successful search-turbopuffer executions (23 on Aug 27, 10 on Aug 28) with no exact [search-turbopuffer] Error logs, plus 747 get invocations carrying an explicit.

prod, checked Aug 29, 2026, 11:21 AM
The semantic-search tool no longer offers the unsupported hybrid search mode, preventing agents from selecting a mode that silently performs only vector search instead of promised vector-plus-keyword matching.Workinglow confidence

Verifier did not provide positive runtime evidence tied to this intent, so Libra is keeping it inconclusive. Original assessment: In staging since 2026-08-27T02:25:22Z, telemetry shows 33 search-turbopuffer invocations and 33 corresponding completions, with 0 errors. All 33.

staging, checked Aug 29, 2026, 10:23 AM
Email HTML converted for agent context now retains eligible HTTP(S) anchor URLs as Markdown or bare URLs so agents and semantic search can retrieve links embedded in prior messages.Inconclusivelow confidence

Unable to complete the production verification because the telemetry query found no matching OTEL spans, and the required follow-up telemetry query could not be executed in this loop.

prod, checked Aug 29, 2026, 10:19 AM
Agent context continues to omit non-HTTP(S), inline-image, and excessively long anchor targets while retaining any visible link text, preventing image payloads, unsafe schemes, and tracking redirects from consuming context.Inconclusivelow confidence

Verification reached its 120s wall-clock budget before a tied verdict. Libra is keeping this intent verifying until the next check.

prod, checked Aug 29, 2026, 10:21 AM
Semantic conversation search exposes only vector and FTS modes so agents cannot select the unimplemented hybrid mode that silently behaved as vector-only search.Inconclusivelow confidence

Verification reached its 120s wall-clock budget before a tied verdict. Libra is keeping this intent verifying until the next check.

prod, checked Aug 29, 2026, 11:23 AM

Failures attributed to this change

No prod customers are affected while this is only in staging. If promoted, prod impact is unknown because Libra still needs a concrete exception, route, and failed user action before assigning severity. 0 hits · no retained affected-user count · no retained trace sample.Introducedmedium confidence

internal_only

staging, first seen Aug 26, 2026, 11:02 PM
No prod customers are affected while this is only in staging. If promoted, prod impact is unknown because Libra still needs a concrete exception, route, and failed user action before assigning severity. 0 hits · no retained affected-user count · no retained trace sample.Introducedmedium confidence

internal_only

staging, first seen Aug 27, 2026, 10:01 AM
No prod customers are affected while this is only in staging. If promoted, prod impact is unknown because Libra still needs a concrete exception, route, and failed user action before assigning severity. 0 hits · no retained affected-user count · no retained trace sample.Introducedmedium confidence

internal_only

staging, first seen Aug 27, 2026, 10:01 AM
No prod customers are affected while this is only in staging. If promoted, prod impact is unknown because Libra still needs a concrete exception, route, and failed user action before assigning severity. 0 hits · no retained affected-user count · no retained trace sample.Introducedmedium confidence

internal_only

staging, first seen Aug 27, 2026, 10:20 AM

Libra has verdicts on 1 of 4 tracked behaviors on prod; 3 are still being checked. Libra checks hourly for 3 days after each deploy.

Three retrieval fixes from one customer's reports. All are cases where the agent looked bad at finding things, and the cause was upstream of retrieval.

1. Link URLs were destroyed before anything could find them

"the chat cannot retrieve basic things from previous emails e.g. links embedded in text"

htmlToAgentMarkdown maps headings, lists, tables and breaks to Markdown, then runs a catch-all <[^>]+> strip. <a href> had no case above that line, so every anchor lost its URL , before the model read the mail, and before the text was embedded for search.

That was collateral, not a decision. The catch-all exists so <img> payloads (base64 and /api/inline-image) can never reach the model , mail-sync-memory-fix Ph4. The existing test asserted the resulting behaviour rather than a requirement, so it's updated with a comment saying why.

Anchors now render [text](url), with the Ph4 guarantees intact:

  • http(s) only , data: is refused outright, since a data: href would reintroduce exactly the base64 Ph4 removed
  • /api/inline-image stays suppressed, including as an href rather than an img src
  • hrefs over 400 chars dropped, so a tracking redirect can't cost hundreds of tokens; visible text survives
  • a bare autolink (text === href, which Gmail produces constantly) emitted once, not [url](url)
  • an image wrapped in a link reduces to the link, never the image data

Reaches every surface reading mail through this converter: message bodies, draft bodies, activity bodies, taxonomy extraction.

2. Whole-transcript reads when the user names one call

"Don't look verbatim, but just look at the transcript and see"

The agent searched a 90-minute call and returned quoted fragments. Search gives a top-k slice , 10-15 chunks of a ~10,000-word transcript. "What did they mean" and "what did we decide" can't be answered from a sample.

Both halves already sat one line apart in conversation-read's description: action (1) get with expandEventId is the family's wrapper over fetch-full-event-content and reads the event whole. Nothing pointed at it. Action (3) now does.

Edited on the family surface, not the granular tools , buildMasterFamilyToolset() hands the chat agents the shared FAMILY_TOOLS facade, so the granular descriptions never reach the model. Scoped to one named event deliberately: these transcripts run 19k,60k chars, so a general "prefer whole reads" rule would put hundreds of KB in a turn.

3. hybrid search was a silent no-op

if (searchType === 'vector' || searchType === 'hybrid') {   // vector leg
if (searchType === 'fts') {                                  // BM25 leg
// TODO: Implement hybrid search (vector + FTS with fusion) if needed in the future

hybrid ran vector and nothing else , while the enum offered it and its description promised "combines both." An agent reaching for keyword matching silently got pure semantic search.

Removed from the tool enum (no caller passes it). The service type and branch stay, so the seam remains for whoever implements the fusion. The two real modes now say when each is right rather than just what they are: vectors for meaning, fts for a string that must appear verbatim , an order number, a URL, a name , because vectors blur exactly those.

Verification

  • pnpm --filter @zero/server run types clean
  • pnpm deps:check clean (1572 modules)
  • eslint clean on all changed files
  • html-to-agent-markdown 21 tests (was 14), covering every new rule: data-URI refusal, inline-image suppression as an href, the 400-char cap, autolink dedup, image-in-link, links inside list items
  • master-surface + context-formatting suites: 47 passing
  • No test asserted the hybrid enum or the old description

Not included

Re-embedding existing mail. Fix 1 only helps new content until a backfill runs; backfill-workflow.ts exists if you want it.

🤖 Generated with Claude Code

Show production surfaces and changed-file mapping

Production surfaces

Libra has not measured any production surfaces for this change yet.

Changed files → surfaces

  • apps/server/src/mastra/tools/conversation/conversationTool.tsno production surface mapped
  • apps/server/src/mastra/tools/conversation/searchTurbopufferTool.tsno production surface mapped
  • apps/server/src/mastra/utils/context-formatting/__tests__/html-to-agent-markdown.test.tsno production surface mapped
  • apps/server/src/mastra/utils/context-formatting/event-context.tsno production surface mapped
  • apps/server/src/services/turbopuffer/search.tsno production surface mapped