Media followups (round 3)
Status summary: implemented, both live e2e lanes green against local dev. All five checklist items done; deep-link resolver is origin-strict and unit-tested; the agent-retrieval e2e finds the swimming lesson among decoys through the same door agents use.
Checklist#
- drawer label
Media→/media, matching the/reposconvention (Integrations in #2453 slots into the same drawer style) - clearer image paths mediaFilePath in lib/media.ts: files move from
/media/inbound/<sha256>-<name>to/media/<sha256>-<name>— one namespace, so a media file's path is/media/whatever.png; document the mental model in the media-search example description (stream/media= events, files/media/<hash>-<name>= bytes; payloadpathis authoritative, so pre-rename captures keep resolving) - in-app deep-links for special streams lib/in-app-links.ts + components/markdown.tsx: the shared Markdown component's
onLinkPresscurrently sends every URL to the system browser; add a small pure resolver mapping same-deployment URLs to in-app routes —/media/...→ the Media screen (item focused via search),/repos/...→ the repo screen — falling through toLinking.openURLotherwise. Extensible table so/integrations/...etc. can join later. - markdown-render media descriptions media.tsx, collapsed rows clip via maxHeight in the list (reuse the chat
Markdowncomponent, preview mode collapsed / full when expanded) - agent-retrieval e2e reconstructing the sushi dogfood thread e2e/media-agent-retrieval.e2e.test.ts, AppKit-rendered fixtures: capture checked-in fake screenshots (a swimming-lesson email screenshot + two decoys), then run the media-search example the way an agent would and assert the swimming lesson's date/time is retrievable by keyword search over transcripts
Guesses and assumptions#
- [guess]
/media/<hash>-<name>(dropinbound/) is the "clearer path" — hash prefix keeps content-addressed dedup, the flat namespace matches the question "is it /media/whatever.png?" - [guess] deep-link focus lands on the Media screen with the search box prefilled from the linked filename, rather than a dedicated item route — cheaper, and search-by-filename uniquely finds the item
- [guess] the eval drives the media-search example via runScript (the same door agents use) rather than a full LLM agent turn — deterministic, cheap, and still proves the retrieval path end to end; a full agent-turn eval can layer on later
Out of scope (tracked elsewhere)#
- QR-scan freshness + state-mismatch switching (separate PR, in flight)
- full-screen viewer chrome (pinch-zoom, tap-toggled overlay, swipe-down)
- identify source app/website per item
- MediaProcessor reduced state +
itx.mediacapability; embedding search
Implementation log#
- (starting) seam confirmed:
apps/mobile/src/components/markdown.tsxonLinkPressis the single interception point for chat links.