Smoke-testing a real agent in a deployed environment

Use this to verify that an agent works end-to-end in prd, a preview slot, or a running local dev server. The smoke uses the current itx CLI path rather than the removed project oRPC procedures.

Prerequisites#

The target environment is deployed and healthy.
Run commands from apps/os.
Use --project os with Doppler so APP_CONFIG_BASE_URL and APP_CONFIG_ADMIN_API_SECRET are available.

doppler run --project os --config prd -- pnpm cli itx --help

Swap prd for preview_N as needed. For local dev, start pnpm dev first and pass --base-url http://localhost:<port> using the port in apps/os/.dev-server/dev-server.json.

Procedure#

1. Create a throwaway project#

doppler run --project os --config prd -- pnpm cli itx run \
  --eval 'const slug = `agent-smoke-${Date.now()}`; const project = await itx.projects.get(slug).create({}); return await project.__describe()'

create returns the project itx handle; describe() prints the projectId (prj_…) the next steps need.

2. Run the one-turn agent smoke#

doppler run --project os --config prd -- pnpm cli itx agent-smoke \
  --project <prj_id> \
  --agent-path /agents/smoke \
  --message "PING"

The command connects to the agent over itx and calls agent.ask({ message }) — the server-side send-and-wait: it appends a user-role events.iterate.com/agents/context-added item to the agent stream and resolves on the events.iterate.com/agents/web-message-sent reply. On success it prints one JSON object with the assistant message and response event. On agent errors or timeout it exits non-zero.

For a custom timeout:

doppler run --project os --config prd -- pnpm cli itx agent-smoke \
  --project <prj_id> \
  --agent-path /agents/smoke \
  --message "PING" \
  --timeout-ms 180000

3. Inspect the stream manually#

doppler run --project os --config prd -- pnpm cli itx run \
  --context <prj_id> \
  -e 'return await itx.streams.get("/agents/smoke").getEvents({ limit: 100 })'

A healthy turn should include the user context item, LLM request lifecycle events, the generated itx script execution events, and the web-message-sent event. Agent replies are itx TypeScript scripts; for the PING prompt a correct reply usually calls:

await itx.chat.sendMessage("PONG");

4. Clean up#

There is no project-delete API on itx; throwaway smoke projects are cheap and simply accumulate. Use an obviously-disposable slug.

Gotchas#

--agent-path must live under /agents, and --project takes the project ID (prj_…), not the slug.
Use a fresh throwaway project. Messaging a real agent can send real user-visible messages.
If the CLI cannot find the admin API secret, make sure the command is wrapped with doppler run --project os --config <cfg> -- ....

What this validates#

Project, Agent, Stream, and itx Durable Objects wake under the deployed code.
Stream subscriptions deliver events.
A real LLM turn starts, completes, runs the generated itx script, and sends a visible web-channel response.