Agents

An Agent is the AI system you want to test — a call-center bot, customer-service assistant, or any voice or text agent. RubricHQ connects to your agent the same way a real user would (a phone call, a web call, or a text chat), runs your scenarios against it, and scores the results.

Anatomy of an agent

When you create an agent (AI Testing Agent → Create Agent) you configure:

• Name — a label to identify the agent across runs and reports.
• Instructions — the system prompt that defines how the agent behaves. This is also what the Prompt Optimizer iterates on.
• First message — the opening line the agent speaks when a call connects.
• Model — the LLM powering the agent (GPT-4o, Gemini, and others).
• Language & voice — the speech language and voice provider used for voice channels.
• Phone number — required for phone testing; the number RubricHQ dials.
• Web call URL — the endpoint RubricHQ connects to for web (WebSocket / WebRTC) testing.

Channels

An agent can be tested over three channels. RubricHQ picks the default automatically — phone if the agent has a phone number, otherwise web — and you can override it per run.

• Phone — a real call placed over Twilio.
• Web — a browser / WebSocket voice call (LiveKit, Daily, Vapi, or Retell transports).
• Text — a text-only conversation, with no audio.

Internal vs. external agents

• Internal — an agent RubricHQ orchestrates end to end, including provisioning the voice room and joining it.
• External — an agent hosted on a third-party platform (e.g. Vapi or Retell). RubricHQ connects over the vendor’s transport and drives the conversation as the caller.

For web transports that use rooms (LiveKit, Daily), you choose who creates the room:

• RubricHQ-managed — we create the room and your agent joins it.
• Client-managed — your bot runner creates the room and hands RubricHQ the credentials to join.

Configuring the web channel through the API

Everything on the dashboard’s Web channel card can be set with the web_channel object on POST /agents and PATCH /agents/{id}. transport picks how RubricHQ reaches your agent, and decides which other fields apply:

curl -X POST https://api.rubrichq.io/api/public/v1/agents \
  -H "Authorization: Bearer $RUBRIC_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Nora",
    "phone_number": "+15550001234",
    "language_id": 15,
    "instructions": "You are Nora, a collections agent.",
    "opening_message": { "type": "assistant-waits-for-user" },
    "web_channel": {
      "transport": "websocket",
      "websocket_url": "wss://agent.example.com/ws?agent_id=nora"
    }
  }'

A few things worth knowing:

• Updates merge. PATCH only the keys you are changing — {"web_channel": {"websocket_url": "wss://…"}} leaves everything else as it was. Send "web_channel": null to remove the web channel.
• Room management (livekit / daily): rubric means we create the room with your credentials; client means we call your bot_runner_url and you return the room. bot_runner_url is required in client mode.
• Credentials are optional and write-only. Leave them out to use the workspace credentials from Settings → Integrations. They are never returned — responses show livekit_api_secret_set: true instead.
• URLs must be publicly reachable. localhost and private-network addresses are rejected.
• To run a scenario over the web channel, pass "channel": "web" to POST /scenarios/{id}/run.

Standard metrics

Every voice run is automatically scored with standard metrics (latency, interruptions, silence, and more — see Metrics & Evaluation). You can disable specific standard metrics per agent if they don’t apply to your use case.