A familiar API.
A whole new model layer.

Create an account, verify your email, add credit and generate an API key. Replace the example host with your gateway domain.

from openai import OpenAI

client = OpenAI(
    base_url="https://api.example.com/v1",
    api_key="YOUR_RELAY_API_KEY"
)

response = client.chat.completions.create(
    model="auto/balanced",
    messages=[
        {"role": "user", "content": "Hello, Relay."}
    ],
    max_tokens=1024
)
print(response.choices[0].message.content)

Control your routing

Choose an exact catalog model ID, or use auto, auto/cheap, auto/balanced and auto/premium. Rules consider capabilities, tier, availability and configured scores. Task classification uses lightweight heuristics; low-confidence requests default to balanced when no tier is specified.

Explicit models only fall back when you send extra_body={"fallback": True}. Retries remain in the same tier. Ambiguous timeouts require reconciliation to avoid duplicate billable execution.

Streaming, vision & tool calling

Use stream=True to receive SSE chunks. The response identifies the actual model in its model field and X-Routed-Model header. Streaming does not switch providers after output begins. Vision accepts inline PNG, JPEG or WebP image data; remote image URLs are disabled. Model capabilities and supported parameters determine which tools and options are accepted.

Budgeting and idempotency

Maximum context input plus capped output is reserved as a conservative budget. Settlement uses the price version captured for the actual model and its returned usage. Uncertain usage retains a hold until an administrator reconciles it. Cache-write billing and provider-hosted tools require separate pricing and are disabled.

Send a unique Idempotency-Key header. Repeated requests return 409 without executing again. Accepted requests and responses are saved privately in Conversation history. Streaming chunks are saved as they arrive. Read history with the same API key at /v1/history; repeating a completion request still returns 409.