Chat completions

The request#

The body must be a JSON object with a model. The relay reads a few fields for billing and streaming, replaces model with the internal route, and forwards everything else to the model unchanged — messages, temperature, top_p, stop, tools, response_format and so on. Whether a model honours a given parameter is up to the model. If it rejects one, you get a generic 400; the model's own error text is not forwarded.

Parameters that matter here#

model

Required. model-zero, model-one-flash or model-one. Missing: 400 missing_model. Unknown or disabled: 404 model_not_found.

max_tokens / max_completion_tokens

Sets the hold: the most output the relay reserves credit for before the request runs. If both are set, max_completion_tokens wins. If neither is set, the relay reserves 8,192 output tokens.

A hold is not a cap on the bill — you pay for the tokens actually produced. Set it anyway: it keeps parallel requests from blocking each other and bounds what a disconnected request can cost. See Holds.

stream

true streams server-sent events. Any other value, including a truthy string, means no streaming.

stream_options.include_usage

true adds a final chunk with an empty choices array and the usage object. Some models also put a running total on earlier chunks; the last usage you see is the final count. Without the option, no chunk carries usage. You are billed the same either way.

reasoning_effort

How much the model thinks before answering. Values are translated per model — see Reasoning.

n

Multiplies the hold (up to 16). The models may still return a single choice; do not rely on n greater than 1.

Streaming#

Streams use the OpenAI wire format: data: {…} events with chat.completion.chunk objects, ending with data: [DONE]. The SDKs parse them as usual.

import os
from openai import OpenAI

client = OpenAI(base_url="https://api.hmbrd.dev/v1", api_key=os.environ["HUMMINGRELAY_API_KEY"])

stream = client.chat.completions.create(
    model="model-one-flash",
    max_tokens=2048,
    stream=True,
    stream_options={"include_usage": True},
    messages=[{"role": "user", "content": "Write a haiku about relays."}],
)

usage = None
for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)
    if chunk.usage:
        usage = chunk.usage  # the last one seen is the final count

print("\n", usage)

import OpenAI from "openai";

const client = new OpenAI({ baseURL: "https://api.hmbrd.dev/v1", apiKey: process.env.HUMMINGRELAY_API_KEY });

const stream = await client.chat.completions.create({
  model: "model-one-flash",
  max_tokens: 2048,
  stream: true,
  stream_options: { include_usage: true },
  messages: [{ role: "user", content: "Write a haiku about relays." }],
});

let usage;
for await (const chunk of stream) {
  process.stdout.write(chunk.choices[0]?.delta?.content ?? "");
  if (chunk.usage) usage = chunk.usage; // the last one seen is the final count
}
console.log("\n", usage);

data: {"id":"…","object":"chat.completion.chunk","model":"model-one-flash","choices":[{"index":0,"delta":{"content":"Signal"}}]}

data: {"id":"…","object":"chat.completion.chunk","model":"model-one-flash","choices":[{"index":0,"delta":{"content":" hops"}}]}

data: {"id":"…","object":"chat.completion.chunk","model":"model-one-flash","choices":[],"usage":{"prompt_tokens":14,"completion_tokens":23,"total_tokens":37}}

data: [DONE]

Errors inside a stream#

Once the first byte is sent, the HTTP status is already 200. If the model fails mid-stream, the relay sends one error event and then [DONE]:

data: {"error":{"message":"The upstream stream was interrupted.","type":"upstream_error","code":"upstream_stream_error"}}

data: [DONE]

A generation is never restarted midway. Output that already streamed is billed; see How a request settles.

Reasoning#

The models reason before they answer. Reasoning tokens are counted in completion_tokens and billed as output, whether or not the reasoning text is returned to you. Some models return it in a separate field of the message or delta (reasoning_content or reasoning); read content for the answer.

Set reasoning_effort to trade depth for cost. The relay accepts the values below and translates them into what the model understands. A level a model does not support returns 400.

• Turning reasoning off works on model-zero. model-one and model-one-flash reject none / off with a 400 — use low to keep them brief.
• medium is sent as high: there is no separate medium level.
• model-one-flash can be served by a fallback route when its primary one is unavailable. There, medium stays medium, xhigh and max become high, and off turns reasoning off.
• Omit the field to use the model's default.

curl https://api.hmbrd.dev/v1/chat/completions \
  -H "Authorization: Bearer $HUMMINGRELAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "model-zero",
    "reasoning_effort": "none",
    "max_tokens": 512,
    "messages": [{ "role": "user", "content": "Reply with OK." }]
  }'

Text completions#

POST /v1/completions takes a prompt instead of messages and returns choices[].text, as in OpenAI's legacy endpoint. Billing and errors are the same as for chat.

curl https://api.hmbrd.dev/v1/completions \
  -H "Authorization: Bearer $HUMMINGRELAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "model-zero",
    "prompt": "The three laws of thermodynamics are",
    "max_tokens": 256
  }'

Streaming is simulated. With stream: true, the relay fetches the whole completion first and then sends it as a short stream: one event with the full text, a usage event if you set stream_options.include_usage, and [DONE]. Everything arrives at once when the generation ends, so there is no early first token. This is how the relay can bill these requests accurately. Use chat completions when you need tokens as they are generated.

Not supported#

Embeddings (no model serves them), the Responses API, Assistants, batch, files, fine-tuning, moderation, images and audio. Paths the API does not have return 404 not_found; GET on a completion endpoint returns 405 method_not_allowed.