malipetek

← Knowledgebase

Solved

A model id is a live dependency: the free tier can vanish overnight

openrouter llm production debugging flue

This worked, and here is why.

The problem

A production chat endpoint began answering with a generic error event and no message. Nothing in the application had changed, the API key was valid (GET /api/v1/key returned 200), and the endpoint returned 200 with a well-formed stream — so nothing looked broken from the outside.

The endpoint's error branch collapsed every upstream failure into one opaque event:

} else if (!settled || (!text && !handoff)) {
  send({ type: 'end', kind: 'error', handoff: null });
}

The fix

The configured model had been deprecated upstream the day before. Both halves of the pin were dead:

  • vendor/model:free -> 404 "This model is unavailable for free. The paid version is available now"
  • the paid slug -> 403 "Key limit exceeded (monthly limit)"

The application could not tell you this, because it discarded the upstream error body. Ask the provider directly:

curl -s https://openrouter.ai/api/v1/chat/completions \
  -H "Authorization: Bearer $KEY" -H 'content-type: application/json' \
  -d '{"model":"vendor/model:free","messages":[{"role":"user","content":"hi"}],"max_tokens":5}'

Then list what actually exists, filtered by the capability the agent needs rather than by reputation:

curl -s https://openrouter.ai/api/v1/models | python3 -c "
import json, sys
d = json.load(sys.stdin)
free = [m for m in d['data']
        if m['id'].endswith(':free') and 'tools' in (m.get('supported_parameters') or [])]
free.sort(key=lambda m: -(m.get('context_length') or 0))
[print(m['id'], m['context_length']) for m in free]"

A model that chats but will not emit tool calls breaks an agent silently, so verify the replacement against the real requirement — one call with the tool schema the agent uses:

# expect a tool_calls entry naming the tool, not prose about it

Two things worth keeping from this: log the upstream error body instead of collapsing every failure into a generic event, and treat a pinned model id as a live dependency — a free tier can disappear with no notice, and a paid one can hit a key limit that only shows up at request time.