The Hidden Pitfalls of OpenRouter: One Endpoint, Different Behavior

One of the main selling points of OpenRouter sounds tempting: the service itself “handles fallbacks and selects the most cost-effective option for each request.” You call a single endpoint for the model you want, and the system routes the request to the best available backend provider. Convenient — until you start digging into what lies behind that abstraction.

Mohamed Moustafa, whose note Simon Willison links to, lists a whole set of problems that stem from this kind of automation. The root of it is that different providers run different server software — with different optimizations and settings. As a result, one and the same OpenRouter endpoint can serve requests to a supposedly single model, yet behave differently from one call to the next.

One Model, Different Answers

The discrepancies aren't just trivial details. Some providers don't support working with images at all — even when it comes to models that are formally considered multimodal (vision). The way the parameter that sets the “reasoning effort” is handled also differs. In other words, the behavior you're counting on during development can quietly change simply because the request went to a different backend.

For anyone building an application on top of an LLM, this is an unpleasant source of instability: the code and prompts stay the same, but the result “drifts” for no visible reason.

How to Take Back Control

The good news is that routing can be managed manually. OpenRouter provides a provider.only option that lets you hard-pin exactly which provider your requests go to. And to figure out what you even have to choose from, there's the /endpoints method — it returns a list of available providers for a specific model identifier.

The takeaway is simple: the convenience of automatic routing has its price. If reproducibility and predictability matter for your task — say, you need guaranteed image support or stable handling of reasoning — you're better off not relying on the “magic” of picking the most cost-effective backend, but explicitly specifying the provider yourself.

Source: Simon Willison