Which Services Do AI Agents Choose: Lessons from 17,000 Sessions of Claude, Codex and Cursor

When a developer asks an AI agent to add a database, a payment gateway or email delivery, the choice of a particular service is increasingly made by the agent itself — and both experienced senior engineers and “vibe coders” with no background alike rely on that choice. The team at Armature (which, as it candidly notes in its disclaimer, sells marketing services for dev tools) set out to learn exactly how these decisions get made, and staged what it calls the largest experiment of its kind.

How they counted

The researchers assembled 75 repositories across 10 languages, with fake companies, commit histories and keys, but real lockfiles. Tasks were posed on behalf of four personas — from a novice describing only symptoms to a large-enterprise engineer with compliance requirements. In all, that made for 1,163 request variants. They added a “simulated human” (built on Gemini) into the loop, because without clarifying questions the agents tended to write everything themselves. Out of nearly 17,000 runs, 5,292 valid sessions were selected for the first publication.

What they found

The agents behave differently. In two-thirds of cases Cursor leans on the web, Codex searches almost always (94%) and likes operators such as site:, while Claude Code relies on its own knowledge and reaches for the web only a third of the time — but browses three times as many pages. All three fully agree in just 42% of categories: for voice agents, Claude picks Twilio, Codex picks the OpenAI Realtime API, and Cursor picks Vapi.

The repository context decides almost everything. For the very same request about sending email, the winners differed by the project's language: Resend on TypeScript, Sendgrid on Python, Postmark on Go, Azure ACS on Java.

A storyline of its own — fame is no guarantee of getting picked. PayPal was mentioned 139 times and was never chosen once (Stripe won). LangChain is the most-mentioned framework (194 times), yet it was chosen only four times. Supabase is named more often than any other database, but loses to Neon — largely because of an imposed “bundle” of superfluous features.

Small details on a service's page can flip the decision: Mailgun regularly lost to Postmark because of the “1-day retention” wording on its free tier. At the same time, mentions of costs often worked more as a way of framing information than as a genuinely disqualifying factor.

Who wins

Some markets are monopolized: Stripe wins in 9 out of 10 cases, yielding only in specific EU-regulation scenarios (Paddle, Mollie). Neon takes 66% among databases, S3 45% among storage. Armature has published all the traces — from prompts to actual code diffs — and promises to keep digging.

Source: Hacker News