API integration: how it works and what makes it hard

API integration is how your software talks to a service you don't own, over an interface that service publishes for exactly that purpose, so the two exchange data on their own instead of someone copying it by hand. A checkout page calling a carrier's API for a shipping rate is one team's code calling another team's code, changed on a schedule you don't control. System integration is the wider problem of making systems you own act as one; this is the narrower case, where only one side of the connection belongs to you.

Request-response and event styles

Most API integrations move data through one of three styles, and picking the wrong one is usually a performance problem, not a functional one.

REST answers a specific request for a specific resource at a specific URL, one call at a time. It's simple to reason about, easy to cache, and fits any case where your own code decides when it needs something. A data pipeline usually pulls from exactly this kind of endpoint on a schedule.

GraphQL keeps the request-response shape but lets the caller name exactly the fields it wants across several resources in one query, instead of chaining multiple REST calls together. It earns its complexity when a client's data needs vary screen to screen; for one predictable shape of data, it's usually more machinery than the job needs.

Webhooks flip the direction: instead of your code asking whether something happened, the vendor pushes an event to a URL you host the moment it does. That fits anything you'd otherwise poll for constantly, a payment clearing, a shipment changing status, in exchange for a public endpoint that has to assume delivery can repeat or arrive out of order.

Authentication in practice

API authentication mostly comes down to three mechanisms, and the cost of each shows up once the integration is running, not while it's being built.

An API key is a static secret sent with every request, usually in a header. It's simplest to add and easiest to get wrong: a key checked into source control stays valid until someone notices and rotates it, since nothing about it expires on its own.

OAuth 2.0 covers two situations that get conflated. A user delegating access to their own account uses the authorization code flow, which gets your app a token without seeing their password. Two systems talking directly with no user involved use the client credentials grant instead, which issues your application its own token. Access tokens are short-lived either way, so the integration has to renew one automatically or it stops working the moment a token expires.

mTLS goes further: both sides present a certificate, so the connection itself proves identity instead of a credential that could be copied and reused. It costs more to run: someone has to issue and rotate a certificate per service, and a missed rotation fails the connection outright, which is why it mostly shows up in regulated, higher-trust integrations.

Rate limits, retries, and idempotency

Every vendor caps how many requests you can make in a window, and how you respond to hitting that cap decides whether the integration is reliable more than the cap itself does.

A vendor at its limit returns 429 Too Many Requests, sometimes with a Retry-After header saying how long to wait. Retrying immediately, from every failed caller at once, only adds to the load that triggered the limit, a retry storm that keeps a struggling service throttled longer. Exponential backoff, waiting longer each time, plus jitter to randomize it, spreads retries out instead of letting them land together.

None of that makes retrying safe by itself. A request can succeed on the vendor's end while the response gets lost to a timeout, and retrying it plainly repeats whatever it did, a record created twice, say. An idempotency key fixes that: a unique value attached to one attempt, so the vendor recognizes a repeat and sends back what the first attempt already produced, rather than doing the work twice.

Mistakes that make API integrations unreliable

The same problems keep showing up once an integration carries real traffic, mostly because none of them are visible while both sides are still testing.

• A sandbox that doesn't match production. Simulating real rate limits, latency, and malformed responses is its own project most vendors skip. An integration that has only seen clean success in testing meets its first real edge case live.
• Undocumented behavior. Docs describe the happy path only. A specific error code, a field only sometimes populated, a rate limit tighter than advertised: none of it appears until the integration runs into it directly.
• Breaking changes with no version to pin to. A vendor renames a field or retires an endpoint without a version bump or notice, and the integration breaks next time that code path runs, with no changelog to explain why.
• Silent partial failure. A batch call can succeed overall and still fail for some records inside it without ever returning an error, hidden inside an ordinary-looking 200 unless something checks every item individually.

The point where integration becomes a build

One integration is a manageable task: pick an auth method, handle retries, done. That changes once a product depends on a dozen vendor APIs at once, each with its own auth scheme and failure behavior. Someone has to own that surface as a whole, or the tenth integration repeats a mistake the first nine already paid to learn.

Money moving through the integration raises the stakes on the retry problem above: duplicating a charge is a different order of mistake than duplicating a log line, and a vendor outage handled badly can double-bill a customer before anyone checks the numbers.

Compliance adds a different kind of weight, and it shows up somewhere unglamorous: most frameworks log full request and response bodies by default, and a log line holding a card number or a patient record is a liability no one meant to create. Redacting it before it's logged is a design decision, not an afterthought.

Custom software development and product development teams take over integration work at exactly this point, once it has outgrown what one engineer can own alone. The clearest version of it is payment gateway integration: the same mechanics above, concentrated where getting them wrong moves someone else's money.