The One Line Missing From Most Production Code: Timeouts

 By the team at DevSpero, practical software engineering for developers who ship.

Your service works perfectly in development. Tests pass, staging looks great, and the deploy goes out without a hiccup. Then one afternoon a third-party API gets slow. It isn't down, just slow. Within minutes your whole application is unresponsive, dashboards are red, and your on-call phone is buzzing.

The bug isn't in your logic. It's in something you never wrote: a timeout.

Why "slow" is worse than "down"

When a dependency is down, your call fails fast. You get "connection refused," your error handling kicks in, and life goes on.

When a dependency is slow, your call just waits. Every waiting request holds onto a thread, a connection, or a worker. New requests keep arriving, the pool fills up, and eventually your perfectly healthy service has no capacity left to serve anyone.

This is a cascading failure: one slow dependency takes down everything that depends on it, which takes down everything that depends on that.

User → Your API → Payment Service (slow) 
                       ↓
        Threads pile up waiting...
                       ↓
        Thread pool exhausted
                       ↓
        Your API stops responding to everyone

The default is often "wait forever"

Many popular libraries ship with no timeout at all. Here are the common traps:

Python (requests)

import requests

# Dangerous: can hang indefinitely
requests.get("https://api.partner.com/data")

# Better: (connect timeout, read timeout)
requests.get("https://api.partner.com/data", timeout=(3.05, 10))

JavaScript (fetch)

// Dangerous: fetch has no built-in timeout
await fetch("https://api.partner.com/data");

// Better: abort after 5 seconds
await fetch("https://api.partner.com/data", {
  signal: AbortSignal.timeout(5000),
});

Go (net/http)

// Dangerous: http.DefaultClient has no timeout
resp, err := http.Get("https://api.partner.com/data")

// Better: configure a client with a timeout
client := &http.Client{Timeout: 5 * time.Second}
resp, err := client.Get("https://api.partner.com/data")

The same problem shows up in database drivers, Redis clients, message queue consumers, and gRPC calls. If you haven't set a timeout explicitly, assume there isn't one.

How to choose a timeout value

There's no universal number, but there is a sound method.

Connect timeout: keep it short (1 to 3 seconds). If a connection can't be established quickly, something is wrong, and waiting longer rarely helps.

Read timeout: base it on real data. Look at the dependency's p99 latency and add headroom. If p99 is 800ms, a 2 to 3 second timeout is sensible. A 60-second timeout is not a safety net, it's an outage waiting to happen.

Total deadline: shorter than your caller's patience. If your API promises a response within 5 seconds, every downstream call (including retries) has to fit inside that budget. Otherwise your caller gives up while your service keeps working on a request nobody is waiting for.

Client timeout:        10s
  └─ Your API budget:   8s
       ├─ DB query:      2s
       ├─ Partner API:   3s  (+ 1 retry = 6s worst case)
       └─ Buffer:        ~1s

Timeouts need friends

A timeout alone converts a hang into an error, which is progress, but not the whole answer. Pair it with these three patterns.

1. Retries with exponential backoff and jitter

Retry transient failures, but do it politely:

import random, time

def call_with_retry(fn, attempts=4, base=0.2, cap=5.0):
    for attempt in range(attempts):
        try:
            return fn()
        except TransientError:
            if attempt == attempts - 1:
                raise
            delay = min(cap, base * (2 ** attempt))
            time.sleep(random.uniform(0, delay))  # full jitter

Two rules matter here. Only retry idempotent operations (or ones protected by an idempotency key), and always add jitter so thousands of clients don't retry at the exact same instant and hammer a recovering service.

2. Circuit breakers

After a threshold of consecutive failures, stop calling the dependency for a cooldown period and fail immediately. This protects your resources and gives the struggling service room to recover. Libraries like Resilience4j (Java), Polly (.NET), and opossum (Node.js) implement this for you.

3. Fallbacks

Where possible, degrade gracefully instead of failing outright: serve cached data, show a default recommendation list, or queue the action for later. A slightly stale response usually beats an error page.

A quick audit checklist

Run through your codebase this week and check each item:

• Every outbound HTTP call has an explicit timeout
• Database queries have a statement timeout
• Connection pools have a maximum wait time
• Background jobs have a maximum runtime
• Message consumers have a processing deadline
• Timeouts are derived from measured latency, not guessed
• Your total request budget is smaller than your caller's timeout

A useful shortcut: search your code for requests.get(, fetch(, http.Get(, and new HttpClient(. Every hit without a timeout is a candidate for your next incident.

Common mistakes

• Setting one giant timeout "just to be safe." A 60-second timeout protects nothing.
• Forgetting the connection pool. Even with a request timeout, waiting for a connection from the pool can block forever.
• Retrying without a budget. Three retries of a 10-second timeout is a 30-second request.
• Not logging timeouts. If timeouts aren't visible in your metrics, you'll only find out when users complain.

Takeaway

Treat every network call as something that will eventually be slow. If you don't decide how long you're willing to wait, the library decides for you, and its answer is often "forever."

One line of configuration per call is the cheapest reliability improvement you'll make all year.

Enjoyed this post?

This is Day 1 of a daily series on production-minded backend engineering. Visit devspero.com for more posts on reliability, performance, and building software that holds up in the real world.