Connection Pooling

Reuse database and HTTP connections instead of opening a new one per request, cutting latency and resource usage.

Why

Opening a database connection is expensive — it involves a TCP handshake, TLS negotiation, authentication, and session setup, totaling 50-100ms. Without a pool, every request pays this cost, and under load the database runs out of connections (Postgres default max is 100) and starts rejecting requests. A connection pool keeps a set of warm connections and hands them out per request, so the per-request cost is 0ms (the connection is already open) and the total connections are bounded by the pool size. This is the difference between a service that handles 100 RPS (no pool, connection per request) and 10000 RPS (pool, reused connections). Every database client and HTTP client supports pooling; the only question is whether you configure it correctly.

When

Always use a pool for database and HTTP connections in any service that handles more than 1 request. Configure the pool size based on the database's max connections and the number of service instances.

How

1. Enable pooling in your database client
   - Node/pg: new Pool({ max: 20, idleTimeoutMillis: 30000 })
   - Python/psycopg: psycopg2.pool.SimpleConnectionPool(minconn=5, maxconn=20)
   - Java/HikariCP: HikariConfig.setMaximumPoolSize(20)
   - Go/database/sql: db.SetMaxOpenConns(20); db.SetMaxIdleConns(5)

2. Size the pool correctly
   - Formula: pool_size = (db_max_connections - 10) / num_service_instances
   - Example: DB allows 100 connections, 4 service instances → pool size = 22 per instance
   - Too small: requests queue waiting for a connection (latency spikes)
   - Too large: DB runs out of connections (other services starve)

3. Configure idle and lifetime timeouts
   - idleTimeoutMillis: close connections idle for > 30s (frees DB resources)
   - connectionTimeoutMillis: fail fast if no connection available in 5s (don't queue forever)
   - maxLifetime: recycle connections every 30 min (prevent stale connections after DB restart)

4. Use a single shared pool per process, not per request or per module
   - Create the pool once at startup, share it across all handlers
   - A pool per module creates N × M connections and exhausts the DB
   - Use dependency injection or a singleton to share the pool

5. For HTTP clients, use a persistent connection agent
   - Node: new http.Agent({ keepAlive: true, maxSockets: 50 })
   - Python: httpx.AsyncClient() (connection pooling built in)
   - Go: http.Client with a custom Transport (MaxIdleConnsPerHost)
   - Reuse the client instance — do not create a new one per request

Example

// Database connection pool (Node + pg)
import { Pool } from "pg";

// Create ONE shared pool at startup — do not create per request
export const db = new Pool({
  connectionString: process.env.DATABASE_URL,
  max: 20,                    // max connections in the pool
  idleTimeoutMillis: 30000,   // close idle connections after 30s
  connectionTimeoutMillis: 5000, // fail fast if no connection in 5s
  maxLifetimeSeconds: 1800,   // recycle connections every 30 min
});

// Use the pool — checkout and release is automatic
async function getUser(id: string) {
  // pool.query checks out a connection, runs the query, returns it
  const { rows } = await db.query("SELECT * FROM users WHERE id = $1", [id]);
  return rows[0];
}

// HTTP client with connection pooling (Node)
import http from "node:http";
import https from "node:https";

const agent = new https.Agent({
  keepAlive: true,        // reuse connections across requests
  maxSockets: 50,         // max concurrent connections per host
  maxFreeSockets: 10,     // keep 10 idle sockets for reuse
  timeout: 30000,         // socket idle timeout
});

// Share the agent across all HTTP calls
export const httpClient = new https.Agent({ keepAlive: true });
// fetch(url, { agent: httpClient }) — reuse connections

// Pool sizing formula:
// DB max connections: 100
// Service instances: 4
// Pool per instance: (100 - 10) / 4 = 22
// Leave 10 connections for admin, migrations, monitoring

Common Problems

⚠Creating a new database connection per request (new Client() in every handler) — each request pays 50-100ms of connection setup, and under load the database runs out of connections and rejects requests.
⚠Setting the pool size too high (100 per instance with 10 instances and a DB max of 100) — the instances collectively request 1000 connections, the DB allows 100, and 900 requests queue or fail.
⚠Creating a separate pool per module or per request instead of one shared pool — 10 modules each with a pool of 20 creates 200 connections, exhausting the database that allows 100.
⚠Not setting idle or lifetime timeouts — connections sit idle forever, the DB eventually closes them, and the pool hands out a stale connection that fails with a connection reset error.

Best Practices

✓Create one shared pool per process at startup and inject it into all handlers — a single pool bounds the total connections and ensures reuse across the entire application.
✓Size the pool with the formula (db_max_connections - 10) / num_instances — this leaves headroom for admin connections and ensures all instances collectively stay under the DB limit.
✓Set idle timeout (30s) and max lifetime (30 min) so stale connections are recycled — this prevents the pool from handing out connections that the database has silently closed.
✓Use keep-alive agents for HTTP clients with a shared maxSockets setting — this eliminates the TCP handshake on repeated calls to the same API, cutting latency by 50-100ms per call.