Preparing a Codebase for Horizontal Scalability (On-Premises / Self-Hosted)

Many early-stage startups and solo developers begin with a classic, streamlined stack: a React SPA frontend, a single Node.js REST API, a single database instance, and Caddy acting as a reverse proxy and SSL terminator.

A Note on Cloud vs. On-Premises:
If you run on managed hyperscalers (AWS, GCP, Azure), much of horizontal scaling is abstracted away behind managed services: AWS Application Load Balancers (ALB), CloudFront, ElastiCache, and Amazon RDS with built-in connection proxies and read replicas.

This guide is specifically tailored for on-premises, bare-metal, VPS (Hetzner, OVH, DigitalOcean), or self-hosted environments, where you don’t have managed cloud magic out of the box and must configure your own load balancing, connection pooling, and lifecycle coordination.

It is simple, cheap, and fast to develop. But when traffic spikes, vertical scaling (buying a bigger box) quickly hits diminishing returns. You decide to run 5 or 10 instances of your API instead.

Scale the Monolith First, Avoid Premature Microservices:
Horizontal scalability does not mean you have to split your codebase into microservices, adopt Kafka, or introduce complex event-driven architectures. You are running identical replicas of the exact same monolithic API. This provides the smoothest, most pragmatic growth path: you keep a single cohesive codebase while solving the real bottlenecks—statelessness, database connection pooling, data consistency, and infrastructure-level traffic distribution.

Suddenly, users get logged out randomly, database connections saturate, cron jobs fire five times simultaneously, and uploads disappear.

The 5 Pillars at a Glance Link to heading

Before diving deep, here is the breakdown of what needs to shift:

1. 1. The Edge: Decouple static asset delivery, configure health-checked least-connection routing in Caddy.
2. 2. Stateless Node.js: Eliminate local disk storage, in-memory sessions, and local caching in favor of S3 pre-signed URLs and Redis.
3. 3. The Database: Cap connection pools with PgBouncer and decouple migrations from application boot.
4. 4. Async Work: Replace in-process node-cron with distributed BullMQ workers or Temporal workflows, and use row-level locks for concurrent mutations.
5. 5. Lifecycle: Expose /healthz and handle SIGTERM to drain in-flight connections cleanly.

Architecture Overview Link to heading

Before looking at each pillar in code, here is how a horizontally scalable, production-ready system fits together:

1. The Edge: Caddy Load Balancing, Health Checks & Static Decoupling Link to heading

A common misconception is that the frontend needs to know which backend instance to reach. It shouldn’t. The React app should only ever talk to a single entrypoint: relative /api/* requests.

Let Caddy handle load balancing, health detection, and static asset delivery:

myapp.com {
    # 1. Decouple static SPA: serve directly from disk (bypass Node completely)
    root * /var/www/react-app/dist
    file_server
    try_files {path} /index.html

    # 2. Reverse proxy API traffic across backend instances
    handle /api/* {
        reverse_proxy node-1:3000 node-2:3000 node-3:3000 {
            # Route requests to the least busy instance
            lb_policy least_conn

            # Active Health Checks: ping every 5s; drop dead nodes automatically
            health_uri /healthz
            health_interval 5s
            health_timeout 2s
            health_status 200

            # Passive failover: if an in-flight request fails, retry on another node
            fail_duration 10s
            max_fails 2
        }
    }
}

By having Caddy serve the static build directly, your Node.js event loop isn’t wasted streaming JS/CSS bundles.

2. Stateless Node.js: Offload Sessions, Caches & Uploads Link to heading

Horizontal scaling requires every API instance to be completely disposable. If Request #1 hits Node-1 and Request #2 hits Node-3, local state falls apart.

The Traps & Fixes: Link to heading

• Sessions & Auth Tokens: Never use memory-backed sessions (MemoryStore). Use stateless JWTs or store server sessions in Redis (e.g., connect-redis).
• In-Memory Caches: Ditch global JavaScript objects and node-cache. If data must be shared across requests, store it in Redis with strict TTLs.
• File Uploads: Do not save uploads to local disk (e.g., fs.writeFile to ./uploads). If Node-1 saves an avatar, Node-2 won’t have it. Instead, generate S3 pre-signed upload URLs and have the React frontend upload directly to object storage.

// Good: API generates direct S3 upload link; no file bytes hit your Node servers
app.post('/api/upload-url', async (req, res) => {
  const url = await s3.getSignedUrlPromise('putObject', {
    Bucket: process.env.S3_BUCKET,
    Key: `uploads/${crypto.randomUUID()}.png`,
    Expires: 60,
  });
  res.json({ uploadUrl: url });
});

3. The Database: Connection Limits & Safe Migrations Link to heading

When you scale from 1 to 10 Node.js instances, your database suddenly experiences 10x the connection pressure.

A. Connection Pool Saturation Link to heading

If your Node app configures a default pool of 20 connections, 10 instances will attempt to open up to 200 concurrent connections. Most managed databases (PostgreSQL / MySQL) will quickly choke or reject connections.

• Solution: Keep application pool sizes small (e.g., 3–5 connections per instance) or place a dedicated connection pooler like PgBouncer in front of your database.
• Read Replicas: Direct read-only endpoints (dashboards, search) to a Read Replica, reserving the primary database instance strictly for writes.

B. Migrations: Never Run on App Boot Link to heading

Running prisma migrate deploy or knex migrate:latest inside your Docker startup command is a disaster when 5 containers boot simultaneously: they race, acquire conflicting table locks, or crash.

• Solution: Extract database migrations from application startup. Run migrations once in your CI/CD pipeline or as a one-off pre-deploy job before the new app containers start receiving traffic.
• Use Expand/Contract: Always make migrations backward-compatible (add columns as nullable first, deploy code, then enforce constraints later).

4. Async Work: Kill In-Process Crons & Lock Critical Data Link to heading

A. The “Duplicate Cron” Disaster Link to heading

If you have node-cron or setInterval scheduled to send daily invoices at midnight, scaling to 5 instances means your customers get billed 5 times.

• Solution: Decouple background work from the API. Use a distributed queue like BullMQ (Redis) or a durable execution engine like Temporal.
• For standard background jobs (emails, webhooks), BullMQ ensures dedicated worker processes pull each job exactly once across instances.
• For complex, multi-step business transactions or long-running orchestrations (e.g. payment collection followed by provisioning, retries over days, or request coalescing), Temporal eliminates race conditions and state management entirely by making execution state durable out of the box.

import { Queue } from 'bullmq';

// Any API instance can safely enqueue without duplicates
const emailQueue = new Queue('emails', { connection: redisConnection });

await emailQueue.add('sendWelcome', { userId: user.id }, {
  jobId: `welcome-${user.id}`, // Idempotency key prevents duplicate execution
});

B. Race Conditions Across Instances Link to heading

Two concurrent requests hitting two different Node instances might attempt to deduct balance from the same user or update the same resource simultaneously.

There are two primary ways to eliminate this:

1. Database-Level Pessimistic Locking: Lock the row inside a transaction across all instances:

   -- Lock the row for the duration of the transaction across all instances
   SELECT balance FROM accounts WHERE user_id = $1 FOR UPDATE;
   

2. Temporal Entity Workflows (Actor Model): Instead of coordinating locks in SQL, designate a single Workflow per entity (e.g., workflowId: account-${userId}``). Because Temporal guarantees that only one workflow instance runs per Workflow ID at any given time, concurrent operations can be sent as Signals or Updates. Temporal serializes them in arrival order inside the workflow, completely eliminating race conditions without manual distributed locks or deadlocks.

5. Lifecycle: Health Checks & Graceful Shutdown Link to heading

Autoscaling dynamically adds and removes containers based on load. When an instance shuts down or deploys, you must avoid dropping active requests.

A. The /healthz Endpoint Link to heading

Expose a fast liveness check for Caddy to verify that the instance is ready to accept traffic:

app.get('/healthz', async (req, res) => {
  try {
    await db.query('SELECT 1'); // Ensure DB connectivity
    res.status(200).send('OK');
  } catch {
    res.status(503).send('Unavailable');
  }
});

B. Graceful Shutdown (SIGTERM) Link to heading

When an orchestrator (Docker, Kubernetes, Nomad) stops an instance, it sends a SIGTERM. Handle it by stopping incoming traffic, finishing active requests, and cleaning up connections:

const server = app.listen(3000);

process.on('SIGTERM', () => {
  console.log('SIGTERM received. Starting graceful shutdown...');
  
  // 1. Stop taking new requests from Caddy
  server.close(async () => {
    console.log('HTTP server closed. Draining database and queue pools...');
    
    // 2. Close active resources cleanly
    await db.pool.end();
    await redisClient.quit();
    
    console.log('Cleanup complete. Exiting.');
    process.exit(0);
  });

  // 3. Force exit if cleanup hangs longer than 10s
  setTimeout(() => process.exit(1), 10000).unref();
});

Summary Checklist Link to heading

Horizontal scalability doesn’t require tearing your codebase apart into dozens of microservices or over-engineering an event-driven architecture before you need it. By scaling identical replicas of your existing API, making them strictly stateless, safeguarding database connections and consistency, and letting Caddy handle traffic distribution, you get the highest-leverage stepping stone—capable of supporting millions of requests with minimal operational friction.