How to scale a web app: a step-by-step guide to system design basics

Most apps never need “web scale”. But almost every successful one hits the same few walls in the same order, and each wall has a well-known fix. This guide walks through them as a story: one app, growing from its first users to a lot of them, adding one piece at a time and only when it’s needed.

Stage 0: one server#

Everything on one machine: the web app and the database.

Users ──► [ App + Database ]

This is the right way to start. It’s cheap, easy to understand, and easy to change. A single modern server handles far more traffic than most people expect; plenty of profitable businesses never leave this stage.

The first wall: the app and the database compete for the same memory and CPU, and one bad deploy or full disk takes everything down.

Stage 1: separate the database#

Users ──► [ App server ] ──► [ Database ]

Move the database to its own server, ideally a managed service (such as Amazon RDS, Google Cloud SQL, Neon or Supabase) that handles backups, updates and failover for you. Now each can be sized for its job, and the database has a proper backup plan.

Stage 2: make the server bigger (vertical scaling)#

When the app server gets busy, the simplest move is a bigger machine: more CPU, more memory. This is vertical scaling. No code changes, often a few clicks.

Before paying for more hardware, check that the slowness isn’t a fixable problem: a missing database index or an N+1 query (a loop that runs one query per item) can make a 10-second page take 50 milliseconds, for free.

The wall: machines only get so big, the biggest are expensive, and one server is still one point of failure.

Stage 3: more servers behind a load balancer (horizontal scaling)#

                    ┌──► [ App server 1 ] ──┐
Users ──► [ Load ] ─┼──► [ App server 2 ] ──┼──► [ Database ]
          [balancer]└──► [ App server 3 ] ──┘

A load balancer spreads requests across several identical app servers and stops sending traffic to any that fail a health check. Need more capacity? Add servers. One crashes? The others carry on. This is horizontal scaling, and most hosting platforms do it for you automatically.

It has one requirement: the app servers must be stateless. Any request can land on any server, so nothing can live only in one server’s memory or disk:

• Sessions go in the database or Redis, not in memory. See authentication explained.
• Uploaded files go to object storage such as Amazon S3 or Cloudflare R2, not the server’s disk.
• Scheduled tasks run in one place, not on every server at once.

Stage 4: cache what you can#

Now the database is usually the bottleneck, because every app server talks to it. The cheapest relief is to not ask it as often:

Users ──► [ CDN ] ──► [ Load balancer ] ──► [ App servers ] ──► [ Cache ] ──► [ Database ]

• A CDN serves images, scripts and even whole public pages from servers near each visitor, so those requests never reach you.
• An application cache such as Redis keeps the results of common queries, so the database answers each one once a minute instead of on every request.

Caching explained covers how to do this without showing stale data.

Stage 5: move slow work to the background#

Requests that send emails, generate PDFs or call other companies’ APIs hold a server busy while they wait. Move that work to a queue and background workers: the request saves a job and answers immediately, and workers process the jobs at their own pace. Bursts of traffic become a longer queue rather than an outage, and you can scale workers separately from web servers.

Stage 6: scale the database#

When the database itself is busy even after indexing and caching:

• Read replicas. Copies of the database that follow the main one and answer read queries. Most apps read far more than they write, so sending reports, searches and listings to replicas takes much of the load off the main database. The catch: replicas lag behind by a moment, so read “what I just saved” from the main database.
• Connection pooling. Each app server opening many connections can exhaust the database; a pooler such as PgBouncer shares a smaller number between them.
• Partitioning large tables, for example by month, keeps indexes small and old data easy to archive.
• Sharding, splitting data across several databases (for example by customer), is the last resort. It’s powerful and complicated: queries across shards, rebalancing and transactions all get harder. Most apps never need it.

Stage 7: split the system, if you must#

At some point, separate teams want to deploy separately, or one part of the system has very different needs (say, video processing). That’s when splitting into separate services starts to make sense. It’s an organisational decision as much as a technical one; monolith vs microservices covers when it’s worth it.

What to measure at every stage#

You can’t fix a bottleneck you can’t see. From the start, track:

• Response times, especially the slowest 1–5% (the p95 and p99), not just the average.
• Error rates and which endpoints produce them.
• Database: slow queries, connections, CPU.
• Queues: how many jobs are waiting, and how long the oldest has waited.

Then scale the part that’s actually struggling, not the part that’s easiest to change.

The short version#

If your site or app is starting to struggle with growth, I can find the bottleneck and fix it as part of my custom API integration and Wix apps work.