5 min read

How to Scale a Ruby on Rails App to Thousands of Concurrent Users

A practical, battle-tested checklist for scaling Ruby on Rails from a few hundred to thousands of concurrent users — covering database tuning, layered caching, background jobs, connection pooling, and the metrics that actually matter.

Ruby on RailsPerformancePostgreSQLScaling

Ruby on Rails gets an unfair reputation for not scaling. In my experience the framework is rarely the bottleneck. The bottleneck is almost always the same short list of things: an unindexed query running on every page load, background work that is not actually in the background, a connection pool that is smaller than the number of Puma threads, or a request that quietly makes forty HTTP calls.

I have scaled Rails products for a high-visibility EdTech launch, a payroll system handling sensitive financial data, and a merchant platform processing payments. The pattern was consistent: fix the data layer first, then the request cycle, then the infrastructure. In that order, because optimising the wrong layer is how teams spend a quarter and move nothing.

1. Start with the database, not the web tier

Before you add a second web server, run EXPLAIN ANALYZE on the queries your slowest endpoints call. Nine times out of ten you will find a sequential scan over a table that grew from thousands of rows to millions of rows.

  • Every foreign key needs an index. Rails creates them for references with index: true, but hand-written migrations often forget the composite case.
  • Composite indexes should match the column order of your WHERE clause, most selective first, then the range column.
  • A covering index can turn a query that reads ten thousand rows into one that reads zero table pages.
  • Never index a low-cardinality boolean on its own — the planner will ignore it. Index it as part of a composite.
CREATE INDEX CONCURRENTLY idx_orders_merchant_created
  ON orders (merchant_id, created_at DESC)
  INCLUDE (status, total_cents);

Use CREATE INDEX CONCURRENTLY in production so you do not lock writes while the index builds. This one habit has prevented more incidents for me than any other database technique.

Kill the N+1 before it becomes an incident

Enable strict loading in CI (config.active_record.strict_loading_by_default = true) so an accidental N+1 fails the build instead of silently issuing hundreds of queries under load.

2. Add caching in layers, from cheapest to most expensive

Caching is not a single switch. There is a hierarchy, and you should climb it from the bottom, because each layer costs you freshness and complexity.

  1. HTTP caching and ETags — the cheapest win. If the browser and CDN can revalidate a response, the request never reaches Rails.
  2. Fragment caching with Russian-doll keys — cache rendered partials keyed on updated_at so a single record change invalidates exactly what it should.
  3. Low-level caching for computed values — Rails.cache.fetch around expensive aggregates, with a sensible expiry.
  4. Full-page or CDN-level caching for public, non-personalised pages.
Rails.cache.fetch(["merchant", merchant.id, "stats"], expires_in: 5.minutes) do
  merchant.orders.completed.sum(:total_cents)
end

A Redis-backed cache is almost always worth it. The mistake teams make is reaching for a cache before they have fixed the query underneath it — caching a slow query just makes the slow query happen less often, and hides the real problem until a cache miss storm takes you down.

3. Move work out of the request cycle

If a user has to wait for an email, a PDF, a webhook, a payment reconciliation, or an AI call, your request is doing too much. Accept the minimal payload, enqueue a job, respond immediately, and let Sidekiq handle the rest. This is the single biggest lever on perceived speed.

class PaymentReconciliationJob < ApplicationJob
  queue_as :payments
  sidekiq_options retry: 5

  def perform(order_id)
    order = Order.find(order_id)
    Orders::Reconcile.call(order)
  end
end
  • Separate queues for fast user-facing jobs and slow batch jobs so a nightly report never blocks a password reset email.
  • Make every job idempotent. Jobs retry, and a non-idempotent job double-charges customers.
  • Bound your queues. Unbounded queues hide backpressure until memory runs out.
  • Set explicit rate limits for third-party APIs like Stripe, Twilio, or an LLM provider, and let the job retry with exponential backoff.

4. Right-size the connection pool

This is the most common configuration bug I find in scaled Rails apps. The database connection pool must be at least as large as the number of Puma threads per worker multiplied by the number of workers. If it is not, threads silently block waiting for a connection and your p99 latency climbs for no obvious reason.

# config/database.yml
production:
  pool: <%= ENV.fetch("RAILS_MAX_THREADS", 5).to_i * ENV.fetch("WEB_CONCURRENCY", 2).to_i %>

Watch the Postgres side too. Each connection costs memory. Past a few hundred connections, a pooler such as PgBouncer becomes essential, and long-running analytical queries should move to a read replica so they never compete with user traffic.

5. Serve assets and media from a CDN

Your app servers should never serve a static byte. Fingerprinted Rails assets belong on a CDN with immutable, one-year cache headers. User-uploaded images should be stored on object storage and transformed at the edge rather than streamed through Puma. Moving images off the app tier routinely cuts server CPU by a third.

6. Measure the things that actually predict failure

Average response time is a vanity metric. Under load, the average stays flat right up until the moment the system falls over. Track the tail.

  • p95 and p99 request latency per endpoint, not the mean.
  • Database query time as a share of total request time.
  • Sidekiq queue latency and the age of the oldest job per queue.
  • Garbage collection time and Puma thread pool utilisation.
  • Error rate by class, so a spiking NoMethodError is visible before users report it.

A practical checklist

  1. Enable strict loading and fix N+1s in CI.
  2. Find the top ten slowest queries and add the right composite indexes.
  3. Add Redis for caching and for Sidekiq.
  4. Move every slow side effect into an idempotent background job.
  5. Set the connection pool from thread count, not by guessing.
  6. Serve all static assets and media from a CDN.
  7. Alert on p95 and p99 latency, queue depth, and error rate.

Scaling is not about finding the one clever trick. It is about removing the unglamorous, compounding inefficiencies before they get a chance to multiply.

Do those things and a well-written Rails monolith will comfortably serve thousands of concurrent users on a handful of servers. I have seen it happen repeatedly, and I have never once needed to rewrite a Rails app in another language to get there.

More writing

All writing