joshua schachter's blog

When scaling from a single web server to multiple web servers, the typical practice is to put a load-balancing reverse HTTP proxy in front. This is a web server that forwards incoming HTTP requests to other internal web servers and thus distributes the load across all the different HTTP servers, allows for failover, and all sorts of good things.

However, a simple trick I learned early on is that even if you have only a single web server, a proxy in front can help out performance significantly. Through the simple expedient of buffering the communication with slow web clients, your potentially heavyweight (especially when mod_perl meant that each process was dozens or even a hundred megabytes apiece) and/or expensive Apache processes don't have to waste time serving every request for the entire length of time the client is connected. This allows you to run vastly fewer Apache processes.

In the past, I've used pound and perlbal. Pound is fast and lightweight, and allows routing based on the HTTP query; for example, everything under /img/ got routed to a high-speed thttpd instead of the Apache itself. Perlbal is much more configurable but slightly harder to get running, and the documentation was sparse.

These days, I'd also investigate nginx and varnish. Pen, a generalized TCP load-balancer with server affinity (connections will go to servers they've gone to recently in the past) is also quite interesting but will not help with the slow client problem. Finally, a second set of apache processes, configured to reverse-proxy via mod_proxy, will also do the trick. A