Let's imagine we have a web server that uses multi-threading as its concurrency approach with a fixed thread pool size of 100. Supposing that the server's request handler sleeps for 2 seconds for every 10th request, how many requests can this server handle every second?
This question persisted at the back of my mind as I was reading Deepu's series of posts on concurrency in modern programming languages. In Deepu's series he performed several web server benchmarks with the same 2 second delay for every 10th request, and the resulting requests per second figures appeared to be capped at a fixed ceiling. This post explores the factors affecting the maximum requests per second that a multi-threaded web server's can handle.