People have given up on performance in favor of Scalability

Scalability has been all the rage since the cloud made horizontal scaling easy. No longer do we have to order parts, lease colo space or rack servers. Instead there is an infinite supply of Virtual Machines out there we can rent at the press of a button. Because of this there is a tendency to start development with an architecture that will scale well horizontally. My entire career has been during the post AWS period. Pre-mature optimization is the root of all evil, but make sure to create a stateless service so we can scale it up later when its slow.

Web Servers

Its interesting to look at examples of projects that did not focus on scaling horizontally.

For example we have stackexchange’s public numbers on their performance. 

https://stackexchange.com/performance

They claim that they handle up to 450 requests/s on 9 servers. From the infographic it looks like these are 1U or 2U servers with 64GB of RAM and although its unspecified I’m guessing they have 12-24 physical cores per machine. 

These machines have around 10 times as much RAM as the VMs my team runs in production and probably over 10x the cpu performance. They handle more traffic per server with lower CPU utilization. A rough estimate from these numbers is that the stackexchange .NET service is 2.5x to 10x as performant as my Java service. That could just be the bare metal vs Virtual Machine cost since our stack has significantly less CPU. 

You might think that stackexchange is operating at an absurdly low CPU utilization at 5%, but I haven’t seen anyone operating cloud servers above 20% utilization with a sample size of 4 companies. 

Big Data

This study was done comparing single threaded performance on a modern CPU vs distributed big data algorithms. 

Single thread outperformed distributed big data computations on many (most?) problems. 

https://www.usenix.org/system/files/conference/hotos15/hotos15-paper-mcsherry.pdf

They found that optimized single threaded code outperformed distributed code in the datasets they tested. Admittedly, not all datasets will fit on a single machine. But we have to remember a single machine can now have over a TB of RAM and 100s or more TB of SSD. Single threaded performance is clocking over 5GHZ now. A single server can handle all your big data needs until your dataset exceeds dozens of Terabytes. 

I’m working on learning awk to experiment in this area. It is a relatively simple domain specific language for text processing and formatting. 

If you like my writing, please buy my book on Amazon.
The Sledgeworx Guide to Getting into Software