Request for stack 2024

Every few years it’s good to ask, “What is the best stack to build new software in?”. Lately I’ve been thinking about the resurgence of vertical scaling over horizontal scaling. If you app is only going to serve millions of requests per day, there is no reason not to vertically scale it. The days of 2GB ram, 1 vcpu servers are long gone. Now you can get terabytes of ram and thousands of virtual cpus. 

https://medium.com/@fengruohang/postgres-is-eating-the-database-world-157c204dcfc4
https://motherduck.com/blog/big-data-is-dead/

My prediction is that going forward AI will consume 99% of compute spend  with actual code consuming only 1% of the total compute. Those things in mind lets look at the stack. 

Application Server 

  • Rails if it has a UI
  • Java if it does not

Cache 

  • Redis

Database 

  • Postgres

Analytics Database

  • Postgres

Each ‘cell’, to use Amazon terminology, consists of one application server, a cache and a database. Just three servers. Then for reliability setup another ‘cell’ in a different availability zone. In total you have 6 servers. 

For someone who has run projects with 100s of cloud ‘servers’ 6 seems rather few. But with today’s cloud we can have 1000 vcpus across 100 ‘servers’ or across 6 ‘servers’. And the performance of the application built with only 6 will be better. 

Today’s default micro-service architecture is useful for improving deployment cadence in large teams. But it no longer provides scaling advantages compared to old school vertical strategies. If your project only includes 40~ servers with 2 cpu each, why not just use 3 servers? 80 vcpu is 80 vcpu, whether you draw 40 boxes or 3 boxes around them. 

Unless you are using 1000+ vcpu you might want to consider just using one box for your entire application.