Your product is an asset. Your tech stack is a liability.
#work 4 min.
In software engineering, it’s generally accepted that every step up in complexity has major trade-offs. But what exactly do you lose by adding complexity?
Let’s consider web applications as an example since that’s what I know best.
Static website (like this one)
Everything that can be a static website should be a static website. Expensive things, like scalability and security, essentially come for free. There’s almost no infrastructure to manage, no state to monitor.
Unfortunately, you still have to be a bit technical to consider static site generators “easy”. I’m hoping the next generation of user-centric static site generators and CMSes will make it the default choice.
Stateless application
Imagine something simple, like a screenshot microservice for internal use. You send a request with an URL and get an image as a response.
As soon as you introduce any back-end processing, infrastructure and scaling get much more expensive.
Deployments become more involved. You need monitoring tools to be sure your application is actually serving users.
Security becomes a concern as soon as you start accepting user inputs.
Stateful application (using filesystem)
Let’s keep the same example as before, but add some file-based caching and a SQLite database for user management. Still nothing too fancy.
Adding filesystem will once again increase infrastructure costs. You now have to monitor disk usage and health, or pay someone else to do it.
As soon as you introduce state, backups become a thing. The cache can be regenerated, but that SQLite database better be backed up. At the very least you want to have daily snapshots.
If you take your service seriously, you will make sure to have at least one backup in a different data center. Two is one, and one is none.
Stateful application (using database server)
Let’s level up to a separate database server, say Postgres, Redis, or MongoDB.
In my experience, 90% of all performance and scaling issues can be traced to a database.
You can get quite good at managing the database yourself, but it takes time and mistakes. You can also pay someone else to do it. Databases are no trivial thing, so you will be paying for them either with time on with money.
App with long-running processes
Let’s take it up a notch again and introduce queues!
Additional infrastructure and monitoring costs are expected, but the biggest shift is in mental overhead.
Synchronous processing is a privilege you don’t appreciate until you lose it. It’s a wonderful thing to be 100% sure your code will always run as A→B→C, because that’s the only possible way. Split A, B, and C across different processes, and this inherent constraint no longer applies. Some languages do asynchronous processing better than others, but it inevitably complicates things.
You will find yourself writing a lot more code to handle errors, race conditions, and edge cases.
Distributed application
There comes a day when your web application can no longer run effectively on a single machine and you have to reach for horizontal scaling.
If you’re lucky, you might get away with a few web servers behind a load balancer and some read replicas for the database. Apart from additional costs, this wouldn’t be too bad to manage ever for a single person. There are off-the-shelf solutions for these types of problems.
If you’re unlucky and you need to scale write-heavy traffic, you’re entering a world of pain and I only have sympathy for you.
Unless you just need to store a shitload of documents with no relationships at all (e.g. logs), all solutions come with huge caveats. You will need to spend a lot of time analyzing your specific use case just to find the best tool, and then learn all of its idiosyncrasies.
Wrapping up
In my professional career, much more time has been wasted over-engineering than under-engineering software.
My general recommendation? If you intend to make money with your software, choose the least amount of boring technology that will do the job. If you already did, stay as boring as possible, for as long as possible.
Your product is an asset. Your tech stack is a liability. Treat each accordingly.