[00:00:01] Hello, this is Mark. And welcome back to the Code and Bootstrapping Podcast. Today, I'm going to talk about why I don't think most people starting new tech companies should worry that much about scalability.
[00:00:15] Usually when starting a new project and planning out the technical side of it, people think about a few things. They think about the speed of development, the ease of hiring, the performance and scalability. These things might sound roughly equivalent, but they don't matter equally. In fact, I would say each one matters about 10 times more than the next item following it. That's right. I think the speed of development matters far more than everything else in the list combined.
[00:00:44] You're not going to be in a position to hire anyone until you've either got revenue or funding, both of which generally require you to successfully develop something and found at least a hint of product market fit. Beyond the very low bar of being usable and not frustrating people, the performance of your technology choices matters even less. In 2019, competing hardware is abundant and cheap.
[00:01:09] Even games are often made in high level languages like C Sharp and for that matter, Microsoft has even done research on operating system development in high level languages. So it's probably going to be more a matter of you not using a horrible algorithm or design as opposed to you having chosen a language or a database or anything like that, that's too slow.
[00:01:35] Poorly designed algorithms will definitely overcome the fastest of programming languages and hardware and so forth, though. Just one place in your program with factorial of N performance will basically overwhelm anything once N is 15 or 20 or so. So, I'd worry about that, but I wouldn't worry about the difference between, say, using NGINX versus Apache or C versus Python, even.
[00:01:57] Now, the next point, scalability, is the one that I really want to talk about. Scalability doesn't matter at all, until you're already very successful. And yet, for some reason when I go to various places online with lots of people writing about startups like, say, Indie Hackers or Hacker News or Medium, seems that scalability is the one that everyone is writing about and worrying about the most.
[00:02:23] But in a lot of ways, scalability problems are the first world problems of first world problems. They only happen when everything is going really well, and you've already done something that almost all companies fail to, which is make something that people want—that so many people want—that they're overwhelming your capacity to supply it to them. It is still a problem, but it's generally the kind of problem that when you encounter, you'll have plenty of resources at your disposal to deal with it.
[00:02:56] And in fact, I've seen this happen at more than one company where I interviewed when they were really early in and really thought, do I want to work there or not? And then kept in touch with people. Or places where my friends were working.
[00:03:11] It's not uncommon for a company to start off with a horrible tech stack or no tech stack. Groupon, where I previously worked, long before I started there, they they began with just an email list. And then later they were a WordPress site, which became completely untenable. And when I was working there, it was driven by Rails. But they also had a lot of other technologies in other places. And they were and they've gone through a lot of work to deal with performance problems. And when I was there, they probably had more problems from having acquired a lot of clones that were operating in different countries and had different tech stacks and they just really needed to get everything unified.
[00:03:53] Another company that famously had a lot of scaling issues was Twitter. They didn't have the best architecture in the world. The fact they were using Ruby may have been a minor issue as well, but really it's just their architecture was not built for the behemoth that they became. And again, this was mostly an issue of just being ridiculously popular and one of the most highly trafficked sites on the Internet. And when they had this "problem", they also had lots of money and lots of really smart people that wanted to work for them and fix the problems which they did. And like Groupon, it's also a multi-billion dollar company now.
[00:04:20] Other companies that started off with really simple tech stacks... Uber, I believe, started off as just like a one page PHP app. And then later, they moved to Node back when Node was really young and untested and it was seen as a really risky choice. Stripe also started off as a very, very simple app on the technical side. Actually, at the very beginning it was just manual. People were doing all the transactions behind the scenes on their own. So I think it's a really, really common thing and successful startups to have a totally unscalable architecture at the beginning, but get started quickly and then over time gradually move more and more towards something that is scalable and also reliable, performant, all of these things.
[00:05:26] Worrying at the very outset about being able to scale to handle the level of traffic that say Stripe or Twitter have now really strikes me as similar to what one of my friends in college used to worry about. She was on a big health kick and she was doing a lot of cardio and she decided she wanted to start weightlifting a bit as well, but she was worried that she would accidentally get the physique of a male bodybuilder. And I totally understand that what a lot of male bodybuilders look like would have looked horrible on her. But the thing is, like, if you look at like Arnold Schwarzenegger in the 70s, or even someone smaller like Frank Zane, they were not only taking steroids, but they were also working out more than two hours a day on top of that. And there were many, many other men who were also taking illegal supplements and working out hard that they beat. So it's not like some random college student will accidentally get this gigantic physique from working out at the gym once or twice a week. And that's about how ridiculous I think it is to worry that your new startup will not only be very successful and popular, but it will be so successful and popular that you'll grow to Twitter or even Facebook scale on the first iteration of your product.
[00:06:55] Now, obviously, if everything else were equal and you just had a choice of choosing stack one which was scalable, or stack to which was completely identical to stack one, except that it was less scalable, then you should always go with a scalable option. Of course, it's not like that in the real world. If there were no trade-off, the clearly inferior option would just disappear. So pretty much all of the choices, the popular ones and even the somewhat niche ones do have some advantages and some reason why people are choosing them. So you need to think about what am I giving up to get scalability. If you're giving up speed of development, it's probably a really, really bad choice.
[00:07:40] With that said, I'd like to go over a few concerns people have brought up regarding scalability and tech choices on places like Indie Hackers or Hacker News. First one is, we need to use AWS or GCP in order to be sure we can scale. It is true that both offer very flexible solutions. It will scale to just about any level of traffic thrown at them. But that doesn't mean that scaling isn't an issue at all, because for a lot of people, in fact, almost everyone I've spoken with who's used AWS, predicting how much their costs will be is a serious problem.
[00:08:18] And the same is true on Google's platform or Microsoft's. There a pretty high profile article a while back called "Firebase Costs Increased by 7000%". And it was written by someone running a small startup who is shocked their bill had gone from $25 a month to $1750 a month, due to a few changes in the pricing structure. And there are many other people who simply got far more traffic than they had ever planned on or made some sort of mistake that ended up consuming a lot more resources than they expected.
[00:08:54] And what's going to happen is, if you're on one of these flexible cloud providers, you'll get a lot of assurance that your site won't be shut down or go down because of consuming too many resources. But you won't easily be able to know how much you'll be spending on this service. So if you have a lot of money, but you suspect you'll have very spiky or unpredictable traffic because you're doing a PR initiative or you have an extremely viral product, then maybe it makes more sense to be on one of those platforms, if you can do it without sacrificing much speed of development.
[00:09:31] Another thing I've seen on IH, and actually heard in person a number of times, is you should break your product into micro-services and dockerize everything. I think this is pretty bad advice, actually. Maybe if you're really familiar with Docker and it's just everything in your workflow uses it and you're slower if you don't use it, maybe that makes sense. But, breaking a product in a micro-services is even pretty irrelevant for scaling the number of users your site can handle. That's more about scaling the size of the team that you can have working it. And if you're just one person, it's insane to add complexity and slow yourself down so that very large teams could work under a code base. By the time you have a very large team. You will almost definitely have rewritten everything more than once.
[00:10:25] And believe it or not. I've seen even crazier advice that the one that really sticks out in my mind. I saw an indiehacker where someone said I avoid relational databases because of their scaling issues. I'm not sure if that commenter was talking about how with a no sequel database you can be assured that you'll be able to shard everything easily because you'll put some documents on some shards and others and others. But, if you're worried that a relational database isn't enough for you, that basically means you expect to be bigger than Facebook was when a billion people were using it, which is the most extreme level of arrogance about your product success and your scalability imaginable. And guess what? When it's that big, you will be able to hire many, many people to help you fix it.
[00:11:21] Probably, the most reasonable concern that I've seen on these kinds of forums "is what if we hit number one on Hacker News?" And it's probably no mistake that a lot of people on Hacker News are really worried about hitting number one on Hacker News because they see those few companies who did or even those few bloggers who did and then couldn't handle the traffic. Well, okay. So, let's look at about how much traffic you expect to be getting from that. Of course, this is always a moving target because Hacker News is growing. But, based on the best information I have in early 2019, it's about one to maybe six or seven hundred concurrent users on your site, if you're at number one. And what does that mean exactly? It's not the same thing as that many people hammering your site every second, unless just the very nature of your app is such that every users generating a round trip to the server every single second they're on the site. More commonly, it's going to be a few times a minute.
[00:12:30] But in any case, looking at a low performance back end server like Rails, or actually Python like Django is probably the slowest thing you could possibly be running on the backend, that's popular. You can still get about 50 requests a second or you can still handle about 50 requests a second on the cheapest Digital Ocean droplet. So we're saying $5 a month and you can handle 50 requests a second, which means you can handle more than the number of people that Hacker News will send at you. If you're number one on the site and this is with running NGINX in front of Python, and if we're running Elixir with Phoenix framework, then you can handle more like one to two thousand requests per second. Even being number one on Hacker News, pinned number one on Hacker News, isn't really going to be an issue for you on $5 a month of hardware.
[00:13:29] So, I don't think this is a big thing to worry about for most apps. If you've got something highly interactive like some kind of in browser game, that might be different. And it's also changed over time, because the amount of computing power you get for any given amount of money per month has been increasing year after year.
[00:13:52] The worst case scenario that's likely would be if you're running WordPress and you have a lot of plugins, maybe you're not very technical, but you've just cobbled together all the functionality you need through 30 plugins. In that case, your a $5 a month droplet will possibly not even be able to run it at all because of around a memory and maybe even a $10 a month droplet will fall over.
[00:14:19] In that case, you've still got a couple of reasonable options. One would be just pay $40 a month, Get a larger server that you're sure can withstand all the traffic that would come, even from being number one on a popular site like Hacker News. And $40 a month is actually less than a lot of people spend on their initial setup with Amazon or Heroku or whatever other provider.
[00:14:43] The other option would be you just pay $10 a month. You set up some kind of caching system so that when you hit the front page, everyone's served a static site, and the worst of it will only last an hour or maybe even just a few minutes. And then you get the benefits of your publicity and you're not overpaying all the rest of the time on the hopes that maybe you'll hit number one on an aggregator. And, if your traffic starts going up in terms of its overall trend and your average daily traffic is getting higher and higher and close to too much for the $10 a month droplet, then you just go to 20 or 40 and you should actually have enough revenue even if you're just running Adsense ads or Amazon affiliate links, then you can support your site because that's actually a lot of traffic, once you're in need of more than a $10 a month server.
[00:15:35] There's obviously an optimal point somewhere in the spectrum, and that depends a lot on both how much money you have and what your skills are. For some people, the difference between $10 a month and $40 a month on server costs is negligible because maybe they're working a full time job in San Francisco and they they just don't care about such small amounts of money. For other people, that's really big difference and it's worth it to optimize the last dollar.
[00:16:03] Not spending too much time setting up infrastructure is important for pretty much everyone, though. And the difference here is, some people may not be able to use something like Digital Ocean or just a bare Amazon EC2. They may not be comfortable installing things on their server or setting up a database, even.
[00:16:26] So, depending on who you are, maybe you should be using a lot of third party services and integrating them together. Or, maybe you should be running a WordPress site with a whole lot of plugins, if that's ecosystem you know. For someone like me, it's probably fastest just to use a VPS and all the common Unix tools that I'm used to. But no matter who you are, your odds of suddenly scaling to the point where a VPS can't handle your traffic, or you can't use a relational database, is about the same as your odds of going to the gym once or twice a week and suddenly looking like a champion bodybuilder.
[00:17:05] Thank you for listening to code and bootstrapping.