Tuesday night, I migrated the PointBuzz forums to the hosted forum app, and the site that served as v1 more than 20 years ago came full circle as the first customer. Both of the sites had been running on a similar version for some time, but there's something different going on here that provides a mountain of technical validation that I didn't have before.
Without nerding too hard, I really wanted to prove that all of the stuff I wrote to scale the app worked. Not worked to scale, but worked at all. If you go back in history, the forums were always written to be this self-contained thing that would work on shared hosting (hello 1999!), so breaking it up into cloud-first pieces was a lot of work. But several big things worked out:
- I moved search to ElasticSearch. In my case, I'm using the managed version from Elastic itself, running in Azure. I'm spending less than $20/month to start, and while it's not super fast at that spend, it gets real results that are useful! My home-grown solution wasn't terrible, but it wasn't great.
- All of the background stuff that happens like sending email and indexing text for search, happens in serverless instances (Azure Functions). What's awesome about this is that it costs a few bucks for millions of executions.
- It all works across multiple nodes! Since moving to the cloud, I always wanted to have more than one web head just for redundancy. There have only been a few times where I'm sure the node failed and a new one was spun up, but having two is nice. I have to keep the affinity on (sticky sessions) though in order to make SignalR work, but this doesn't seem to really impact performance.
- The two-level caching works. I use StackOverflow's pattern here, sort of, replicating and invalidating the cache in local memory by way of the message bus in Redis. I don't have a great way to measure the hit/miss ratio in local memory, but I imagine it's pretty great. User data is what gets cached the most. I'm estimating that with PointBuzz alone, I'm avoiding the database about 50,000 times per day. That gets really important with more customers. (Sidebar: This failed a lot when I was prototyping a few years ago, as the old version of the StackExchange Redis client was prone to failure.)
- Average response time went from about 30ms running on a Windows app service, to 5ms running on a Linux app service with about the same "hardware" and the same database. That's nuts! It's the same code! Maybe there's something I'm missing, but the only other thing that I can think of is that it's now running multi-tenant, which adds overhead, not reduces it.
I'm also trying out Azure Insights for monitoring. It's pretty cool, but the cost seems a little steep so far, I think averaging a buck a day for relatively low traffic. It also doesn't monitor the Redis service. Like any service, you can feed it custom events, so I might be doing that to instrument the caching both local and on Redis.
So now I just need customers. Selling is not my strong point, but I have some ideas to get the name out there and perhaps line up some "free" customers as promotion. I have a reasonably baked product, and that's very exciting.