Archive: February, 2024

When I look around at what's going on in the world, I hesitate to even go there, but our lived reality is what it is. The world has been throwing us an undue amount of shit the last few weeks. It's not life altering, wreck us kind of shit, but it does feel like one difficult situation after another has been piling up on us. "Well those people have it worse" isn't really the consolation and perspective that I wish it was.

What I hate is that I've probably started a half-dozen blog posts to complain about one situation or another, and then I delete them. Admittedly, in face-to-face real life, it's hard to be around people who mostly complain about whatever, but I don't think culturally we're willing to listen to people who are having a hard time. I try to be better about that for others, and I'm trying to make it OK to vent.

I just want the three of us to have less friction, if only for awhile. Our give/take ratio is way out of whack.

Diana went to Epcot on Monday to spend some time with her cousin. She was on her way out of thee parking lot, when a guy in a Disney maintenance vehicle clipped her in the rear quarter panel. She had the right of way, and there's no telling what the hell the guy was doing, but it clearly was not paying attention. We've got the video footage (below) to prove it.

The damage ended up being about $17k, and with whatever math Florida uses, that makes it a total loss. We had it almost six years and 62k miles. Battery range was still around 290, from 310 originally (you know, for the haters that still think you need to replace your battery). We just had it repaired after the great crowbar incident of 2023. I figured we could easily get a few more years out of it, because outside of cosmetic things, the car was tip-top. And of course, this happens just a few months before we paid it off. So now we've gotta buy another car, which I hoped not to do for quite some time.

And this is our second totaled car in about three years. Don't forget the teenager who blindsided Diana after work one night, killing the leased Nissan Leaf we had.

This isn't going to break us, but it also feels like we can't catch a break.

Monday we once again joined our friends from Upstate New York to do a VIP tour at Walt Disney World. We did this with them about two years ago, concentrating just on Magic Kingdom then. This time it was a combination of Hollywood Studios and Epcot. That means we got to do all of our favorites without waiting, in the company of people we really enjoy. It's totally worth it.

There have been some recent articles with economists talking about the "experience economy," a term that has recently been co-opted by people post-pandemic who want to describe the act of buying stuff that you do instead of stuff that you have. You know, experiences, not stuff. I've talked about that a ton over the last decade or so, and now I believe more than ever that it's a meaningful way to approach life.

Given my middle age, it's clear that time moves very quickly when it becomes routine. That's not a great feeling when you're on the back nine. (Not sure why I used that cliche. I hate golf.) So the value of doing things and going places is hard to put a price on. I have three travel itineraries between now and June, and I'm trying to figure out what to line up after that (it's hard with a kid in school). Some people are critical of others who would spend ridiculous money on a VIP tour like I did, at a theme park that I live next to, no less, but I still treasure the memory of the last time two years ago. Why wouldn't I want to try and replicate that? I'm lucky enough that I have friends like this who show up every year or so and want to do things with us. It's hard to put a price on that. If I can afford to do the thing, I'm going to do that.

I bought so much crap in my 20's, and I thought at the time that it made me happy. And certainly it did, for a little while. In-person retail shopping, you know, where you went to a store and walked around and then bought stuff, was what people did before they micro-dosed dopamine by getting likes on their phone. I couldn't tell you what most of that stuff was. And the worst part is that I bought most of it on credit. If I could tell 28-year-old me that I was being stupid, I would, because the things that I do remember are the road trips, the after-work meet-ups, the little gatherings at my apartment, that sort of thing.

If I'm going to burn through cash for non-essentials, it's going to be to travel and create the experiences. That's what I find satisfying, whatever the cost.

You know how people spout that annoying thing about living in the moment? It's in that category of silly motivational poster nonsense, but generally speaking, there is some truth to the idea. Ferris Bueller wisely said, "Life moves pretty fast. If you don't stop and look around once in a while, you could miss it."

I've come to realize, however, that while we struggle to latch on to the good moments right in front of us, it sure seems easy to wallow in the shitty moments. I've found myself buried in those recently. Clinging to those is antithetical to the idea that we have a finite number of moments.

So yeah, stop doing that.

I wrote yesterday about my memory pressure on the web apps... turns out there's a magic cure.

First off, I'm not an expert in memory management, which is why I enjoy writing code for a managed framework. It abstracts all of that away for you. But also, the thing about stateless web apps is that all of the objects they create are ephemeral. It hears the request, puts some bits together, sends them back, and forgets it ever happened. This is one of the reasons it was so frustrating to see CoasterBuzz pushing 500 MB in memory. If all this stuff is ephemeral, why is it hanging on to all of this memory? I've seen some traffic increases in the last year, but nothing that dramatic.

As it turns out, I learned that there are two different modes of garbage collection, server and workstation, and oddly enough, the server variety does background GC less frequently to devote all the resources to servicing requests. That's neat and all, when you're squeezing a bunch of containers into 3.5 gigs of a virtual Linux box, it gets tight. It can't be sloppy.

Today I learned that .Net 8 shipped with something called "dynamic adaptations to application sizes." I guess it's a better way to decide how many managed heaps to have, and how often to run GC. It just requires adding a couple of environment variables, so I did that. While CoasterBuzz was bumping up against 500 MB or higher before, now it's leveling off at 300.

The only reason I can afford any of this redundancy and elastic scalability is because I moved everything into Linux app services on Azure a few years ago. The cost is one-third of what it was for Windows. However, that does come at a price, because while Windows treats each site as its own process, Linux wraps everything in containers, and all of the overhead that includes. So one Windows app service node with 1 vCPU/1.75gb RAM costs $75, but on Linux two nodes of 2 vCPU/3.5gb RAM is $50.

The hosted forum product is running on two nodes of vCPU/1.75gb RAM for $25, and it consistently uses less than 200 MB. Page rendering times are generally sub-50ms. Considering they're under 10ms on my local computer, I imagine perf could be great if I was spending more than $25!

Anyway, magic fix. The erratic part below is when I was deploying and scaling, but after the tweak, very consistent memory usage.

My mid-year lipid panel was, as I expected, not great for triglycerides. Cholesterol is still great (statins are a medical miracle), but something about my body doesn't connect the two as others might. While I've never had the crazy high levels, like 500 mg/dL, normal is under 150, and my best score over the years has been 190. This time I came in at 294, after a somewhat more palatable 222 in July. This is not a medical mystery... I've fallen off of the exercise train, I'm eating my feelings, and I'm addicted to carbs. I gotta have my tots.

You move that number by way of diet and exercise, so let me take one at a time. My diet is, oddly enough, not full of as many terrible things as you might expect. I don't have soda at home, as I stopped buying it years ago. We don't usually have a ton of chips or snacks lying around, except when we have people over. I haven't had red meat in over 15 years. Mostly I don't eat after 7. I generally try to pay attention to sodium intake. But I do eat a lot of rice, and I'll have chips and a Coke when I'm out. Alcohol can spike the tri's as well, but I doubt that a couple of drinks on the weekend are the problem relative to running through a 2-pound bag of tots every two weeks. I love carbs.

But it's also likely what I'm not eating, like broccoli and brown rice. (Sidebar: Is Chipotle the only fast casual chain that still has brown rice? Pei Wei and Bento both dropped it.) I know I can get down to five or six tots at lunch, and I just need to start making batches of brown rice to have around, as it reheats much better than white.

Exercise is the other half of the equation, and I actually got into a pretty good rhythm pre-Europe last summer. I have a treadmill in my office, and I'd just start the first hour of my day on it, and shower after that. It's easy enough to bang out two plus miles every morning that way, and I know I feel better when I do it. But this hasn't just been a matter of "just do it."

I've been struggling with anxiety since the pandemic started. Job situations made it worse. These days it's more because of parenting. Some of it may just be chemistry, and for the bits that aren't, I'm working with my therapist to figure that out. Anxiety isn't like stress, where you feel pressure in the moment for certain things to happen. Anxiety is more about the future and what might happen. My brain won't shut off, and it's like a multi-threaded stream of nonstop, ADHD-fueled thoughts. This in turn has led to pretty serious insomnia, where I can't fall asleep after hours lying in bed, and then I get up every hour or two. Needless to say, that makes me tired, most of the time, and I can't bring myself to do a lot of things beyond work. Exercise is the first thing to go. I'm not exaggerating when I say that it has become a quality of life issue.

So I need to treat the anxiety. As I said, therapy is part of that, including realizing that so many of the things I worry about are not in my control. I'm trying to reduce CPU time on those things. I feel like the biggest thing to get back on track is consistent sleep, and that requires turning off the brain. I know from very infrequent use of lorazepam that it helps a ton. It's like getting off the freeway on to an empty road with no other cars and no landscape to look at. But that stuff is highly addictive, and it's there mostly just for the increasingly rare panic attack. I also know that medical marijuana works, as little as 2.5mg of THC. So since it's legal in Florida, that's what we're gonna try to get the sleep normalized.

And to get political for a moment, reclassifying weed at the federal level is long overdue. It's pretty clear that there are some benefits when used in certain ways, but the classification prevents quality research from happening. We don't ban alcohol, and it has absolutely zero medicinal value, at all. It makes no sense.

When I turned the forums into a product a few years ago (that I've never marketed, so I have as many customers as you'd expect), I built in a lot of redundancy and performance features, so you could host many forums under different names, but under the covers it would be just one app. In other words, the PointBuzz forums and the testing forums are the same app, they just look different. On top of that, there are two copies running, and the traffic is split between the two. It runs in just 200MB and can still easily handle a thousand requests per minute (that works out to 8 per second on each instance, so there's obviously room to grow). And that's on one virtual CPU and 1.75 gigs of RAM. I could scale it up (more power) or out (more instances), for a very long runway. Even with the two nodes, it rarely replaces one or "goes bad."

CoasterBuzz and the not-forum part of PointBuzz (and this blog and other stuff) run on a separate virtual machine, and it has 2 virtual CPU's and 3.5 gigs, but it was just one node. The whole point of having more than one is if it crashes or something else bad happens, the other one keeps up until a new one can be provisioned automatically. It has been crashy for a long time, and uses a lot more memory. Usually this means it stops responding for a minute or two, but comes right back. I started to notice it more, so I wanted to bump it to two or more nodes. Doing this requires some plumbing I didn't have in place, so I added it, and went to two nodes. And that's when it got weird.

The first thing that I noticed is that both instances would get traffic for a bit, then it would only go to one. No point in paying for two if only one is working. If I scaled it up or down, it would go back to two for awhile, then drop down back to one. My first hint came when the portal said that both instances were unhealthy, which I didn't understand, because I could hit both (you can hack your ARR cookie to go to either one).

My first suspicion was that CoasterBuzz had a memory leak. Sure, it does more things outside of the forum, like serve photos and do news and stuff, but it's not substantially more. I started auditing the code, and I did find some database connections that weren't in a C# using block, but I did close them and they should have been disposed at the end of the request. But whatever, I fixed them up. Overnight, same thing, it dropped down to one.

I was poking around the instrumentation and found that there were hundreds of requests that were redirects, and it was a flat line. That meant something mechanical was doing it. Then, as I was looking around at the site code, I realized that I had a mechanism that redirected any request that wasn't to coasterbuzz.com. Not www.coasterbuzz.com, and definitely not the underlying included name that included azurewebsites.net. Then it hit me... the health check mechanism was hitting the root of the site, and it was doing it at the azurewebsites.net, which I was 301 redirecting. The health check mechanism considers anything not a 2xx return code as unhealthy.

I made a new health check URL on the site, and excluded it from the redirecting, just for Azure's health check mechanism. Suddenly, both of my instances appeared healthy, and traffic was consistently being served to both. Not only that, but the flood of 301's stopped appearing in the logs. That made me happy, though I didn't really understand what was going on.

It wasn't until I RTFM'd that I really understood what was going on. If one of your instances is deemed unhealthy, it diverts traffic to the healthy ones, and spins up a replacement. But it'll only replace one instance per hour, and three per day, or maybe some other limit based on the scale level. The doc says both. But either way, it means you get down to one instance no matter what, and it keeps running it even though it thinks that it's "unhealthy." In reality it wasn't, it just thought it was because of my redirects. That's why I would end up with just one, but also two would stay up if I scaled it in or out (which I did because I would go to three instances when I deployed new versions of the app, as it never appears down that way).

Clouds are hard. But everything is easier when you read the documentation.

I think it was grade five that my class had a field trip to Severance Hall, home to the Cleveland Orchestra. We generally had at least two major field trips in those days, and I believe it was the same year that we saw some kind of performance in the Ohio Theater downtown. It was a pretty big deal, those trips, and it's one of those things where you just assume that everywhere is like where you live. Everyone has one of the world's best orchestras, exquisite concert halls and theaters (and huge amusement parks like Cedar Point). Of course, when you grow up, you discover that's not the case, and so those outings become a bigger deal in your memory.

Fast forward to 2007, when Diana and I on one of our first few dates go to Blossom Music Center, another iconic venue, to see the Cleveland Orchestra conducted by John Williams. That was one of the great things about "our" orchestra, was that you could see them inexpensively with lawn tickets. They were not out of reach for people.

Of course, I couldn't imagine at that time that we'd be married in less than two years. But to really stretch the imagination, one could never think that we would see this orchestra play in Orlando, in a building that hadn't been built yet, where we would be active donors, Diana would work for nearly a decade, and by the way, that building would be one of the most acoustically sophisticated in the world. In Orlando.

But that's the way the math worked out, and here we are. I was not familiar with the pieces that they played, but part of what made it so interesting was the way different instruments, especially the woodwinds, had a conversation. The flute would respond to the oboe, the clarinet to the saxophone. In that room, you could hear every bit of nuance. If you closed your eyes, it was like having headphones on but better. The spacial placement of sound was not a trick, it was what was really happening around you.

Not surprisingly, we talked to several people who were also from Cleveland, and a few that even return there in the summer (wealthy east-siders, unsurprisingly). It's always amusing how many people here are originally from Ohio.

That place has brought me so much joy over the last few years. I was kind of indifferent about the construction of Steinmetz Hall, I guess because it's not where the Broadway tours play, but it turns out we've seen a ton of stuff there. Our gigantic local community orchestra and choir played in there, as well as the Orlando Philharmonic Orchestra, which is itself pretty good for such a young orchestra. The London Philharmonic Orchestra played a residency series in the building, and we saw them with Beck and Sutton Foster. But also, we've had social events in there, with the floor seats flipped underneath into gala mode. I love to see and hear things in that beautiful place.

My first car was a rusted-out beater of a 1989 Ford Escort, which was 5-years-old when I got it. It famously threw a rod in the first two weeks that I had it, which was pretty devastating, but my dad helped me replace the engine. I picked up the habit of logging all of my gas filling and maintenance, mostly because I felt that I could see data in the log. If there were problems, the gas efficiency would most certainly show it, like when the gas tank started leaking. I also appreciated the variability of winter, when I'd get more like 20 mpg instead of 30 when it was really cold.

But what I never did was worry about how many miles I could drive. Especially then, car range was at best a guess, and not one that anyone would ever make. That's because if the fuel gauge was low, you stopped and put gas in the car. That beater probably could go around 250 miles on a tank, maybe a little more if it was warm and mostly on the highway, but I never tested it, because of the gas gauge.

This is why the range anxiety that people have over EV's is comically ridiculous. Oh, the haters will rattle off a million anecdotes about why you should worry about it, especially those around road trips, but they refuse to admit that the majority use case for American driving is not only perfectly fine for EV's, but better.

People drive an average of 29 miles per day. Two-thirds of Americans have a garage. So already, range and charging is irrelevant to a majority of Americans because they basically have a giant cell phone that they charge every night. We keep our cars at 80%, because it's better for the battery, and I couldn't tell you what the range says when leave the house. One of them is six-years-old, and presumably has some battery degradation, but I couldn't tell you what that is. It doesn't matter.

So what about the third that doesn't have a garage? They obviously have to rely on public charging, but beyond that, why is it any different than having a gas car? When the gauge gets low, you charge. At this point, chargers are pretty ubiquitous, and this is doubly true when your car can work with a Tesla charger, and more locations are opening up for non-Tesla cars. The only slow part of charging is the last 10%, which most of the time you don't need because you only drive 29 miles per day. Our cars can generally get to 80% or 90% in 15 minutes, and that extra 10 minutes relative to gassing up is inconsequential.

That leaves the road tripping scenario. I don't know if we're typical, but I suspect we need this at most five or six times. The only additional "burden" is having the car figure out where we can stop, and it generally works out to the point at which we need potty breaks or food anyway. The app has notified me about being almost finished charging countless times while in line to pay for food. Are there other anecdotes not like these scenarios? Sure, and they account for a fraction of a percent of real life.

But I go back to my original point... no one looked at range with gas cars, so why would you with an EV? The situation is actually better when you don't have to stop for "fueling," almost ever.

Do you ever wonder if some people are just wired to experience certain feelings as much as possible regardless of context? I say that because I feel like I've been dealing with serious anxiety since around the start of the pandemic. It hasn't subsided. The side effect is that I don't sleep well, and because I don't sleep well I'm less willing to get up and move around, which certainly has consequences for my health. And I say that at a time when work is mostly solid, I do feel physically OK for the most part (like, I don't get winded going up four or five floors of stairs), and I don't live where there's snow.

This is why I plan to ask my doctor next week about some kind of treatment for the anxiety and insomnia, which I imagine is medical marijuana. But that's after she's disappointed that my triglycerides are still high, though I now speculate that it's because of my hypothyroidism, which looks "good" because of the specific lab numbers related to it, but hypothyroidism can also cause the triglyceride problem. I'm at like 1/3 the dose of levothyroxine recommended relative to my weight, so I'm gonna ask her about it.

But there are a number of smaller things nagging me, and I want to complain about it. Discomfort in life is relative and not easily scored, so I'm not sure why I feel bad about it. Also, a lot of things to rope me in are a bit off, including three planned vacations, the first of which is still six weeks away.

I found myself making some optimizations to the forum app this week. The short story is that I serve user photos and avatars out of the database, and optionally uploaded images in posts (though the better way is to serve it right out of some kind of storage, which is what I do on my sites). The way I've had it working forever is that it reads all of the bytes from the database, then sends them all to the user's browser. But for some time, there has been a mechanism to make this a stream, basically a pipe from the database to the browser, which is more efficient and doesn't take up potentially huge blocks of memory buffering big images. So I did some work to serve those images that way.

It does relieve some memory pressure, so it was a good idea regardless. But my intent was really to figure out how to squeeze all of that stuff running into a smaller "server" and save $24 a month. Yeah, that's where I am now with this stuff. The forums actually do this right now, which is awesome, but despite being on different URL's, they're all the same copy of the app running (technically there are two instances running for redundancy). To that end, its memory footprint is pretty small. They're running under 1 virtual CPU's and about 1.75 gigs of memory, easily.

Everything not the PB forums is running on a shared "server" with 2 CPU's and 3.5 gigs, also two instances at $24 each. I have the redundancy because that's just kind of the way stuff can and should work in the cloud. If one thing goes bad, another one picks up the slack, and there's no interruption. But I'm sure users have noticed interruptions because I've tried a couple of times to squeeze the shared stuff into the smaller service, and of course it chokes. This shouldn't really be a surprise, because all of CoasterBuzz (including its forum), the main PointBuzz site, their separate image sites, cstr.bz, my blog, the dev forum build, my music cloud and a bunch of random things that are just parked there, and CB and PB alone are gonna need a good chunk by themselves.

None of this really matters, but I still feel pretty strongly about delivering the best possible technical outcome, despite the shitty ad revenue. I mean, traffic was up 35% last year on CoasterBuzz, which is pretty fantastic. And ironically, some of that might be the Google juice you get when you have a fast site. But the same Google isn't paying dick for ads, and without covering the page in floating ads and video, there's no great way to improve that.