The business of assumptions and up front design is an inefficient way to build software

posted by Jeff | Tuesday, August 23, 2011, 11:38 PM | comments: 4

I have to admit that I've become somewhat of a process geek. Maybe even an anti-process geek, but that sounds negative, and wouldn't entirely be accurate. The process of software development has changed a great deal, and part of the reason for that is we rely far less on shrink-wrapped product, and more on the Web. Even your mom is comfortable calling the thing going on in her Web browser an application these days.

Back in the day, you wrote software, and you put it on some kind of media (floppy disks!), then you put it in a box that was shipped to a store. The Internet was not something most people had never heard of, though really fancy people had modems that connected to their phone so they could call another computer. The point is that the stakes were very high for developing software. If you got it wrong, fixing a problem would not only be costly, but it could tank your business.

To combat this risk, development was an enormous process that involved a ton of meetings, documents, a rigorous QA process, and worst of all, enough up front design to make any kind of change incredibly costly. It was really hard to be innovative this way, even in the days when computers were far less powerful and there was only so much memory you could fit your code into anyway.

The unfortunate thing is that a lot of organizations still build software this way. It's unfortunate because the rules have changed so dramatically. Most software truly does run on a server somewhere, and your interaction with it happens in a browser. What this means is that you, as a developer, are no longer bound to the high risk world of disks in boxes that go to thousands of people.

The real problems with the old way of developing software really boil down to two things. The first is that everything you do is based on assumptions. You assume that it will take a certain number of days to develop some component. You assume that your user wants to do this certain thing. You assume your business model will be readily adopted. You assume that your design is the right thing. You know what they say about assuming stuff.

The second problem is that all of this planning is really just guessing. Because you've made so many assumptions, the planning you do hasn't really been vetted against the real world. There will be problems you couldn't predict. You haven't received any feedback from users based on a real product. You don't know how the market will respond. You certainly can't know if your design, whether it be architectural design or a user interface, is ideal given the lack of feedback. The only thing you can really guess will happen is that stuff will change.

And yet, I've watched this unfold time and time again. Development organizations will put days, even years into designing the crap out of everything before they write a single line of code. I worked in a place where analysts would produce huge documents outlining a use case, a single action for a single feature, and it would be further analyzed by a committee. A developer wouldn't even see it until it was "approved," by which time the agreement that the document was supposed to solidify was already tainted by the fact that no one from the developers to the users gave any feedback on it. The assumptions already made it obsolete.

Massive attention to up front design is bad, and here's why. A proponent of up front design will argue that you need to spec-out things in detail, in part to build consensus about what you're doing, and also because you don't want to leave developers and others to interpret things for themselves. Fair enough, but you're still doing all of that design work based on assumptions. It doesn't matter how visionary you are, because your assumptions could still be wrong. If that's the case, all of the time spent on this design, and the subsequent work to build out that design, is wasted, and you're still far away from delivering value to your customer.

Now let me tell you how I would do it. I would describe the smallest thing in the simplest terms while keeping the big picture in mind, and work with developers to do that. None of that throw-it-over-the-wall nonsense. We'd break it down into tasks that only take a few hours each, and prioritize them. We act on the items with the highest priority, re-prioritize weekly, and after four weeks, "ship" what we have. Maybe that means getting it in front of customers, maybe it means using it ourselves, whatever, but the point is we have something real that we can act on. If we're getting it wrong, we can correct our course and move toward the right thing in as little as a few weeks, incorporating feedback, challenging assumptions and delivering value quickly.

The proponent of up front design will say that you've just moved the design work to a different stage of the pipeline (and maybe suggest that it's bad to "allow" other stakeholders to affect the design, but that's a different cultural problem). Yes, I did move the design, but what's so great about this is that I didn't invest a lot of time into it. I didn't waste time building consensus or creating documents that no one will read, I just went and built something. The price of getting it wrong is much lower, and the integration of feedback happens much faster. In other words, I'm building stuff that delivers value faster, and when I get it wrong, I can correct quickly. The risk, and therefore the cost, is lower. I've managed out assumptions as a source of risk and cost.

This is still a hard culture nut to crack. If you're a big agile fan, with a great deal of success under your belt, how do you convince successful waterfall process folks that they're doing it wrong? It's not that they're doing it wrong, it's just that they're doing it slower. When you move too slowly, your competition kicks your ass. Unfortunately, it's the part that goes with "u" and "me."


Mike Morgan, August 26, 2011, 5:38 AM #

Investors wonder why Microsoft stock just floundered years, while new companies and Google had so much success.

Oracle stock ran from 15 to 35 during the past five years.

Google stock ran from 380 to over 700 during the five year period

Microsoft stock traded from 28.84 DOWN to a current price of 25 during the five years and was 30 ten years ago. One of the few real tech companies to go nowhere.

Why? Microsoft is too soft. Their executives are soft as compared to associates and executives at Google, Oracle and other successful competitors that steal their secrets . . . build their own companies . . . and then sell them to Microsoft! I know. My brother-in-law, Peter Leonard, is an Project Manager at Microsoft, so I've seen first hand just how soft and lazy these guys can get, not to mention the "who cares" attitude. Or talking too much and dragging projects out so long, that the competition is ready to go to market by the time Microsoft is ready to go to Beta.

All Microsoft execs and managers care about is a cushy job, nice dining room with great benefits and tons of time off. And when they don't get enough time off, they create it. Yes, I kid you not, they create time off hours off and fun time. They have the benefits of working at home . . . or from a beach in the Islands or anywhere they please, as long as there is a phone and an Internet connection. Believe me, I've been on those trips when he was "working" on the beach, in the pool, sipping pina coladas and enjoying the life of leisure.

If Larry Ellison caught one of his executives on a "working" day like this, he'd probably hang him from the top floor of the Oracle Building or from the mast of his sailing yacht, maybe even drag him behind the yacht as shark bait. But when he is really on the ball, he finds them a job at Microsoft.

Look at all the time Microsoft has had to catch Google with a Search Engine. Their first attempt failed miserably. Second attempt was to buy Yahoo, but Ballmer bungled that with an ego as big as his head, and probably a hundred execs and managers below him advising him . . . we can do it better. How? On the beach? At home? In the hot tub? Now they have Bing. Well, Bing This, because Bing is a complete failure because of guys like my brother-in-law who are there for the benefits, and not willing to take any chances or come up with anything new that might rock the boat of their executives that care more about the security of their positions than the guys and gals below them.

They talk about things they should not be talking about, so the competition can either leap frog them or incorporate their good ideas into their development, and avoid the mistakes made at Microsoft. I've heard things I should not have heard, so I'm sure others have. If you can't keep your trap shut at Microsoft, you've got problems.

So my advice to Steve Ballmer is to go out and hire as many Googlers as you can and start canning your army of drones, even if that means a short set back in Microsoft plans. Why? Because sooner or later, Microsoft will go the way of Sears, IBM, Kmart and others that refused to kick their managers and executives in the butt or out the door.

Jeff, August 26, 2011, 3:07 PM #

Not only does this not have anything to do with my post, but you don't know what the fuck you're talking about. Those are completely asinine generalizations to make about a company with 90k people.

Mike Morgan, September 6, 2011, 7:03 AM #

Jeff - The other day I posted a piece about my brother-in-law, Peter Leonard. I would like to apologize to you, as I have to him, and ask that the piece be removed from the blog. I made the post in anger and depression, having just lost my parents, and finding out my actual brother forged their Wills. I struck out at my sister and her husband, Peter Leonard in blind anger. In fact, Peter is a dedicated Microsoft associate that works his ass off day and night. When we are on vacation with him, he spends more time working than enjoying life, so it is truly embarrassing for me when I read what I wrote about him in anger at my own brother.

Please let me know what I need to do to have the post removed.

The post began with:

Investors wonder why Microsoft stock just floundered years, while new companies and Google had so much success.

Oracle stock ran from 15 to 35 during the past five years.

Google stock ran from 380 to over 700 during the five year period

Microsoft stock traded from 28.84 DOWN to a current price of 25 during the five years and was 30 ten years ago. One of the few real tech companies to go nowhere.

Thank You
Mike Morgan

Chris Hammond, September 6, 2011, 10:49 AM #

I got the same blog comment over on, time to remove it I think. Mike Morgan appears to be a spammer

Post your comment: