Every user you'll ever have is trying to make you solve an engineering problem that few engineers have ever seen, and fewer yet have ever solved. Nobody hires a programmer because they have a pre-solved problem, because if their problems are all known and catalogued and solved then their solution would be on the shelf of a store already. That you are a developer or entrepreneur is evidence that what you're solving is unique enough that the benefit is in your favor, but the odds are not. You won't know what you're doing because your customer doesn't know what he's doing, and your customer doesn't know what he's doing because he's in the middle of discovery as well. There are good reasons why your problems are underestimated by your bosses and investors, though, and the biggest is that most things are similar. Most people look and behave alike, and most objects can be compared to other things. So any unique problem a man encounters can be miscompared to a similar problem that's already been solved. And there is such a strong economic incentive to solve a new problem with an old trick that good problems go misdiagnosed. The problem of vision ("We need faster horses, Mr. Ford")Visualizing the nature of a computer program before its built is nothing more than a bit of useful guesswork, and is akin to picking a random house to begin with on your quest to find a new home. Each new house that you visit will show you something that you didn't know you wanted before. You may have thought you wanted a big garage, fireplace and swimming pool, but by your third or fourth house you've discovered that you want a spiral staircase, finished basement and brick construction even more, and you've re-evaluated your need for a pool and garage.
The same happens when gathering requirements for a new software project, and very few customers understand that they cannot possibly know what they need the computer to do for them until they've seen how it works. The developer must be part of the requirements discovery process, and everyone must be aware that the discovery phase will continue all through the development cycle and into the maintenance cycle forever. Maybe it's even more apt to say that the discovery phase is the software, and working programs are disposable artifacts found along the way. Indeed, if discovering what you want from a computer is like discovering what you want from a house, then our analogy would continue by saying that sometimes--after visiting and living in a dozen different houses--you realize that you'd be better off spending your money on a backpack and camping gear. The problem of desire ("But I wanna pony")False needs and requirements pollute the vision of a program and are the means by which one can force a project to fail. If you wanted to sabotage an undesirable project, then one must merely gold-plate the requirements or demand inapplicable standards. But ponies, unicorns and rainbows don't really sneak into program requirements because the request was malicious, but because someone really had their heart set on it. In 2009 fantasies take the form of AJAX, Ruby, SOA and REST. Ten years before it was XML, Java, Extreme Programming and Design Patterns. Ten years in the future it will be Functional languages, Solid-State hard drives and schema-less databases. These problems enter the design because desire was aroused by something that had nothing to do with the problem. When an engineer wants a pony it's because there's something interesting about it, and when a customer wants a pony it's because they feel they need it to be competitive. They hear "relational databases are passe, now" and insist their payroll application has to run on a sharded Map-Reduce cluster. But the universe itself is the most malicious partner; it works hard to contrive problems that--by nature--will get worse if you try to solve them with the ponies and unicorns of your time. It sets you up for a fall before you've even started writing code. The problem of change ("Now it needs to be a submarine")The Enterprise Resource Manager (ERM/ERP) is a classic example of institutionalized cluelessness. Every business thinks it has to have one, and they all think they know what they want from it. But there aren't any businesses who have a successful installation of an ERM/ERP because the company's nature will change before they've finished configuring it. It will change in at least one way that wasn't anticipated in the design of the software, and so the company's employees either begin their inevitable trend of working around and avoiding the software, or the engineers begin their inevitable trend of damaging the software's design to cope. Here are some specific examples of changes that broke ERM/ERP designs that I've seen:
In addition to changes in the business model, there are changes in low-level protocols that frequently force compromises, hacks, or even entire system rethinks:
Changes like these are always justified in the short term or immediate scope; they always have a benefit for somebody. And the reason why major changes occur is because business growth itself is driven exclusively by change. A business grows because it convinces a customer to change from wet shaving to electric, or from charge-cards to credit-cards and so-on. And businesses compete by changing their nature to be strong at what others are weak at. The problem of organic growth ("I see lots of features, but what does it do?")If the solution to fuzzy vision is to experiment and wander, and the solution to desire is to bring in the accountants, and the solution to change is to stay agile, then the penalty for all three over time is a confused product. Data doesn't like to live in multiple places, so the programs closest to the data accrue functions to be everything to everyone. Now you get software that's both a floor wax and a salad dressing, and your training costs for both users and maintenance developers increases--for doing nothing more than explaining to them why things are they way they are. This is ultimately loss aversion: nobody wants to throw away good code, so they try to keep it, but make it stay relevant somehow by morphing it. Alas there is no anti-wrinkle cream for software, and most code-base owners are far too tolerant of ugliness. Software must be treated like it is, like tissue paper. It's a step on the road to programming humans instead of machines; each development project is an academic exercise with the side-effect of being useful to somebody for a while. In a way, state and community colleges have already figured this out and have their students rebuild the school's web site and class signup software every few years as part of their CS curriculum. It gets thrown away a few years later by another batch of kids, and they always do a shitty job of it each time, but this is what it's like in the enterprise, too. Progress must come from the evolution of computer languages, and not from the artifacts built with them. Each generation folds another layer of experience and insight into new syntax, not new modules. Next year's batch of kids will still make a shitty web site, and they'll graduate to make shitty payroll and inventory management systems for the companies they work for. Everything they make will have to be thrown away. But each generation's produce will get better and better in quality because the tools they use will get better. Since the dawn of computers in business, each re-imagination of software for managing tea shops gets better and takes less time to develop. A treadmill, forever. But we can make the treadmill nicer. |
 About The AuthorChris Wenham develops iPhone/iPad apps in MonoTouch. Take a look at my new Personal Finance app, Spent Better--the All-In-One financial calculator with a twist.  If you like my style, follow my other writings at ChrisWenham.com |
Home >
Why nobody knows what they are doing |