Home‎ > ‎

How to design big systems

 Having multiple perspectives is everything. The more ways you can see it, the more likely you'll know what to build. The worst thing you can do is start designing and building a large system after hearing only one person's idea of what it should be. If you do nothing else to prepare, if you read no further in this article, then at least walk away with the mission to ignore your boss and learn about the problem from everybody else's point of view.

The Five Rules of Big Systems

1: All big systems that work invariably evolved from a small system that worked1

 Under no circumstance can it be concluded that a large system designed from scratch and rolled into production at once will ever work. This is not because of human fallacy, and therefore cannot be defeated by assuming you're smarter than the average human. This rule is a consequence of the second rule, which you can see below, but know that it means any big system will only work because it has been deployed in increments.

 "Big Bang" deployments almost always fail, and they've begun to give software development, and IT particularly, a lingering odor. The FBI's Virtual Case File System, the IRS Modernization, countless and anonymous business IT failures have "too much at once" as a major contributing factor.

2: Every system changes its own requirements

 This counts systems as small as levers and single feedback loops (think about how the flush toilet changed the world--and that uses just one feedback loop), but is more apparent sooner the bigger it is. The minute you deploy any system into a production environment that environment will change, and it will even change the requirements you were given for the system in the first place. The act of implementing a system will change the justification for implementing that system, and soon you will find yourself fielding change requests that ultimately trace themselves back to the system's own presence and behavior.

  • Web applications that required the user to remember an account name and password created the demand for Single Sign-On systems like Open ID, which came back as new standards and requirements that the web applications had to support
  • Your company's fulfillment department comes to you with a requirement to manage all of the picking slips your system is printing every day, but which cannot be fulfilled immediately due to inventory fluctuations caused by issues with the user interface you designed to keep it updated
  • The pre-internet service CompuServe was created to make money from leftover CPU time on a timesharing minicomputer originally purchased to run a life insurance company. The service eventually became the sole justification for purchasing more computer hardware, then spun-off as an independent company that was later acquired by H&R Block, and then again two decades later by AOL
  • You throw a random number into the URL of every link on your web site to defeat the aggressive caching policies of ISPs and web browsers like AOL, so while it's logged by the web server it's also ignored by the application server. Now, for this behavior, the marketing department has begun to exploit that number to make it act like a tracking ID that identifies the source of each visit
  • One of the largest costs of the US Government is the operational overhead of the IRS, literally demanding more taxes in order to pay for the collection of taxes
 It doesn't matter how trivial you think it is, by plopping it into the fertile ground of human commerce it will grown its own branches and you will find yourself obliged to maintain them. A more pessimistic view is that every system tends to oppose its own function2, such as how austerity budgets tend to cause record deficits, or how safety equipment is now the leading cause of injuries.


1 - John Gall, The Systems Bible