Stop and think: which of our tools are making us smarter and which of our tools are making us dumber. In my opinion tools and habits that support complexity literally train us to be dumber.Tools exert pressure. Your hand eventually reshapes to better hold the hammer. Your mind retains a plastic imprint of its repeated experiences. A work environment (either one laid by design or one risen up by accretion) colors both what is produced and the behaviors of those trying to produce. Repetition is training.
Tools that support complexity (versus eliminating or reducing complexity) introduce repeated ritualistic procedures (or more simply: bad habits). Without critical winnowing bad habits and bad tools accumulate in the workplace. The extra complexity and useless activity brought in by bending to bad tools creates additional procedures, rituals and stereotyped activities. These new activities attract new bad tools to support the additional complexity. In an uncritical environment this cycle of complexity and bad tools becomes self reinforcing and starves out useful production.
Most software development environments are desperately in need of an organized weeding. One criterion to consider for each tool is: does the tool support production (the presumed goal of your organization) or does it support complexity (the enemy of clarity)?
I will use the common tools of computer science and software engineering as my example.
Computer science started as a set of concerns coming from a tool that kept getting re-invented: the primitive computer (for example: Jacquard’s automated looms of 1801, Babbage’s uncompleted Difference Engine of 1822, Hollerith’s counting machines of the 1880s). The field of computer science was formed in the wake of these tools (and their successors) and codified in the 1930s by work of Church, Godel and Turing. The first tool of computer science was the computer and the first side effect was the founding of the field of computer science itself (a fairly expensive consequence).
The second generation of tools were simple and direct: loaders, linkers and eventually assemblers. Managing a computer was initially a hard job. These tools largely made managing a computer easier without introducing large side effects. The tasks were concrete and already implied by the invention of the computer (moving code and data into memory, letting different bits of code know about each other and producing the number-coded computer instructions from more readable text mnemonics). I believe it is this
generation of tools that set the expectation (in computer science) that tools are safe in that they tend to simplify pre-existing tasks (tasks that may or may not have been previously identified).
Then we moved to the age of software engineering. Newer tools such as: editors, interpreters, compilers and file systems took the field. Each of these tools had larger impacts outside of operating a computer. Editors changed the way we write (largely for the better). Interpreters and compilers allowed the introduction of new computer languages (which are essentially new models for processes, abstraction and thought). File systems become the dominant bad metaphor for organizing information.
Then an explosion of systems: graphical user interfaces, integrated development environments, source control systems, document management systems, wikis, bug trackers and issue trackers. And here is where we really need to start exerting some criticism and prudence. Some of these tools are dominated by their supporting rituals. Some of these tools support accumulation of complexity. Some of these tools specialize in managing concerns that did not exist prior their own introduction. I’ll try and focus on the unintended consequences of a few of these tools to try and make their insidious cost a bit more apparent.
Graphical user interfaces and integrated development environments are necessary for new users (and we all must spend a lot of time as new users, else we are not growing). The ability to push a single button to re-build an entire system is initially empowering. Until you find out you have to be there to push the button. You can’t automate the build (as only the IDE understands the build and eventually it can no longer export working scripts, Makefiles, ant files or other editable build files). This is so silly it has to be repeated: you can no longer automate a process that exist entirely within the computer. Then you have to push two buttons (“update files from source control” button and then “build” button). Then you have to push three buttons (refresh “source control state” button, then “update files from source control” button and then “build” button). As the tool traps more and more of the configuration knowledge you become more and more beholden and less clear on which steps are needed and which are mere superstition. Software engineering shops where checking out files takes over an hour, builds take a half hour and unit tests take an hour are not uncommon. And a user has to sit with the process to press “continue” throughout. As you would expect- under time pressure steps that are expensive are skipped and code is checked in untested (it being less effort to circumvent the source control change list managers and issue trackers than to jump through the near infinite number of hoops a semi-automated build system can support).
Document management systems and wikis also start very strong. Until they hit a certain critical mass where both their search function starts to fail (as too many irrelevant documents refer to critical search terms and mask the desired documents) and any desire to properly document or organize is lost to “it’s already in the wiki.” Meaning that a lucky search may find an out of date mis-informed document that purports to answer the question at hand. Eventually new engineers learn not to ask questions because they know not only will they have to find the wiki page, but they will be asked to update it.
My real invective is saved for issue trackers and bug trackers. Both of these systems start out solving a real problem: documenting required tasks in the first case and documenting tasks that fix flaws in the second case. But the goal should be to close issues and fix bugs, not track them. Issue trackers are routine abused by product managers. The existence of an issue tracker seems to relieve a product manager of: producing requirements, documenting requirements and answering any “a or b” question with any answer other than “both.” In a functioning business you can divide issues into: what we are working on now and everything else. It was once the product manager’s job to plan, prioritize and track issues and assign them to the development team in coherent linear order. Instead every idea (good or bad) is stored in a morass of systems with priorities, dependencies and a bunch of other knobs that grow to consume all free time. Bug trackers are similarly abused by development teams. When you submit bug: you want it fixed, not a requirement to produce copious documentation to enter into a system where it will be held for years (and become unreproducible as the software evolves forward). Both groups (product managers and development teams) should have tools for storing and organizing things- but these should be within the group, not used for communication.
There is a very solid reason to not favor tools that try to hide complexity. Such tools make complexity seem cheap and infact cause much more complexity. At some point the tool has encouraged so much complexity that the tool becomes indispensable and displaces useful work. This is a variation of the Jevons Paradox. If you make the unit cost of a good cheaper people tend to consume more of it. They use it as a substitute for other (more expensive) goods and usually consume so much more of it that they end up spending a larger total amount on the good. For example if 30% price decrease encourages a 50% increase in use, then: your total expenses go up by 5% (not down!). You lower the perceived unit cost on complexity you likely get more overall complexity as a result. These changes seem small, but they accumulate. Every time you make managing complexity and delaying decisions easier you get more overall complexity. Complexity is used as a cheap substitute for product research, user studies, making decisions, writing correct code and actually fixing bugs.
Of course it is all a matter of degree. When these tools are used to simplify an already existing workflow they are good. When they allow a useless ritual sequence to live and grow they are bad. There is a surprisingly effective way to measure the degree of damage tools are inflicting. Stand quietly and listen to the office sounds. If you mostly hear typing and talking: things are likely good. If you mostly hear mouse clicks: things are bad. Typing is usually part of a proactive task and mouse clicking is usually part of a reactive ritual. If you are an interested manager with a very strong stomach: attend new engineer training (if you have it) or shadow a new engineer for a day; it will be painfully obvious if your environment is empowering or teaching helplessness.
Uncritical repetition of bad habits literally trains you in being dumber. A work environment that accumulates bad tools becomes a mind dulling quagmire of required rituals. We must be vigilant in encouraging tools that support production and discouraging tools that support needless complexity. There should be no natural desire to “manage complexity,” the goal should always be to eliminate complexity. Prefer a board of Post-Its to an issue tracker. When your project is a mess the board will look like a mess, whereas the issue tracker will hide and support the mess. An earlier crisis is a cheaper crisis. Unexamined your organization’s procedures perform a random walk (bringing in new tools and habits) that is naturally biased towards ruin; you must continuously apply concious corrections to avoid this.
Distaste for tools is , of course, not new. In Plato’s “Phaedrus” Socrates recounts an Egyptian god warning that the invention of writing will create: “hearers of many things and will have learned nothing; they will appear to be omniscient and will generally know nothing; they will be tiresome company, having the show of wisdom without the reality.” Or that the greater quantity of text (supportable only by the invention of writing) will cloud minds. The distinction is: whether a tool is more forceful in its primary purpose (in the case of writing: archiving information) or in its parasitic supporting rituals (in the case of writing: the production of papers, pencils, inks, book binding and building shelves).
Do not reward building shelves unless building shelves is your actual business.
Categories: Opinion
jmount
Data Scientist and trainer at Win Vector LLC. One of the authors of Practical Data Science with R.
Nice rant. I would include continuous builds as a tool that tends to generate work (or at least spam), via spurious false positives, plus the work needed to babysit them.