The Importance of Conceptual Integrity

Recently, I had the opportunity to re-read one of my favorite books on software engineering and design, The Mythical Man-Month. There are so many interesting nuggets of wisdom in that book, but the one that resonated with me this time around with the idea of "Conceptual Integrity". This is the idea that successful software projects are the result of the vision of one person, one mind. In that way, the design of the software reflects a consistency with all of the features fitting together a logical fashion.

The opposite of a design with "Conceptual Integrity" is a piece of software that is designed "by committee". In this situation, you have competing visions battling to insert features into the design. When this happens, it is inevitable that compromise and miscommunicaton will take place. It is difficult for all the members of the committee to agree on the same vision, so everyone will have their own ideas as to what the design should look like. What you end up with is a piece of bloatware that is confusing to use and often full of bugs. A lot of mass market software suffers from this malady. I'm sure we can all think of pieces of software that are burdened with tons of little features that are confusing to use because they don't seem to fit together.

I see this happen all the time at my Fortune 500 company. Often you have different groups going off and designing programs and systems without consideration with how they fit into the overall strategy of the company. One example of this is something as simple as the design of a logging system. Every group as its own vision as to what should be logged, where it should be logged, and how it should be logged. Therefore, everyone ends up building their own logging component that reflects that vision. Obviously this is a problem because now you have applications with log files in different places, in different formats, and with different user interfaces. Aside from the duplication of labor issue, it becomes hard to support systems that have applications with different logging "visions".

The solution was to try to build a common logging component. Of course, the common component reflected a totally different vision that was often incompatible with everyone else's vision. As a result, nobody wanted to use it!

The underlying problem is that there is nobody who is the overall technical architect with the authority to set the vision for the entire company. That person would have been able to say, "this is what the logging component vision should be, and my vision is that everybody should use it." Thus, the systems at my company would have some semblance of Conceptual Integrity.

Obviously the role of that one overall architect becomes very crucial, and you don't want to give that power to just anybody. That is why it is important for companies to identify promising software designers, cultivate them, and keep them happy. Their performance can make of break a company's software product so it is an investment that is worthwhile. The alternative is to continue to design by committee and keep designing software that makes no sense!