Lehman’s Laws on software evolution demonstrate the complexity of software-intensive systems and the organization and process for developing them. I find the last law central to the whole theory: “E-type evolution processes constitute multi-level, multi-loop, multi-agent feedback systems and must be treated as such to achieve significant improvement over any reasonable base.”
We can demonstrate this with a little help from system dynamics theory. System dynamics tries to breakdown the complexity into variables and looks then at their interactions, and especially loops of interactions.
The picture above is a system dynamics diagram for software-intensive systems. Green arrows show positive correlations, while red arrows show negative ones. The variables are also categorized into three areas: feedbacks from the market, characteristics of the software and activities in development. The multi-levels, multi-loops and multi-agents (market vs. software development organization) are also visible in this diagram. A loop with an even number (0 is an even number) of red arrows is a reinforcing loop, while a loop with an odd number of red arrows is a balancing loop. Balancing loops stabilize variables, while reinforcing loops push them in the positive or negative direction.
Let’s take a look at some reinforcing loops (green circular arrows):
- Increasing market acceptance drives demand for new features, which drives feature implementation and total feature set. In turn we get higher market acceptance. Beware that the reverse is also true: if we reduce feature implementation if market acceptance is dwindling, we will end up with lower market acceptance.
- Similarly market acceptance increases number of reported bugs due to higher usage. Which in turn drives bugfixing activities, increasing external quality and again increasing market acceptance.
- If internal quality is low, development organization cannot get familiar with the codebase and cannot do useful refactoring in the codebase, causing further decrease of internal quality.
- Technology innovation & obsolescence is modeled simplistically by driving itself: new technology breeds the ground for further technology, which in turn drives porting and adapting activities of our software.
These are all quite obvious loops, which most software developers understand by intuition. My key takeaway from this analysis is that vicious cycle = virtuous cycle. If you can manage to turn it the other way around, you can move the variables in the desired direction.
Balancing loops (red circular arrows):
- Increase in the total feature set tends to decrease the internal quality, which in turn also decreases the feature implementation speed, reducing the increase in total feature set. In other words velocity will decrease over time if we don’t maintain the internal quality of the codebase.
- Demand for new features will be balanced by feature implementation efforts. However the bigger loop with market acceptance fights with this trend to demand more features.
- Similarly bug reports will be balanced by bugfixing efforts. Similar to features, they will still tend to increase due to higher market acceptance.
Coming back to Lehman’s Laws; we can elaborate on them using this analysis:
- Continuing change: As the loops driven by market acceptance show, unless software stopped being useful, demand for features and bug reports never end; so the development never stops. Technology innovation adds to this by requiring porting and adaptation efforts. In short, as long as a software is used, it has to be developed.
- Increasing complexity: The continuous increase in feature set pushes internal quality (reverse of complexity) downwards, unless refactoring is emphasized.
- Self regulation: The balancing loops make sure the activities stay balanced, as well. Too much feature implementation will drive your velocity down, too much refactoring will drive your market acceptance down. The activities are balanced, almost by themselves.
- Conservation of organizational stability: This law states that the overall amount of activity for a software doesn’t change much. This is not directly visible on the diagram. We can understand its consequence as the decision for the software development organization is not so much the total amount of activities, rather how to allocate activities among features, bugfixing, refactoring and porting & adaptation.
- Conservation of familiarity: The familiarity of the organization (not just development, but also sales) has to be preserved, as it drives the ability to release software as well as drives internal quality indirectly.
- Continuing growth: This is reflected best in the total feature set. Increased market acceptance drives new features, decreased market acceptance doesn’t directly remove features.
- Declining quality: This can be considered a corollary of increasing complexity, the negative tendency of internal quality driven by feature set, also results in a negative tendency of external quality.
Of course this diagram is just a model, an approximation of reality. The model can be adjusted, enriched with more details, further loops could be considered… My goal with this model was to show the inherent complexity of software-intensive systems to make the point that improvements are only possible with a good understanding of the dynamics, a well-established north star and only in iterative steps. It is somewhat similar to economies: you cannot control inflation just by ordering prices to go down, you have to appreciate the complexity and achieve the desired changes gradually, for economies as well as for software.
p.s. Do check out the book Accelerate, where the authors showed a similar interplay of variables around software development based on a vast collection of data.