” If you haven’t done something before 3 times, successfully, don’t assume you can do it.”
People of complain that the engineers are not predictable. They complain that while you can put pressure on a Sales person to take a larger quota, you can’t put pressure on an engineer to bring in the date. It does not seem fair.
They believe that while the competition has great engineers who deliver great product quickly, their own engineers are just not working hard enough or producing the right results. Nice people, but not world class engineers. “How do we get our engineers to be more serious and hardworking?”, is a common refrain.
The basic reason that people are unhappy with engineering is that they believe engineering to be just like their own work, just with poorer results. They think in terms of knowing what you are doing and then executing to it. The problem with software engineering is that “we haven’t done the same thing 3 times” and so we don’t know if we are going to be successful. Most engineering projects are unique, and therefore, not as predictable as everyone would like. When you get down to it, software engineering is not yet a predictable profession. We really are still guessing about how to build it. (see Vasa Disaster).
David Snowden has developed “The Cynefin Complexity Model” that explains the different types of work, the management styles, and expectations around different types of work. The model helps put things in perspective about how different types of work behave and what to expect from them.
At the bottom right of the figure is work that is Obvious. When I recently got rear ended (for the 2nd time in 18 months), I went to the Auto Body shop. He had a book that defined the time for each of the items that needed to be repaired. The insurance company agreed with that book and so the price was known within an hour of the start of the project.
At the top right-hand side of the figure, we have things that are Complicated. Complicated things have unknowns, but you can make a list of them. When I go on a business trip, I’m not sure which airline, when I will leave, how long I will be gone, and other unknowns that I put on a list and solve. Sales fits into this part of the model. Sales is highly probabilistic. If you know the percentages in each of the steps of the funnel, you know how to manage it. I’m not saying it is easy, I’m saying it is complicated.
Software Engineering fits in the top left-hand side of this model, Complex. Each engineering project is the first time something is being done. We haven’t done things successfully 3 times which means we should expect surprises. We just do not know where they will come from, however. In software, the unknowns are going to smack you in the face, in ways that are often not foreseeable.
I have one client that found a key part of their product relied on an Azure capability that was deprecated. They literally had to take all of engineering, re-write all the needed software and then cause all the customers to upgrade to the new capability. They last more than a quarter to this unknown surprise. I could go through many more examples, but if you are reading this note, you probably have your own examples of the engineers declaring a surprise and a slip to the delivery.
I am not arguing that software engineers should do everything they can to reduce risk and eliminate surprises. They should absolutely do risk management. But given the domain they are dealing with, there will always be surprises that creep in.
So why is it that people are so disappointed with software engineering? It does not meet their expectations of behaving as if it was in the Obvious or Complicated parts of the model. To a great extent people living and working on the right hand side of the Cynefine Complexity Model judge competency by how closely people behave within the rules of the model they know best, ignoring the fact that software is in a different part of the model and needs to be judged in a different way. The rest of the company’s experience is not a good experience for judging software, and they are prisoners of their own model. We need the rest of the company to understand software is a different model and judge them according to the rules of the Complex domain.
The reason I started with the Ray Dalio quote, is that it exemplifies software well. There is much more “newness” to the tasks than in the other disciplines. Planning does not work as well in software because the various unknowns are unknown, so they can’t be planned. Unlike some other engineering disciplines, software is not yet a science. It is not math, it is an art, with people doing their best to figure out how to do what is being asked, which is usually not precise either.
This lack of understanding the real challenges of software ends up putting too much pressure on software engineers to do more than they say they can do. A frequent management belief is that hard to achieve goals cause people to do more and causes them to rise to the occasion. In software this backfires and results in poor quality. The engineers try to please but end up cutting corners, impacting quality. In the examples that follow, I would argue that the business people were ducking their own tough business decisions and forcing the engineers to make the business decisions, with the predictable result of poor quality.
Looking at the recent problems of Wells Fargo and the cheating done by their people should put some caution for all parts of a company. Or look at the Boeing 737 Max problems. Were they at all related to the incredible pressure to get the plane delivered in a short time frame? Or the original website for the ACA? Or the scandal at VW over lying about the Diesel emissions? But these are just the most egregious public ones we can see. It is usually less dramatic and more insidious because it is not fixed systematically, it is compounded by future pressure.
The examples above are of so much pressure to deliver that people cut corners to get to the end. If you ask people to do the impossible, they will cut corners. I am not talking right or wrong here, I am talking about human nature. This pressure is created by those who are not building the software and ends up being pressure to those who are building it. That the engineers lack of political power to say “no”, is a giant contributor to their taking on the challenges they fail at. They are trying to be cooperative. Unknowingly, the rest of the company is contributing to the low quality of the hurry up solution. They are working in their own model while putting pressure on a different model. It does not work, as can be seen by all the missed deadlines and poor quality.
Most software groups are under too much pressure and so they cut corners. The people who create this pressure think they are helping, they are not. They are causing their own companies to fail, and often causing the attrition that follows. Their disappointment with the engineers is clear to the engineers and only fuels the pressure fire.
The cost the company pays for this is not obvious enough to those who are driving the pressure. The cost of fixing a bug if it is found the same day it is created is 100 times cheaper than fixing it in the field. Having many bugs pile up because they are ignored in order to get things out the door, creates a huge liability for most software companies. Understanding these issues and playing this game well is life or death for most companies.