Changing System Behavior

It seems almost logical for people who don’t understand systems, that if participants in a system exhibit highly undesirable behavior even after considerable warnings, you will get better results if you replace the individuals that are responsible for the “bad” behavior. Unfortunately, it often doesn’t work out that way. We’d like to understand why.

People generally do their best to further their own interests in a rational way. The difficulty arises because people can only act on what they know, rarely see the full range of possibilities and often fail to foresee the impacts of their actions on the whole system. People also fail to interpret the information they do have access to and frequently misperceive risk. Given the complexities of most systems today, people are often content just to meet their most important needs well enough to get by.

When a new person is placed within a system in a role that is subjected to information flows, incentives and disincentives, conflicting goals and the constraints of that position, they begin to feel the pressures of that role and frequently respond in exactly the same manner as others have in identical situations. There are numerous examples that justify that assertion. One of the most famous is the Standard prison experiment.

From Wikipedia:

“The Stanford prison experiment was a study of the psychological effects of becoming a prisoner or prison guard. The experiment was conducted in 1971 by a team of researchers led by Psychology Professor Philip Zimbardo at Stanford University. Twenty-four undergraduates were selected out of 70 to play the roles of both guards and prisoners and live in a mock prison in the basement of the Stanford psychology building. Roles were assigned at random. They adapted to their roles well beyond that expected, leading the guards to display to authoritarian and even draconian measures. Two of the prisoners were upset enough by the process to quit the experiment early, and the entire experiment was abruptly stopped after only six days. The experimental process and the results remain controversial. The entire experiment was filmed, with excerpts soon made publicly available, leaving some disturbed by the resulting film…

In psychology, the results of the experiment are said to support situational attribution of behaviour rather than dispositional attribution. In other words, it seemed the situation caused the participants' behaviour, rather than anything inherent in their individual personalities. In this way, it is compatible with the results of the also-famous Milgram experiment, in which ordinary people fulfilled orders to administer what appeared to be damaging electric shocks to a confederate of the experimenter.”

So in very complex systems with strict incentives and disincentives, conflicting goals and other constraints, undesirable behavior rarely changes when individuals are replaced unless the underlying structure of the system is altered. This result is often very surprising to those who don't understand the intricacies of how systems work.