The Shirky principle is the adage that “institutions will try to preserve the problem to which they are the solution”. More broadly, it can also be characterized as the adage that “every entity tends to prolong the problem it is solving”.
For example, the Shirky principle means that a government agency that’s meant to address a certain societal issue may hinder attempts by others to address the issue, in order to ensure that the agency remains relevant. Alternatively, the agency may become so focused on the current way in which it addresses the issue that it will fail to adopt better new solutions as they become available, thus prolonging the issue.
The Shirky principle has important implications in various domains, so it’s important to understand it. As such, in the following article you will learn more about this principle, and see what you can do about it in practice.
Examples of the Shirky principle
An example of the Shirky principle are tax-filing companies who lobby the government to prevent it from offering a free and easy way to file taxes, to ensure that the companies can continue to make a profit. A similar example of this are private prison companies who lobby the government to support policies that increase the number of incarcerated people and the duration of their incarceration.
Another well-known example of the Shirky principle is described in “Cognitive Surplus”, a book by Clay Shirky that contained one of the first discussions of this principle:
“PickupPal.com is… a carpooling site designed to coordinate drivers and riders planning to travel along the same route.
In May 2008 the Ontario-based bus company Trentway-Wagar… petitioned the Ontario Highway Transport Board (OHTB) to shut PickupPal down on the grounds that, by helping coordinate drivers and riders, it worked too well to be a carpool. Trentway-Wagar invoked Section 11 of the Ontario Public Vehicles Act, which stipulated that carpooling could happen only between home and work (rather than, say, school or hospital.) It had to happen within municipal lines. It had to involve the same driver each day. And gas or travel expense could be reimbursed no more frequently than weekly.
Trentway-Wagar was arguing that because carpooling used to be inconvenient, it should always be inconvenient, and if that inconvenience disappeared, then it should be reinserted by legal fiat. Curiously, an organization that commits to helping society manage a problem also commits itself to the preservation of that same problem, as its institutional existence hinges on society’s continued need for its management. Bus companies provide a critical service—public transportation—but they also commit themselves, as Trentway-Wagar did, to fending off competition from alternative ways of moving people from one place to another.
The OHTB upheld Trentway-Wagar’s complaint and ordered PickupPal to stop operating in Ontario. PickupPal decided to fight the case—and lost in the hearing. But public attention became focused on the issue, and in a year of high gas prices, burgeoning environmental concern, and a financial downturn, almost no one took Trentway-Wagar’s side. The public reaction, channeled through everything from an online petition to T-shirt sales, had one message: Save PickupPal. The idea that people couldn’t use such a service was too hot for the politicians in Ontario to ignore. Within weeks of Trentway-Wagar’s victory, the Ontario legislature amended the Public Vehicles Act to make PickupPal legal again.”
In addition, the Shirky principle can also apply to entities other than institutions. For example, an individual employee who’s in charge of a certain process in their workplace might resist attempts to automate that process, in order to ensure that the employee remains necessary to their employer.
Finally, note that the broad phenomenon described by the Shirky principle—entities prolonging a problem to which they are the solution—doesn’t necessarily have to result from intentional actions. For example, a company may inadvertently perpetuate the problem that it solves, because its processes are so focused on the mediocre solution that they’re currently selling, that they don’t realize a better solution exists. Similarly, a company may discourage the use of a certain approach to solving a problem because it previously failed for them, even after technological advancements make this approach viable.
One real-world example of the Shirky effect in this context is the cobra effect. It describes a case where British colonial officials in Delhi (India), set a bounty on dead cobras, in order to reduce the cobra population. However, this led citizens to breed the cobras for profit, and eventually to release them when the bounty was canceled.
A similar incident occurred circa 1902 in Hanoi (Vietnam), which was under French colonial rule at the time, when French officials sought to reduce the rat population in the city:
“To fight the infestation citywide, the colonial administration added vigilantes to its team of professional killers. Appealing to both civic duty and to the pocketbook, a one-cent bounty was paid for each rat tail brought to the authorities (it was decided that the handing in of an entire rat corpse would create too much of a burden for the already taxed municipal health authorities).
Unfortunately, this scheme backfired. Despite initial apparent success, the authorities soon discovered that the best laid plans of mice and men often go awry. As soon the municipal administrators publicized the reward program, Vietnamese residents began to bring in thousands of tails. While many desk-bound administrators delighted in the numbers of apparently eliminated rats, more alert officials in the field began to notice a disturbing development. There were frequent sightings of rats without tails going about their business in the city streets. After some perplexity, the authorities realized that less-than-honest but quite resourceful characters were catching rats, but merely cutting off the tails and letting the still-living pests go free (perhaps to breed and produce more valuable tails).
Later, things became even more serious as health inspectors discovered a disturbing development in the suburbs of Hanoi. These officials found that more enterprising but equally deceptive individuals were actually raising rats to collect the bounty. One can only imagine the frustration of the municipal authorities, who realized that their best efforts at dératisation [extermination of rats] had actually increased the rodent population by indirectly encouraging rat-farming.”
— From “Of rats, rice, and race: The great Hanoi rat massacre, an episode in French colonial history” (Vann, 2003)
Origin and formulations of the Shirky principle
The Shirky principle was proposed in a 2010 blog post by Kevin Kelly, editor of Wired magazine, who based it on the speaking and writing of scholar Clay Shirky.
Specifically, Kelly attributed the adage that “Institutions will try to preserve the problem to which they are the solution” to a statement that Shirky made in a recent talk, and noted that similar statements were made by Shirky in an associated blog post (“The Collapse of Complex Business Models”) and book (“Cognitive Surplus”). There, Shirky states that “an organization that commits to helping society manage a problem also commits itself to the preservation of that same problem, as its institutional existence hinges on society’s continued need for its management”.
In addition to mentioning the key quote that is now known as the Shirky principle, Kelly also says the following in his blog post:
“The Shirky Principle declares that complex solutions (like a company, or an industry) can become so dedicated to the problem they are the solution to, that often they inadvertently perpetuate the problem.”
Later, he also says the following with regard to this principle (bold added here for emphasis):
“In a strong sense we are defined by the problems we are solving. Yin/Yang, problem/solution, both sides form one unit. Because of the Shirky Principle, which says that every entity tends to prolong the problem it is solving, progress sometimes demands that we let go of problems.”
Essentially, in his writing on the topic, Kelly offers three formulations of the Shirky principle, which differ in subtle but important ways:
- The first formulation—“Institutions will try to preserve the problem to which they are the solution”—refers to institutions, and states that they will try to preserve problems, which implies that they do so intentionally.
- The second formulation—“Complex solutions (like a company, or an industry) can become so dedicated to the problem they are the solution to, that often they inadvertently perpetuate the problem”—refers to complex solutions, and states that they often inadvertently perpetuate the problem, which implies that they do so unintentionally.
- The third formulation—”Every entity tends to prolong the problem it is solving”—refers to entities, and states that they tend to prolong problems, without making any claim about their intentions.
The first formulation is the one that’s most commonly used when people discuss the Shirky principle, though Kelly does not actually refer to it as the Shirky principle in his original blog post. The third formulation, on the other hand, is the most general, though one issue with it is that it states that “every” entity engages in this kind of behavior, which is too absolute of a claim. However, this issue can be addressed by slightly changing this formulation, into “entities tend to prolong the problems they are solving”.
Note: In his post, Kelly states that Shirky’s observation reminds him “of the clarity of the Peter Principle, which says that a person in an organization will be promoted to the level of their incompetence. At which point their past achievements will prevent them from being fired, but their incompetence at this new level will prevent them from being promoted again, so they stagnate in their incompetence.”.
Caveats about the Shirky principle
There are some caveats about the Shirky principle that are important to keep in mind:
- The Shirky principle is just a general observation. As such, there are many situations where it’s incorrect. For example, an institution may successfully solve the problem to which they are the solution because there’s greater profit to be made that way than by prolonging the problem.
- The Shirky principle can involve various types of entities. Though the best-known formulation of the Shirky principle refers to “institutions”, this principle can apply to various types of entities, including individuals and small social groups. This is noted in the general formulation of the principle (“every entity tends to prolong the problem it is solving”).
- The Shirky principle can involve various causes. For example, one company may prolong a problem unintentionally, due to passivity or inertia, whereas another company may prolong a problem intentionally, due to greed or self-preservation. This is reflected in the general formulation of this principle, which doesn’t make any claims regarding the causes or intentionality of this phenomenon.
- The Shirky principle can involve various patterns of behavior. For example, one company may prolong an existing problem by not dedicating resources to developing new solutions, whereas another company may actively prevent others from developing such solutions.
In addition, the behaviors associated with the Shirky principle can vary in other ways. For example:
- An entity may not just preserve an existing problem, but also exacerbate it.
- An entity may create a problem that did not previously exist, if they can be the solution to it.
- An entity may perpetuate a problem that it benefits from, even if the entity is not actually a solution to the problem, though the entity may pretend that it is.
Based on this, a broader version of Shirky’s principle can be expressed as:
“Entities often promote problems that they benefit from”.
Accounting for the Shirky principle
Accounting for the Shirky principle can be beneficial when it comes to several things:
- Understanding past and current behavior. For example, it can help you understand why certain institutions are seemingly so bad at solving certain problems, despite all the resources—like time, effort, and money—that they dedicate to those problems.
- Predicting future behavior. For example, it can help you predict that an executive will keep perpetuating a certain problem, in order to improve their own status within a company, even though this leads to worse outcomes for the company itself.
- Modifying behavior. For example, if this makes you aware of someone’s incentive to prolong a problem, that could lead you to either eliminate the perverse incentive or create a stronger disincentive. Similarly, this could lead you to point out the issue to the entity in question, in order to encourage them to try and change their behavior themselves if doing so can benefit them in the long term.
When deciding how and whether to use your understanding of the Shirky principle in practice, it can help to assess relevant factors pertaining to your situation, such as what’s causing someone to act in accordance with this principle, and what outcomes their behavior leads to. For example, you will likely respond differently to a government agency that’s perpetuating a problem due to inefficient bureaucracy, than to a private company that’s perpetuating a problem out of greed, or to an individual who’s acting out of desperate self-preservation.
Finally, there are also two useful concepts worth keeping in mind when accounting for Shirky’s principle:
- Cui bono, which is a Latin phrase that means “who benefits?”, and which is used to suggest that there’s a high probability that those responsible for a certain event are the ones who stand to gain from it.
- Hanlon’s razor, which is the adage that you should “never attribute to malice that which is adequately explained by stupidity”, and which, when applied broadly, suggests that when assessing people’s actions, you should not assume that they acted out of a desire to cause harm, as long as there is a reasonable alternative explanation.
Related concepts
Parkinson’s law is the adage that “work expands so as to fill the time which is available for its completion” (or more generally, that “work expands to consume the resources available for its completion”). It relates to Shirky’s principle, since both concepts present a common way in which entities are inefficient or ineffective in dealing with problems that they’re supposed to solve.
Shirky’s principle also relates to another phenomenon that was identified by Parkinson, whereby the growth of a bureaucratic or administrative body is often associated with a substantial decrease in its overall efficiency. This is attributed to the desire of officials to increase the number of their subordinates, and to officials’ tendency to create work for each other.
In addition, a similar famous concept that’s related to Shirky’s principle has been expressed by novelist and social reformer Upton Sinclair, who said that “It is difficult to get a man to understand something when his salary depends on his not understanding it.”
Summary and conclusions
- The Shirky principle is the adage that “institutions will try to preserve the problem to which they are the solution”.
- For example, the Shirky principle means that a government agency that’s meant to address a certain societal issue may hinder attempts by others to address the issue, in order to ensure that the agency remains relevant.
- This principle can be expressed more broadly as “every entity tends to prolong the problem it is solving”, since it can involve entities other than institutions (e.g., individuals), and various patterns of behavior (e.g., unintentionally focusing on an outdated solution vs. intentionally interfering with competition).
- This principle can also be extended to say that “entities often promote problems that they benefit from”, since entities can also create new problems, exacerbate existing ones, and perpetuate problems that they don’t actually solve.
- Accounting for this principle can help understand past and current behavior, predict future behavior, and modify problematic behaviors (e.g., by removing perverse incentives).