Examining the limitations of digital market design for public goods

MIT Sloan professor’s new book explores the coordination challenges involved in planning markets, and the obstacles that can lead to failure — using the California electricity crisis of 2000 as a case study.

Cambridge, MA, August 28, 2024 — The California electricity crisis in 2000 was one of the greatest financial disasters of the past century. Decades later, the question remained: Why did the newly created electricity markets fail?

In his new book, “Failure by Design: The California Energy Crisis and the Limits of Market Planning”, MIT Sloan School of Management assistant professor and sociologist Georg Rilinger explores practical obstacles to market design to offer a new explanation for the crisis — one that moves beyond previous explanations that have typically blamed incompetent politicians or corrupt energy sellers.

Rilinger reveals that the crisis was a case of market design failure. California’s electricity markets were the product of deliberate design. Market designers build digital platform architectures that structure the interactions between buyers and sellers. They pursue a distinct project of engineering, trying to nudge market actors toward desirable behavior. “This is really an experiment in social engineering,” Rilinger said. Despite careful planning by some of the world’s leading experts, California’s markets became susceptible to persistent forms of manipulative behavior that derailed designers’ goals.

Originally a rather academic pursuit, Rilinger noted, market design is quickly becoming the guiding principle behind the organization of commerce in the digital age. As more and more of our economic life migrates into the platform economy, the opportunities for “economic engineering” multiply. If designers succeed, the markets identify the cheapest combination of generators to serve demand. In the case of electricity, the system operator always ensures that there is enough electricity to go around, and the markets are designed to identify the best way to deploy the existing generators.

“Using the creation and collapse of California’s first electricity markets as a case study, I identified several practical problems that make it difficult for market designers to strike the right balance between different strategies to enforce the desired behavior among market actors. These difficulties are particularly salient when they try to build markets for complex allocation problems,” Rilinger said.

“While designer markets are a great tool to solve problems that are well defined and can easily be separated from their environment—like assigning students to schools, or auctioning spectrum licenses — my study suggests that they are most likely not great for problems that require the failure-resistant management of complex infrastructures — like electricity, health care, and water,” he noted.

Why is this? A combination of factors that prevent designers from striking a workable balance among simplifying, bounding, and controlling:

• The organizational structures that designers adopt to build complex platform markets can introduce inconsistencies that are difficult to identify and fix; this runs up against the imperative of simplification — one of the key strategies to control designer markets.  
• Professional differences between economists, engineers, computer scientists, and other market designers can both create and obscure structural problems in different parts of the system; this tends to run up against strategies to bound and control the system effectively.

As Rilinger shows in the book, and in the case of California, these practical obstacles to the success of the market design explain why the markets were built in a way that became susceptible to undesirable behavior — including manipulation or fraud — by market actors. They can render market design solutions infeasible when the markets are supposed to help with the management of complex infrastructures.

“The reason is that the platform software for complex optimization problems needs to meet very high consistency standards to create the correct incentives for market actors.” Rilinger said. “This is hard to accomplish for political, organizational, and professional reasons when building complex market systems. Market designers are not a unified group of experts but come from several different disciplines that may be talking past each other, and they often have difficulty achieving political influence. It is hard to coordinate work between disparate design teams in complex organizations.”

More broadly, Rilinger’s book explores the coordination challenges involved in planning markets, to understand practical limitations of market design on its own terms. “Designers are trying to match users to behaviors that fit the logic of an algorithm that solves an optimization problem. Usually we plan organizations and most of our theories are about how we can coordinate people in organizations successfully,” Rilinger said.

“The book suggests that planned markets operate differently than formal organizations and that we cannot simply port our organizational theories to these markets. It suggests that market planning is harder than organization design, because markets cannot necessarily rely on the cooperative orientation of employees and the managerial tools that this makes available to designers.”

About the MIT Sloan School of Management

The MIT Sloan School of Management is where smart, independent leaders come together to solve problems, create new organizations, and improve the world. Learn more at mitsloan.mit.edu.