What you’ll learn: In this opinion piece, MIT Sloan assistant professor Georg Rilinger argues that Agile’s iterative, decentralized approach to system design is incompatible with the complexity of platform markets.
Agile development was supposed to revolutionize how companies operate by empowering experts at the front lines to make quick and independent decisions based on local information.
But in the high-stakes world of platform markets, from ad tech to energy grids, its promises can be elusive. The very principles that made Agile a darling of Silicon Valley — decentralization, rapid iteration, and data-driven decisions — can produce deep vulnerabilities. A fundamental mismatch is to blame: Markets are not cooperative organizations but, rather, adversarial arenas.
Agile, once a niche strategy for software development, has become a dominant approach to making firms faster, leaner, and more responsive. Its core idea is seductive in its simplicity: Organize work into small, cross-functional, autonomous teams and let them iterate their way to solutions using real-time data.
While this promise holds for many applications, for large platform companies — those running our ad markets, labor pools, and even public services like electricity — Agile is running into problems. These failures are often blamed on culture or insufficient buy-in from management, but this diagnosis misses the mark.
My work shows that Agile’s core assumptions collide with the social complexity of markets in a couple of ways.
1. Agile’s focus on local autonomy transforms into a liability when not all actors in the system are collaborative.
The first problem results from Agile’s dependence on modular technology architectures. When systems can be decomposed into well-bounded components, teams can “own” them and iterate independently, minimizing the need for costly coordination. That logic animates Spotify’s “squads” and Amazon’s “two-pizza teams.”
This approach works only because the teams collaborate to maintain the boundaries of the different modules: When unforeseen interactions between subsystems occur, designers collaborate to identify the underlying issues and fix the relationships between the components.
On the surface, platforms mimic this model, dividing their marketplaces into modules — such as market matching, user interface, fraud detection, and payment — with clear relationships.
But these systems are not just modular code architectures; they are dynamic social systems populated with strategic actors who actively probe, exploit, and recombine the very boundaries internal teams depend on. When actors in the system actively try to benefit from problems in the relationship between different components, distributed teams with local autonomy are unlikely to identify these problems on their own.
California’s first electricity markets provide a stark case study in how the promise of Agile can unravel. The system of interrelated auctions was broken into modules that could be designed by dozens of small, modular teams — one for day-ahead auctions, one for real-time balancing, one for ancillary services, and so on. This setup presumed that the relationships between these markets were stable and well specified.
But traders quickly found ways to connect supposedly separate parts of the system, creating destabilizing arbitrage loops. Enron’s infamous “Ricochet” strategy, for example, exploited inconsistencies in the relationship between the import and real-time dispatch modules.
Because the energy companies benefited from these inconsistencies, they had strong incentives to hide them from the designers. Meanwhile, the designers worked in small modular teams that presumed that the relationships between the different markets were intact, and therefore they ignored the markets’ potential interdependencies. This made it difficult to identify and eradicate market actors’ games.
This phenomenon isn’t unique to energy markets. Across platforms, strategic actors — users, sellers, and creators — are constantly deciphering the rules and gaming the system, typically at the exact intersection between modules that internal teams treat as separate. Agile teams, locked in their sprints, can operate with blinders on, unaware that their local iterations are cascading across unseen boundaries.
This reveals Agile’s fundamental vulnerability in a platform context. Agile assumes that module relationships are fixed or elaborated cooperatively, but market actors can profit by finding, exploiting, and obscuring weaknesses in these very relationships. In this context, local autonomy — the very heart of Agile — transforms from a virtue into a global liability.
Leading Technical Professionals and Teams
In person at MIT Sloan
Get information
2. Market reactivity can weaken Agile’s ideal of “evidence-first” decision-making.
Another problem is Agile’s dependence on clean metrics and A/B tests. The core premise of data-driven iteration is that an experiment can isolate a variable and reveal a stable ground truth.
But platform markets are not stable laboratories; they are reactive, interdependent systems. A change in one module (like search relevance) doesn’t produce a clean outcome; it cascades, triggering strategic reactions from users (such as SEO spam) or advertisers (such as budget reallocations).
This interdependence and reactivity mean that the system’s baseline is constantly shifting. By the time an experiment concludes, the conditions it was testing may already be obsolete because the “control” group was never truly a control.
This constant flux can make experimental results from A/B tests and other simple designs fundamentally impure and ambiguous. When a metric (like user engagement) goes up, it’s rarely clear why. Was it a new feature? Was it a competitor’s outage? A seasonal trend? Or is the metric itself being gamed by strategic actors?
This ambiguity is where the Agile ideal of evidence-first decision-making can quickly break down. Because the data rarely provides a single, clean answer, it cannot settle disputes. In that case, it quickly invites political contestation. Data becomes a tool in internal power struggles. Different teams can interpret the same murky results to champion their own conflicting agendas.
Given that even simple experimental results become difficult to assess and invite political contestation, more radical redesigns may become completely elusive: The harder it becomes to build convincing evidence, the more complicated the political work to build a coalition that supports the change.
The bottom line: Data alone cannot substitute for structured authority
The facade of Agile’s data-driven decision-making often masks a political negotiation over whose interpretation of reality will win. When there are multiple visions vying for control in the face of a complex system that has evolved in interdependent ways over time, consistent designs rarely result. This, in turn, is likely to create the sort of problems that market actors can exploit and that are difficult to isolate from within highly localized teams.
This reveals a hard truth for Agile-run platforms: When the environment is this complex and opaque, data alone cannot substitute for structured authority. The Agile ideal of autonomous teams resolving issues with data fails because the data itself is the subject of the dispute. Without a mechanism to rise above the conflicting interpretations to make a judgment, teams are left in a gridlock of competing metrics.
Ironically, the platforms most devoted to Agile — such as Uber, Meta, and Google — face the harshest contradictions. As AI platforms expand into finance and health care, the stakes will only rise. An Agile-driven test-and-learn approach in these domains isn’t a benign failure; it risks systemic financial vulnerabilities or dangerously flawed medical recommendations.
The sheer complexity of ride-sharing, ad ecosystems, and app stores ensures that Agile’s assumptions are constantly violated. Agile remains powerful where systems are truly modular and feedback is credible. But in platform settings, modules blur and experiments rarely yield clarity.
Agile’s blind spot is treating complexity as purely technical, when in platforms it is fundamentally social and antagonistic. This is why many companies quietly betray their own Agile orthodoxy. They establish matrix organizations and integrator roles specifically to reduce pure Agile autonomy.
These structures create the necessary institutional scaffolding that enables teams to reason across modules, anticipate market-level responses, and create a structured authority for moderating the politics of testing. This move is a tacit admission that in complex platform markets, data-driven decision-making and autonomous teams are simply not enough.
Georg Rilinger is an assistant professor of technological innovation, entrepreneurship, and strategic management at MIT Sloan. His research focuses on topics in economic sociology, particularly social engineering in the digital economy, the role of expertise in government, the creation of markets, and regulatory failure. His recent work has explored practical obstacles to the success of market design, the theory of secrecy, the role of economic experts in governmental processes, and the theory of regulatory capture.
