What are power shadows?
A working definition from MIT Sloan
power shadows (noun)
Biases or systemic exclusion of a society, as reflected in data sets.
As a graduate student working on facial analysis software, Joy Buolamwini, SM ’17, PhD ’22, encountered a problem: The software did not detect her dark skin, though it easily identified the faces of people with lighter skin. She had to wear a white mask to be detected by the computer.
The problem was the power shadows that are cast in datasets, Buolamwini, a computer scientist, writes in her book “Unmasking AI.” The dataset she used — created by a government agency with the intent of representing a diverse population — featured images of public figures, often elected officials. Around the world, white men have historically held political power, and that was reflected in the data. The software she used was trained on images in which people were predominantly white and male.
Power shadows have serious consequences. Machine learning models used to inform hiring decisions could reflect past biased decisions made by humans. Models used to diagnose medical conditions could be based on data from mostly white patients, resulting in poor outcomes for Black patients.
The first step to overcoming power shadows is being aware of them, according to Buolamwini. “We must also be intentional in our approach to developing technology that relies on data,” she writes.
Working Definitions: Artificial Intelligence
MIT Sloan's Working Definitions explore the words and phrases behind emerging management ideas.
AI Executive Academy
In person at MIT Sloan
Register Now
Data liquidity leads to AI success
Three levers — data architecture, data preparation, and data permissions — determine whether data becomes a reusable strategic asset or stays trapped in silos.
Pro-worker AI, explained
Artificial intelligence can make workers more capable and productive, but only if leaders design and deploy it to augment human judgment.
5 things to consider when working with AI
Researchers at the MIT Initiative on the Digital Economy share the latest insights about getting the most from working with AI, such as personality pairing and reorganizing job tasks.
Balance AI innovation and risk with ‘minimum viable governance’
As organizations scale generative AI, traditional governance models prove to be too rigid or too loose. Minimum viable governance calibrates oversight to risk, enabling responsible innovation.
The surprising power of warmth in AI negotiations
In MIT’s international AI Negotiation Competition, “warmer” agents achieved better outcomes in negotiations with other AI agents.
Seeing real value from AI depends on being able to verify its outputs
A new paper explores how seeing economic value from artificial intelligence hinges on closing the gap between what AI can do and how humans can verify its outputs.
‘AI gravity’ is pulling you toward dependency. Here’s how to push back
AI systems hold the promise of competitive advantage, but they can usher in cognitive decline among workers, says MIT Sloan School of Management’s Eric So. Learn how to protect cognitive capital.
Heeding the pope’s call to ensure AI protects human dignity
Following Pope Leo XIV’s encyclical on AI, MIT Sloan professor emeritus Thomas A. Kochan argues that firms should partner with workers to ensure AI augments human skills and knowledge.
What senior leaders want to know about AI
Leaders are turning to MIT Sloan Executive Education to learn more about AI, including managing humans amid technological change and rethinking their relationships with IT departments.
What leaders still get wrong about AI
Organizations are struggling to succeed with AI. Research from the MIT Center for Information Systems Research shows common mistakes and how to overcome them.