What are power shadows?
A working definition from MIT Sloan
power shadows (noun)
Biases or systemic exclusion of a society, as reflected in data sets.
As a graduate student working on facial analysis software, Joy Buolamwini, SM ’17, PhD ’22, encountered a problem: The software did not detect her dark skin, though it easily identified the faces of people with lighter skin. She had to wear a white mask to be detected by the computer.
The problem was the power shadows that are cast in datasets, Buolamwini, a computer scientist, writes in her book “Unmasking AI.” The dataset she used — created by a government agency with the intent of representing a diverse population — featured images of public figures, often elected officials. Around the world, white men have historically held political power, and that was reflected in the data. The software she used was trained on images in which people were predominantly white and male.
Power shadows have serious consequences. Machine learning models used to inform hiring decisions could reflect past biased decisions made by humans. Models used to diagnose medical conditions could be based on data from mostly white patients, resulting in poor outcomes for Black patients.
The first step to overcoming power shadows is being aware of them, according to Buolamwini. “We must also be intentional in our approach to developing technology that relies on data,” she writes.
Working Definitions: Artificial Intelligence
MIT Sloan's Working Definitions explore the words and phrases behind emerging management ideas.
AI Executive Academy
In person at MIT Sloan
Register Now
How generative AI ‘persuasion bombs’ users — and how to fight back
When professionals try to validate outputs, generative AI often responds not with corrections or candor but with escalating persuasion tactics.
‘AI helped me turn lab technology into an at-home product for parents’
Learn how Sensopore founder Max Jara Fornerod identified a customer base for his startup using the MIT Entrepreneurship JetPack digital adviser.
How AI is reshaping workflows and redefining jobs
New research shows that AI delivers the most value when organizations redesign workflows, not just when they automate individual tasks.
How to use ChatGPT to plan your retirement
MIT Sloan professor Andrew Lo says Al is good at explaining trade-offs and exploring scenarios but weak at precise tax optimization, math, and regulatory compliance.
How to accelerate AI transformation
A new MIT Sloan executive education course looks at how organizations need to align “the work, the workforce, and the workplace” to succeed with artificial intelligence.
Putting AI to work: The latest from MIT Sloan Management Review
New MIT Sloan Management Review insights cover types of AI startups, using agentic AI tools for knowledge work, and why AI isn't boosting productivity.
The AI developments finance pros should be tracking
A new MIT Sloan executive education course led by professor Andrew W. Lo explores machine reasoning, quantamental investing, AI governance, and more.
The Fed, AI, and economic uncertainty: What investors need to know
MIT Sloan’s Gary Gensler and Peter R. Fisher advise investors to develop an AI investment thesis and avoid overconfident investing during policy pivots.
3 ways to use AI: Are you a cyborg, a centaur, or a self-automator?
A study of consultants found that employees use generative AI three different ways. Each has different implications for on-the-job learning.
How algorithmic data deserts exclude consumers
As AI systems shape more decisions, some individuals and businesses are left out entirely. New research highlights how data gaps create hidden risks for organizations.