What are power shadows?
A working definition from MIT Sloan
power shadows (noun)
Biases or systemic exclusion of a society, as reflected in data sets.
As a graduate student working on facial analysis software, Joy Buolamwini, SM ’17, PhD ’22, encountered a problem: The software did not detect her dark skin, though it easily identified the faces of people with lighter skin. She had to wear a white mask to be detected by the computer.
The problem was the power shadows that are cast in datasets, Buolamwini, a computer scientist, writes in her book “Unmasking AI.” The dataset she used — created by a government agency with the intent of representing a diverse population — featured images of public figures, often elected officials. Around the world, white men have historically held political power, and that was reflected in the data. The software she used was trained on images in which people were predominantly white and male.
Power shadows have serious consequences. Machine learning models used to inform hiring decisions could reflect past biased decisions made by humans. Models used to diagnose medical conditions could be based on data from mostly white patients, resulting in poor outcomes for Black patients.
The first step to overcoming power shadows is being aware of them, according to Buolamwini. “We must also be intentional in our approach to developing technology that relies on data,” she writes.
Working Definitions: Artificial Intelligence
MIT Sloan's Working Definitions explore the words and phrases behind emerging management ideas.
AI Executive Academy
In person at MIT Sloan
Register Now
Seeing real value from AI depends on being able to verify its outputs
A new paper explores how seeing economic value from artificial intelligence hinges on closing the gap between what AI can do and how humans can verify its outputs.
‘AI gravity’ is pulling you toward dependency. Here’s how to push back
AI systems hold the promise of competitive advantage, but they can usher in cognitive decline among workers, says MIT Sloan School of Management’s Eric So. Learn how to protect cognitive capital.
Heeding the pope’s call to ensure AI protects human dignity
Following Pope Leo XIV’s encyclical on AI, MIT Sloan professor emeritus Thomas A. Kochan argues that firms should partner with workers to ensure AI augments human skills and knowledge.
What senior leaders want to know about AI
Leaders are turning to MIT Sloan Executive Education to learn more about AI, including managing humans amid technological change and rethinking their relationships with IT departments.
What leaders still get wrong about AI
Organizations are struggling to succeed with AI. Research from the MIT Center for Information Systems Research shows common mistakes and how to overcome them.
How ‘learnrights’ would compensate creators for AI model training
Learnright laws would give copyright holders the exclusive right to license their content for artificial intelligence model training.
3 types of leaders that drive digital innovation
Organizations that succeed with digital innovation engage three types of leaders: initiative leaders, shared resource leaders, and portfolio leaders.
How generative AI ‘persuasion bombs’ users — and how to fight back
When professionals try to validate outputs, generative AI often responds not with corrections or candor but with escalating persuasion tactics.
‘AI helped me turn lab technology into an at-home product for parents’
Learn how Sensopore founder Max Jara Fornerod identified a customer base for his startup using the MIT Entrepreneurship JetPack digital adviser.
How AI is reshaping workflows and redefining jobs
New research shows that AI delivers the most value when organizations redesign workflows, not just when they automate individual tasks.