What are power shadows?
A working definition from MIT Sloan
power shadows (noun)
Biases or systemic exclusion of a society, as reflected in data sets.
As a graduate student working on facial analysis software, Joy Buolamwini, SM ’17, PhD ’22, encountered a problem: The software did not detect her dark skin, though it easily identified the faces of people with lighter skin. She had to wear a white mask to be detected by the computer.
The problem was the power shadows that are cast in datasets, Buolamwini, a computer scientist, writes in her book “Unmasking AI.” The dataset she used — created by a government agency with the intent of representing a diverse population — featured images of public figures, often elected officials. Around the world, white men have historically held political power, and that was reflected in the data. The software she used was trained on images in which people were predominantly white and male.
Power shadows have serious consequences. Machine learning models used to inform hiring decisions could reflect past biased decisions made by humans. Models used to diagnose medical conditions could be based on data from mostly white patients, resulting in poor outcomes for Black patients.
The first step to overcoming power shadows is being aware of them, according to Buolamwini. “We must also be intentional in our approach to developing technology that relies on data,” she writes.
Working Definitions: Artificial Intelligence
MIT Sloan's Working Definitions explore the words and phrases behind emerging management ideas.
AI Executive Academy
In person at MIT Sloan
Register Now
The 5 forces creating urgency around ethical AI adoption
In his book “Boundaries of Tolerance,” MIT Sloan School of Management senior lecturer Jeffrey Saviano argues that ethical AI strategy and governance are the defining leadership challenges of our time.
Stop debating automation vs. augmentation
As organizations begin deploying agentic systems that can execute entire workflows, a consequential question is emerging: How can leaders turn agentic AI productivity gains into business growth?
How to level up systematic innovation with AI
Artificial intelligence can boost the systematic innovation of products, processes, and services. MIT Sloan professor Steven Eppinger offers four steps for making that happen.
AI and jobs: 4 takeaways from Google’s chief economist
Google’s Fabien Curto Millet examined how AI is reshaping job roles, workflows, and employee training in an address at the MIT Stone Center on Inequality and Shaping the Future of Work.
5 things to know about how companies can create value from AI
As AI tools become more widely available, competitive advantage will depend on the trust, data, expertise, operating capabilities, and work redesign companies build around them.
These new MIT Sloan courses track emerging technologies and business trends
MIT Sloan School of Management course offerings like AI-Augmented Work, Defense and Dual-Use Ventures, and Leading Agentic AI Transformation in Organizations anticipate businesses’ future needs.
‘The Collision: What AI Does to Us’
In his new book, MIT Sloan School of Management professor Eric So examines the ways AI is reshaping human cognition and offers advice on protecting your uniquely human skills while using AI.
4 seats AI can occupy at the negotiating table
AI can negotiate for you, coach you, support your team, or mediate between parties. An MIT Sloan School of Management professor explains those four roles — and why human judgment still matters.
Lessons from Kaiser Permanente’s AI labor deal
An MIT report details how Kaiser Permanente and the Alliance of Health Care Unions reached a landmark labor agreement to incorporate worker input into AI decision-making.
A framework for determining when AI can make decisions
The AI Decision Matrix uses ambiguity and risk to determine the optimal decision-making structure between humans and AI automation.