What is artificial intelligence explainability?

A working definition from MIT Sloan

artificial intelligence explainability (noun)

A quality that enables users of artificial intelligence programs to understand and trust how models operate and make decisions.

Creating successful artificial intelligence programs doesn’t end with building the right system. Stakeholders must have confidence that the programs are accurate and trustworthy.

According to research from Ida Someh, Barbara Wixom, and Cynthia Beath of the MIT Sloan Center for Information Systems Research, artificial intelligence explainability helps by ensuring users that models are “value-generating, compliant, representative, and reliable.”

There are several reasons stakeholders hesitate to trust AI. Because AI is relatively new, there isn’t an extensive list of proven use cases. Models are often opaque — AI relies on complex math and statistics, so it can be hard for average users to tell how a model works, whether it is producing accurate results, and if it is ethical and compliant.

Models can produce biased results if trained on biased data, and they also “drift” over time, meaning they can start producing inaccurate results as the world changes or incorrect data is included and replicated.

AI explainability is an emerging field, and teams working on AI projects are mostly creating the playbook as they go, the researchers write. Organizations can start by identifying units and organizations that are already creating effective AI explanations, continuing to test the most promising practices, and institutionalizing the best ones.

Why companies need artificial intelligence explainability

Working Definitions: Artificial Intelligence

MIT Sloan's Working Definitions explore the words and phrases behind emerging management ideas.

A person in business attire holding a maestro baton orchestrating data imagery in the background

Leading the AI-Driven Organization

In person at MIT Sloan

Load More