How to accelerate AI transformation

As organizations deploy artificial intelligence more widely, routine tasks are being automated, and some work is shifting from people to AI systems. Yet, many firms are finding that the returns on their investments in such systems are elusive. 

The problem is not adoption of the technology alone but how organizations adapt the way they work to use it, according to MIT Sloan School of Management visiting senior lecturer Paul McDonagh-Smith. 

“There is an interdependency between three elements: the work itself, the workforce, and the workplace,” he said. “If organizations fail to align these three elements, it becomes difficult to generate measurable impact within a reasonable time frame.”

McDonagh-Smith teaches MIT Sloan’s new executive education course AI Essentials: Accelerating Impactful Adoption, which is designed to help leaders translate AI into impact by understanding how to redesign work and build the metrics and playbooks needed for adoption.

In a recent interview, McDonagh-Smith shared some ways organizations can close the gap between AI’s potential and its real-world impact. 

View AI as an operating system, not a toolkit 

Many firms still treat AI as something to plug into existing workflows, often to boost efficiency. But this approach is too narrow.

“Too many organizations are thinking of AI as a toolkit,” McDonagh-Smith said. “They are not seeing AI as an operating system.” 

This distinction matters. When AI is treated as a tool, it is layered onto existing processes and measured using outdated metrics. That makes it difficult to gauge its impact, even when value is being created. The result is that AI is deployed in fragments rather than as part of a coherent system. 

Adopt a mindset of exploration and evolution

Understanding models still matters, but it is not enough on its own.

McDonagh-Smith described how AI has developed in stages, with each step addressing a limitation of the last. Early systems relied on fixed rules written by humans. These were replaced by models that learn from data — known as machine learning models. More recent generative AI systems can work with language, images, and other complex information.

Each advancement solves a problem, so understanding the limits of today’s systems — such as their reliance on vast amounts of data and computing power — helps organizations see what may come next.

But organizations need to go beyond technical knowledge, McDonagh-Smith said. “We need to move from models to a mindset of exploration and evolution.” 

That means using AI in practice — testing it on real problems and seeing what works. 

Rethink how work is done

Adopting AI has implications for how work is structured. 

Jobs are no longer the right unit of analysis for understanding work, McDonagh-Smith said. “Gone are the days when you can think of a job role as a unit of measurement,” he said. Instead, “you need to break a job down into its constituent parts. It probably has 15 to 20 core activities.”

AI changes work task by task — automating some, augmenting others as work is divided between people and systems.

That means organizations need to redesign workflows around tasks, rather than layering AI onto existing roles. Managers need to map how work is actually done and define how tasks are split between people and AI.

AI Essentials: Accelerating Impactful Adoption

Attend Online

Register Now

Adopt new performance metrics 

Traditional performance metrics often fail to capture the value of AI.

“The metrics of value and the units of measurement that we’ve inherited … aren’t necessarily going to be the same moving forward,” McDonagh-Smith said. Instead, organizations need new ways to benchmark performance, including decision speed, human-AI collaboration, and how quickly insights are fed back into the business.

Other measures focus on outcomes: improvements in decision quality, the degree of autonomy in AI systems, and how well those systems “understand” the organization.

A starting point is to define a small set of AI-specific metrics within key workflows and use them to test where value is being created.

Close the “last mile” gap

The central challenge in AI adoption comes during what McDonagh-Smith calls the “last mile” — the point where AI systems fail to translate into business value. “AI last-mile engineering is fundamentally about reducing the space between AI’s potentiality and its real-world impact,” he said. 

In many organizations, AI initiatives stall because although the model works, it is not used in everyday decisions and workflows.

To close the gap, firms need to take a structured approach: 

• Start with identifying a problem to be solved. “There’s a common denominator across organizations: the need to clearly define the problems they are trying to solve with AI,” McDonagh-Smith said.
• Involve users in design. Systems should be built with the people who use them, not be imposed on them.
• Focus on context. Prioritize how work is actually done, rather than relying on computing power alone.
• Test and scale. Progress comes from testing small applications, measuring results, and scaling what works. “Ship small, learn fast,” McDonagh-Smith said.
• Build trust with users. Trust should be built in from the start, with governance and continuous oversight.

Watch the webinar: ‘From Models to Mindset, The Last Mile of AI Adoption’  

Explore the course — AI Essentials: Accelerating Impactful Adoption

Paul McDonagh-Smith is a visiting senior lecturer in information technology at MIT Sloan. In his research and teaching, he creates key intersection points between technology and business. He specializes in translating computer and data science into measurable business value that evolves organizational capability, transformation, and strategy.