What It Takes to Move From AI Hype to AI That Delivers

For years, AI lived in the land of promise: bold claims, flashy demos, endless talk about potential. In 2026, the question has shifted from what AI could do someday to what it delivers right now, and how you’d measure it.

That’s the thread Acorn President Keith Metcalfe pulled on with Jim Kunkle on The Digital Revolution Podcast. AI didn’t create the measurement problem most companies have, Keith argues. It just made it impossible to ignore.

Watch the episode below or read on for Keith and Jim’s insights.

Why are most organizations still stuck experimenting with AI?

In short: because they’re rolling out a powerful tool into roles that were never clearly defined in the first place.

Keith points to Acorn’s 2026 State of Learning for AI Fluency Report, which breaks the problem into three parts that stack.

1. First, broken development plans: 58% of organizations say their plans are only somewhat effective or worse, because they aren’t specific to the role, consistent, talked about regularly, or tied to anyone’s career trajectory.
2. Second, AI adoption without infrastructure. As Keith put it, if you haven’t sorted the development-plan piece, “that new tool doesn’t have a place to be relevant in the role.”
3. Third, an assessment void: there’s no agreed way to say whether AI is working or not.

“I don’t actually understand how organizations are even making these decisions. If you don’t have that fundamental tie of AI to role and then expectations within it, you don’t have any data to tell you whether or not people are being effective with it.”

Throw AI at a workforce that doesn’t know what good looks like in its own roles, and you get people acting busy without evidence of productivity.

What does the AI perception gap actually look like?

It’s the distance between what the C-suite believes is happening and what the people doing the work experience.

The same program looks completely different depending on where you sit:

• 78% of executives say AI skills expectations are clearly communicated at the role level. That figure roughly halves at the manager level and falls to 19% among individual contributors.
• 82% of executives are excited about AI. Among individual contributors, roughly 28% are scared or disillusioned, and another 54% have effectively checked out.

The further you sit from the work, the more confident you are that everything is going to plan.

How do you make AI fluency measurable by role?

Start by defining the job in a shared language leaders and learners can both understand.

But don’t get caught up in a more-is-more mentality. Here’s the catch Keith named: jobs carry between 11 and 20 skills on average. Nobody sits down every quarter and works through skill number 1, then skill number 2, all the way up to 20 potential skills. That’s not how employees think about their jobs, and not how managers want to spend their limited time in performance conversations.

Acorn groups those skills into four or five capabilities, about as much as anyone can spend meaningful time on in a real conversation, then defines what good-to-great looks like for each one, including AI use itself.

That’s the part a simple skills tag in your LMS can’t do. That might highlight that someone touched a tool; defining the capability sets the bar for using it well. With AI use written into the role as one of those capabilities, the assessment void closes, and a leader can finally say whether someone is getting better with AI in their role instead of waving vaguely at “use AI more.”

“It’s fine to say, you know, as a podcaster, you should use AI more, Jim. Okay, well, what do you do with it? What does success look like?”

Is AI replacing people or amplifying them?

Amplifying, and the organizations that can prove it are the ones that did the role-level work first.

Anyone who’s actually built skills or projects in a tool like Claude knows we aren’t at wholesale role replacement. The more interesting consequence is internal: AI is flattening the old hierarchy where a “brilliant jerk” held authority purely through accumulated experience, and leaders are being asked to lead with curiosity rather than certainty.

The worrying flip side is what happens without role-level definitions. If you can’t see how people are developing against the capabilities their role needs, how are you deciding who’s effective with AI, or making decisions about redundancies? Keith’s honest answer: “I don’t know, and I don’t think a lot of organizations do.”

Where do KPIs fit without breaking trust?

KPIs belong inside development plans, not bolted on top of them.

Drop a metric on someone who’s already scared or disengaged, with no context, and the conversation stops being about development. People start thinking about pay, the mortgage, their kids’ tuition. But attach a KPI to a role that’s already defined as four or five capabilities with clear standards, and the dynamic flips. It forces the leader to be more thoughtful about what they’re measuring, and it breaks what Keith called the chain of distrust.

The bottom line

AI doesn’t fix a broken measurement system. It exposes and amplifies it.

Whether AI delivers for you is really downstream of whether that system works, which is the thread the conversation kept pulling:

• The organizations pulling ahead aren’t the ones with the most AI enthusiasm at the top. They’re the ones that define their roles in four or five capabilities, set a clear bar for what good looks like, and build a way for leaders and learners to talk about it regularly.
• Without that infrastructure, you get adoption numbers that look healthy on a board deck and a workforce that can’t tell you whether the work actually improved.
• KPIs do their job when they sit inside a defined role, not when they’re dropped on a workforce that has no clear standard to be measured against.

Keith’s advice to leaders starting out is to keep it small: find one problem, treat AI as a tool rather than a person, and give every employee a development plan from day one with regular conversations to capture learnings and progression.

Connect with Keith Metcalfe and Jim Kunkle on LinkedIn for more conversations like this. Hear more of The Digital Revolution Podcast on Spotify and Apple.