How to measure experiential leadership development, and prove it worked

To measure experiential leadership development, define the business problem and target behaviors before the program, capture a baseline, then track whether leaders apply new skills on the job and whether related business indicators move. Research from the Association for Talent Development (ATD) and the Institute for Corporate Productivity (i4cp) found that well over half of high-performing organizations assess leaders’ application of new skills, compared with only about a third of lower performers.

Measurement came up again and again in the study. Across senior and frontline leaders, on-the-job learning and simulations, the act of measuring development was repeatedly identified as a next practice: rarely done, and strongly tied to both market performance and learning effectiveness. This article explains what the research found and offers a practical framework for executive teams.

Why does measurement matter so much?

The study’s findings on measurement point in one direction from both sides.

Not measuring is a warning sign. Twenty-two percent of respondents said they could not measure the effectiveness of experiential learning. That inability showed an extremely high negative correlation with market performance. Seventeen percent said they could not tie experiential learning to business results.

Measuring is a mark of high performers. For frontline leaders, about a third of high-performing organizations measured the development that resulted from both formal and informal on-the-job learning. None of the lower-performing organizations did. Among high performers using simulations, forty percent made measuring effectiveness a priority, and measuring simulation outcomes for frontline leaders showed the most robust correlations of any practice examined.

The study’s authors put it simply: disciplined measurement is characteristic of high-performance companies, because it drives both execution and continuous improvement. Their only surprise was how rarely organizations do it.

What do most organizations measure today?

When asked how they measure the effectiveness of experiential learning for leaders, respondents reported:

  • Learner satisfaction with the experience: fifty-four percent.
  • Learners’ application of new knowledge and skills: forty-eight percent.
  • Career advancement and promotions: forty-seven percent.
  • Managers’ assessment of how learners apply new skills: forty-seven percent.
  • Specific business indicators tied to specific experiences: thirty-five percent.
  • Completion of tasks within the experience, such as solving a business problem: thirty-five percent.
  • Data tracked automatically by simulation software: eleven percent.

Roughly one in five organizations did not measure experiential learning from simulations or from on-the-job experiences at all.

The most common measure, satisfaction, is also the least connected to results. A program can be well liked and still leave behavior unchanged. High performers move past satisfaction toward application and business impact.

A six-step framework for measuring experiential leadership development

The following framework is Morrison Alley’s approach. It builds on the study’s findings and on the practices of the organizations the study profiles, and each step names the finding it draws on. Recommendations that go beyond the research are labeled as ours. It works for a single executive offsite or a multi-month program.

Step one: define the business problem and the behaviors that will solve it

Measurement starts at design. The study’s authors noted that measurement must be considered at inception, shaped by the objectives for the program overall and for each activity.

Begin by naming the business challenge the program addresses, such as slow cross-functional decisions, a strategy that is not translating into execution, or a leadership team preparing for a major transition. Then define the specific leadership behaviors that would move it. Christopher Schuller, then senior manager of learning at Boeing, interviewed for the study, framed return on investment this way: “You have to think about the business problem you’re solving for the company and the impact it has on performance. That’s your ROI.”

Step two: capture a baseline

You cannot show change without a starting point. Useful baselines include a 360-degree assessment of the target behaviors, team effectiveness surveys, engagement data for the leaders’ teams, and current values for the business indicators tied to the problem.

Cisco’s Leaders of Tomorrow program, profiled in the study, used a 360-degree behavioral assessment at the beginning and end of the program, paired each assessment with a coaching debrief, and analyzed aggregated results to measure change across the group.

Step three: measure inside the experience

Experiential programs generate evidence as they run. Track the decisions teams make in a simulation, the quality of solutions produced in action learning, and the gaps or issues the experience surfaces. In the live simulation Schuller described, every issue that emerged during the day went onto a shared list, and by the end each had an owner and an action item. That list became the measurement instrument for everything that followed.

Step four: measure application on the job

This is the step that separates high performers. Well over half of top companies assess whether leaders apply new knowledge and skills, and more than half rely on observations from the leaders’ own superiors. Building on that finding, Morrison Alley recommends structured manager observations at thirty, sixty, and ninety days, follow-up pulse surveys from direct reports, and check-ins against the commitments leaders made during the program.

Step five: connect to business and talent indicators

Link development to the indicators the business already tracks. The organizations profiled in the study used a range of measures:

  • Hyatt tracked turnover, leadership and supervision indices in engagement surveys, and internal movement into key roles.
  • Cisco tracked career advancement and found that about two thirds of participants in its emerging leader program earned a promotion or role change while still in the program.
  • One high-performing organization described assessing changes in market share, customer satisfaction, innovation, and business growth alongside individual and strategic objectives.

For executive teams, Morrison Alley recommends tracking indicators such as decision cycle time, progress on strategic initiatives, cross-functional project delivery, retention of high performers, and succession readiness.

Step six: build in follow-through and keep measuring

Measurement and reinforcement work together. When the study asked organizations how they sustain learning after an experience, twenty-three of about eighty written responses amounted to “none.” The organizations that did better built in specific mechanisms: peer accountability partners, coaching from managers, alumni networks, and mentors.

Follow-through also produces measurement data. After his live simulation, Schuller’s team held monthly meetings for six months to review action items, track what had been done, and test whether the solutions were working. Those meetings reinforced the learning and documented the results at the same time.

How should executive teams think about ROI?

Return on investment is the question every program sponsor eventually asks. The research suggests two principles.

Measure value against the problem, not the headcount. In the study, Schuller cautioned against traditional cost-per-learner thinking. A program for ten executives can look expensive per person, but if it helps fix a flawed operating model that unlocks significant revenue, the return can be substantial. Scale matters for broad programs; impact matters for executive ones.

Expect the strongest returns where development and business work overlap. The study’s recommendations encourage dual-purpose action learning, where leaders develop by generating solutions to real business challenges and then implement them. When the program itself produces business outcomes, the ROI story writes itself.

Common measurement mistakes to avoid

  • Stopping at satisfaction. It is the most common measure and the weakest link to results.
  • Measuring only at the end of the event. Behavior change shows up weeks and months later.
  • Leaving managers out. Manager observation is one of the methods top companies rely on most.
  • Ignoring informal learning. Few organizations measure it, yet doing so is a next practice tied to performance.
  • Treating measurement as an afterthought. If success is not defined at the start, it cannot be shown at the end.

Key takeaways

  • Inability to measure experiential learning showed an extremely high negative correlation with market performance in ATD and i4cp research.
  • Well over half of high performers measure whether leaders apply new skills, compared with about a third of lower performers.
  • Satisfaction is the most common measure and the least connected to results.
  • Effective measurement starts at design, captures a baseline, tracks on-the-job application, and links to business indicators.
  • Follow-through mechanisms both reinforce learning and generate evidence of impact.

How do you measure the ROI of leadership development?

Define the business problem the program addresses, capture a baseline, then track whether leaders apply new behaviors and whether related business indicators improve. Compare that impact with the full cost, including leaders’ time.

What metrics should a leadership development program track?

Strong programs track applied behavior change through manager observation and 360-degree assessments, along with business and talent indicators such as retention, promotions, engagement, and progress on strategic initiatives.

What percentage of organizations measure leadership development effectiveness?

In the ATD and i4cp study, about one in five organizations did not measure experiential learning at all, and only fourteen to fifteen percent measured development from on-the-job experiences for frontline leaders.

Build measurement into your next program

Morrison Alley, based in Boulder, Colorado, designs custom experiential leadership and team development programs for senior executive teams, with success measures defined at the start and follow-through built into the design. To discuss how to build a measurable program for your leadership team, contact Morrison Alley.

Related field notes: business simulations for senior leaders, why there are no independent reviews for leadership training, and the role of a leadership development department.

Sources. Carol Morrison (i4cp), Experiential Learning for Leaders: Action Learning, On-the-Job Learning, Serious Games, and Simulations, Association for Talent Development and Institute for Corporate Productivity (Alexandria, VA: ATD Press, May 2016). Survey of two hundred seventy talent development professionals fielded January 2016. A public preview of the findings is also on the ATD research blog. Quotations are from interviews conducted for the report; titles reflect each speaker’s role at the time of the study. Recommendations that go beyond the research are labeled as Morrison Alley’s. Additional research on experiential learning and team performance is gathered on our Why It Works page.

Start a conversation

If you want a program whose results you can show your executive team, a sentence or two is enough to begin.

Get in touch →

All field notes

Morrison Alley Leadership Boulder, Colorado