They liked the coffee; now what?

Several years ago I went to a conference for training professionals and got a chance to attend a session with Donald Kirkpatrick.  A well-known expert in training evaluation, he developed a widely-used model that includes four levels of evaluation:

1. Reaction of the student – what did they think and feel?
2. Learning – what skills or capability did the student gain?
3. Behavior – how has the student used what they learned?
4. Results – what effect has the student’s performance had on work outcomes or the business environment?

Any training professional will tell you that it’s very tempting to stop with a level one survey that shows whether the participants enjoyed the session, whether the materials were accurate and easy to use, and whether they liked the coffee.  It’s much harder to assess whether the training had any lasting impact on the way we do business.

In the same way, assessing a change effort needs to go beyond stakeholders’ initial thoughts and feelings.  The real question is how the change contributed to achieving the organization’s goals.  This requires much more thought and effort.

One of Dr. Kirkpatrick’s key points is that evaluation needs to be planned up front, and this is as true for change efforts as it is for training activities.  If the first questions is “what do we want to accomplish with this change?”  then it should be followed by “and how will we know that we’ve accomplished it?”  Conversations should occur early in the process about what sort of data needs to be collected, who needs to be consulted, and what sort of reporting will be done.

I’ve been part of several teams that have neglected this step.  When we reached the end of the project, it was too late to go back and get baseline data or mid-term status checks that would have informed evaluation reports.  On the other hand, asking evaluation questions at the beginning can help guide a team’s communication plans and implementation activities.

What methods have you used to build evaluation into the start of a change process?

Dee Anne Bonebright