Thalheimer’s Performance Focused Smile Sheets

Freakin’ revolutionary ideas for learning and performance professionals

 

By Mirjam Neelen

My decision to read Will Thalheimer’s book Performance-Focused Smile Sheets was one out of hope. Measuring if learning and performance solutions are effective in the workplace isn’t easy. There’s a lot of conversation around the topic, varying from xAPI to formal assessments to “professionals know what to do to get better so just let them” to smile sheets to… you get the point.  Nothing has been really convincing as far as I’m concerned. But then again, in general I’m not very easy to enthuse.

Although… not once when I was reading Will Thalheimer’s book did I raise my eyebrows. It’s boringly good. It’s needed. It’s practical.

First, what’s the problem Thalheimer’s book is addressing? The problem is that smile sheets, or happy sheets, are commonly used as a learning evaluation tool in the workplace. And that’s an issue because, as Thalheimer subtly expresses in his book (take a deep breath): Smile sheets suck! They tell us less than nothing. We pretend they tell us something about learning results, but they don’t. Even worse, they misinform us about what improvements need to be made (Like better coffee! Don’t get me wrong, better coffee is always good. However, as far as I know, it hasn’t been correlated with learners’ achievement). In short, they hurt organisations because they don’t do what they pretend to do – yet they’re used all the time.

Usually, smile sheets measure a learner’s perception of the value of the learning experience that they went through. The main problem with this is that learners do not know best (see Kirschner & Van Merriënboer, 2013) and therefore all you’re measuring are either the learners’ likings or their judgements of learning (JOLs) which are, more often than not misinformed[1]. The other problem with smile sheets is that we claim that we use them to improve what we’re doing in the learning space. However, the smile sheets that are used in today’s workplace usually don’t ask the questions that are needed to find out what we need to fix. If we want our learning/performance interventions to be effective and have an impact then we need to measure them the right way (oh, and just a detail, you need to design effective learning/performance interventions as well).

One of the strengths of Thalheimer’s book is that he keeps it real; he doesn’t pretend that his revamped smile sheets are the solution to the bigger problem that L&D is currently facing, namely proving its value. He whole-heartedly admits that smile sheets should never be the only evaluation tool.

“They must be augmented with outcome measures that get at learner understanding, learner remembering, and learner application (p.31).”

What I’m personally delighted with is that Will doesn’t shy away from using the word training. Nowadays it’s almost ‘not done’ to use the word in a workplace learning context. Marc Rosenberg even tried to convince us that we shouldn’t call employees learners. For me, although I do understand where it’s coming from it’s missing the real point, namely that many L&D departments have implemented ineffective learning interventions. The solutions that are thought up are often disjointed and their design and/or development often lack quality and are therefore ineffective. There’s usually no, what I call integrated learning approach where learning is understood as a journey with all the (formal and informal) pieces needed to draw the full picture; which will only work if they’re implemented together. Thalheimer brings up this challenge implicitly by explaining the levels of learning in an organisation and the four pillars of learning effectiveness[2]. Here he’s hitting the nail on the head when he says that the focus in organisations is quite often on awareness and increased knowledge and not on, what he calls performance training or performance training with performance assistance[3].

What are the main differences between original Likert-based smile sheets and Thalheimer’s? First, according to Thalheimer, smile sheets need to do two major things. They need to predict training effectiveness and they need to produce actionable results (after all, we want to be able to improve). Likert scales, used in traditional smile sheets do neither. They’re unrelated to learning results, the subjective responses aren’t carefully analysed, they’re “transmogrified” (thanks, Will for teaching me a new word) inappropriately, and they don’t produce clear guidance for action, to just name a few things that are wrong with them.

Thalheimer introduces a taxonomy to enable indexing the right (type of) questions. After all, you need to be crystal clear on what it is you want to measure. He then gives a ton of strong and weaker examples (and what’s so great about Will, he acknowledges whole-heartedly that the weaker ones are weaker). One thing needs to be said. Smile sheets will always give you subjective data. It will always be the individual smile sheeter’s interpretation of their situation. However, by creating them carefully, they become much more meaningful.

For example, instead of asking “On a scale of 1-5 how able are you to put what you’ve learned into practice on the job”, Thalheimer suggests five concrete answer options. For example, “I am able to work on actual job tasks but I’ll need more hands-on experience to be fully competent in using the concepts taught”.

Thalheimer’s delayed smile sheets are brilliant as well. So, let’s say a month after the learner attended a training, workshop, coaching session, and so forth (s)he receives a new smile sheet. The power of this type of smile sheet is that learners can now indicate the actual on-the-job practical application, and as such it gives them not only a chance to evaluate what this learning intervention actually meant for their job or performance, it also gives the “one in charge” (e.g., L&D) an opportunity to evaluate ‘the real deal’.

While I was reading, I thought of so many other ways you can use these smile sheets. You can use them for peer feedback, self-assessment, and recommender systems. You wouldn’t necessarily use Thalheimer’s taxonomy, but you could create your own, based on what you want to measure. For example [4], “I recommend this book/video/lunch ‘n learn because it enabled me to begin trying to use the new skills on the job” or “I recommend this peer/colleague/mentor because (s)he helped me put together an action plan that (1) determined what goal I wanted to accomplish, (2) decide what key situations I would target for each goal, and (3) come up with a plan of action for each of those key situation”. Instead of: I think this learning intervention/person is great – 5 stars.

Here’s my (biased) main smile sheet question to evaluate Will Thalheimer’s book:

What did Will Thalheimer’s Performance-Focused Smile Sheets enable to you to do?

Select all that apply.

1. I can design questions and statements to evaluate a broad spectrum of learning interventions and more.
2. I can apply the principles of both ‘immediately after’ and ‘delayed’ smile sheets in my evaluations.
3. I can design my own taxonomies for other types of evaluation (e.g., feedback or recommender systems).
4. All of the above.

The answer is D. As Will states himself: The ideas in his book are freakin’ revolutionary! J

 

References

Kirschner, P.A., & van Merriënboer, J.J.G.,  (2013) Do Learners Really Know Best? Urban Legends in Education, Educational Psychologist, 48, 169-183. Retrieved from http://www.tandfonline.com/doi/pdf/10.1080/00461520.2013.804395.

Rosenberg., M., (2015, August 11). Marc my Words: Don’t Call Them Learners [Web Log]. Retrieved from http://www.learningsolutionsmag.com/articles/1763/marc-my-words-dont-call-them-learners.

Thalheimer, W., (2016). Performance-Focused Smile Sheets. A Radical Rethinking of a Dangerous Art Form. USA: Work-Learning Press.2