Worked Examples

Hattie used ONE meta-analysis here where d = 0.57 (Hattie's Rank = 30)

Crissman, J., (2006) The Design and Utilization of Effective Worked Examples, A PhD Dissertation, University of Nebraska.

It is strange that Hattie included this PhD as he has rejected other similar studies, 
"mainly based on doctoral dissertations, ..., with mostly attitudinal outcomes, many were based on adult samples ... and some of the sample sizes were tiny" (VL, p. 196).
An effect size of d = 0.57 is high in Hattie's frame of reference - more than a year's progress!

This study used a more robust design that enabled effect size to be calculated using control and experimental groups - method 1 (see effect size). So it was actually measuring a change in student achievement - unlike Self-Report Grades and Piagetian Programs.

Also, the subjects were mostly college students. Using the benchmarks calculated for different age groups (see a year's progress ?) this could be interpreted as a much higher rank than Hattie's.

This is the technique used by most math's textbooks and the most popular YouTube math's channels.

This study is worthwhile reading as it goes into a range of different ways in which teachers can use worked examples.


The Victorian Education Dept released its 10 High Impact Teaching Strategies (HITs) in 2017 to around 50,000 teachers. Worked Examples, is one of the HITs.

However, Corwin, the commercial arm of Hattie's Visible Learning, has updated (early 2018) Worked Examples by adding:

Wittwer & Renkl (2010) How Effective are Instructional Explanations in Example-Based Learning? A Meta-Analytic Review. They calculated an effect size of d=0.16.

They now publish the average effect size as d=0.37. Which is now below Hattie's "zone of desired effects" of 0.40.

So worked examples, if only using effect size, would now NOT be a high impact strategy.

This illustrates the averaging problem discussed in detail in Effect Size.

Also, ironically, this illustrates more inconsistency with Hattie's new mantra 'the story, the story, the story ...' As Wittwer & Renkl (2010) give more detail as to why they may have attained a low effect size,
'The worked example effect within cognitive load theory is a very well established finding. The concrete effectiveness of worked examples in a learning situation, however, heavily depends on further moderating factors. For example, if learners improve their processing of worked examples by actively explaining the worked examples to themselves, they are usually better able to solve transfer problems. Another way to enhance example processing is to present learners with instructional explanations instead of prompting them to produce these explanations on their own' (p. 393).
Wittwer & Renkl (2010) also challenge one of Hattie's cornerstones,
'In conclusion, the significant but small overall effect of instructional explanations challenges the notion that more direct instruction always results in more learning (cf.Koedinger and Aleven 2007). In other words, maximizing the instructional guidance in example-based learning through the provision of instructional explanations might not be beneficial in any circumstance. Instead, it is plausible to assume that the optimal design of instruction in example-based learning depends on specific learning goals' (p. 407).
It appears that neither the commercial arm of Visible Learning, Corwin, nor the Victorian Education Dept in the publishing of HITs is aware of this story.

Meta-analysis in education:
"I think you’ll find it’s a bit more complicated than that" (Goldacre, 2008).
Greg Ashman and Michale Pershan show this is, in fact, the case:

Greg Ashman unpacks "worked examples" in many of his blogs - e.g., here, he states,
"We now can add a more basic level of research from the field of cognitive science or educational psychology. Cognitive load theory is the area in which I am conducting research for my PhD and many of its experimental findings suggest the advantages of teaching in ways similar to those outlined by Rosenshine. One of the earliest discoveries was that novices learn more by studying maths worked examples than by solving equivalent problems, an effect that reverses once they gain more expertise. This fits with a model of the mind being composed of an extremely limited working memory through which all new academic learning must pass, coupled with an effectively limitless long-term memory."
Michael Pershan also expands on "worked examples," in his useful blog - There’s nothing passive about worked example activities. 

No comments:

Post a comment