Worked Examples

Hattie used ONE meta-analysis here where d = 0.57 (Hattie's Rank = 30)

Crissman, J., (2006) The Design and Utilization of Effective Worked Examples, A PhD Dissertation, University of Nebraska.

There is much inconsistency in how Hattie selects his studies based on their quality. He included this PhD but he has rejected other similar studies giving these reasons,

"mainly based on doctoral dissertations, ..., with mostly attitudinal outcomes, many were based on adult samples ... and some of the sample sizes were tiny" (VL, p. 196).

The subjects in this PhD were mostly college students or adult students and sample sizes were small, often as little as 5 students.

An effect size of d = 0.57 is high in Hattie's frame of reference - more than a year's progress!

I am not arguing against the use of Work Examples, as it is probably the most widely used technique in Maths text books, online demonstrations and by teachers in the classroom.

The issue I and many others have, is the use of the effect size statistic to rank or to determine 'what works best'. Although, Hattie is now retreating from his claims of 'what works best' to a totally different notion, promoting the complexity of teaching - 'the story, the story, the story...'

From the point of view of 'the story...' narrative, this study is worthwhile reading, as it goes into a range of different ways in which teachers can use worked examples.

UPDATE:

Another example of the issues with Hattie's original claims of 'what works best' based on the effect size statistic. The Victorian Education Dept released its 10 High Impact Teaching Strategies (HITs) in 2017 to around 50,000 teachers. Worked Examples is one of the HITs based on the effect size = 0.57 (published in Hattie's book in 2009).

However, Corwin, the commercial arm of Hattie's Visible Learning, has updated (early 2018) Worked Examples by adding:

Wittwer & Renkl (2010) How Effective are Instructional Explanations in Example-Based Learning? A Meta-Analytic Review. They calculated an effect size of d=0.16.

They now publish the average effect size as d=0.37. Which is now below Hattie's "zone of desired effects" of 0.40.

So Worked Examples, if only using effect size, would now NOT be a High Impact strategy.

This illustrates the averaging problem discussed in detail in Effect Size.

Wittwer & Renkl (2010) also detail the complexity of this study and give more detail as to why they may have attained a low effect size,

"The worked example effect within cognitive load theory is a very well established finding. The concrete effectiveness of worked examples in a learning situation, however, heavily depends on further moderating factors. For example, if learners improve their processing of worked examples by actively explaining the worked examples to themselves, they are usually better able to solve transfer problems. Another way to enhance example processing is to present learners with instructional explanations instead of prompting them to produce these explanations on their own" (p. 393).

Wittwer & Renkl (2010) also challenge one of Hattie's cornerstones,

"In conclusion, the significant but small overall effect of instructional explanations challenges the notion that more direct instruction always results in more learning (cf.Koedinger and Aleven 2007). In other words, maximizing the instructional guidance in example-based learning through the provision of instructional explanations might not be beneficial in any circumstance. Instead, it is plausible to assume that the optimal design of instruction in example-based learning depends on specific learning goals" (p. 407).

It appears that neither the commercial arm of Visible Learning, Corwin, nor the Victorian Education Dept in the publishing of HITs are aware of this story.

Meta-analysis in education:

"I think you’ll find it’s a bit more complicated than that" (Goldacre, 2008).

Greg Ashman and Michale Pershan show this is, in fact the case. Ashman unpacks "worked examples" in many of his blogs - e.g., here, he states,

"We now can add a more basic level of research from the field of cognitive science or educational psychology. Cognitive load theory is the area in which I am conducting research for my PhD and many of its experimental findings suggest the advantages of teaching in ways similar to those outlined by Rosenshine. One of the earliest discoveries was that novices learn more by studying maths worked examples than by solving equivalent problems, an effect that reverses once they gain more expertise. This fits with a model of the mind being composed of an extremely limited working memory through which all new academic learning must pass, coupled with an effectively limitless long-term memory."

Michael Pershan also expands on "worked examples," in his useful blog - There’s nothing passive about worked example activities.