High Impact Strategies

The Victorian Education Dept has developed an edifice, The Victorian Teaching & Learning Model (VTLM), around what it claims are the 10-High Impact Teaching Strategies (The HITs):
Goal setting, Feedback, Worked Examples, Multiple Exposures, Explicit Teaching, Questioning, Meta-Cognitive Strategies, Structuring Lessons, Collaborative Learning and Differentiated Teaching.
Gill Callister, the then Head of the Department of Education and Training in Victoria, promoted the 10 HITs saying,
"I wanted to embed across all areas of the Department a persistent sense of curiosity, an overwhelming desire to analyse, evaluate and seek continuous improvement. 
After all, if we're spending tax-payer dollars on policies, programs and initiatives that have the potential to make or break the future of young people in our state, the very least we can do is evaluate what's working ..." (Australian Teacher Magazine, Oct 2018, p. 22).
A "persistent sense of curiosity, an overwhelming desire to analyse, evaluate..."

A collaboration with John Hattie & the Effect Sizes from his 2009 book Visible Learning (VL) is quoted as the prime evidence base, to justify & assert that these are the 10-HITs.

However, somewhat contradictorily, Hattie in the preface to that 2009 book states,
"It is not a book about classroom life, and does not speak to the nuances and details of what happens within classrooms."
Then surprisingly, in Hattie & Larsen (2020), Hattie stated,
"not once have we told a teacher 'how' to teach." (p. 55)

So let us put Hattie's evidence base under the microscope of Callister's paradigm of curiosity, analysis & evaluation:

Firstly, the notion of "High Impact" is based on Hattie's claim that the higher the effect size the higher the impact. 

The Victorian Education Dept has uncritically accepted Hattie's assertion, without reference to the large body of evidence challenging this claim - see Effect Size. 

Secondly, the definitions of each HIT, as defined by the Ed Dept, differ greatly from the definitions of the prime studies that Hattie used, e.g., The Ed Dept defines Feedback as,
"Feedback informs a student and/or the teacher about the student’s performance relative to the learning goals. Effective feedback will redirect or refocus teacher and student actions, so the student can align their effort and activity with a clear outcome that leads to achieving a learning goal."
This definition is not consistent with the research that Hattie cites, e.g., many of the studies provide money or sweets as rewards to students, and others have music playing in the background as feedback.

There has been significant peer review critique of Hattie's collection of feedback studies showing they are not relevant to School-aged students. Hattie has recently removed most of the original 23 studies he cited - see Feedback Revisited for more detail.

Similar problems have been raised about Hattie's use of his category "Self Report Grades" for Goal Setting - here, and Meta-Cognition - here.

Professional Judgment

The Ed Dept does caveat the importance of professional judgment. However, significant accountability measures were put in place to ensure compliance from schools, principals and teachers on the HITs. These measures are in the form of teacher annual reviews & school performance reviews from regional managers.

Also, job applications & promotions require compliance with the Hits and evidence of using them is necessary.

As a result, the business adage - "what gets measured, gets managed" dominates every School's performance measures and predictably, teacher professional judgment is diminished.

The NSW DOE also promotes Hattie's work without reference to the significant peer review critique. They quote a number of influences and borrow Hattie's meme "What Works Best" strategies.

The Queensland DOE has also promoted Hattie and taken it one step further by linking School Improvement Funding to Hattie's work.


The Qld DOE has also blocked my critical blog of Visible Learning.

Marzano

In the early 2000s, the 9 strategies from Marzano's book, "Classroom Instruction that Works", dominated Victorian Education. But, in his article, Setting the Record Straight on "High-Yield" Strategies, Marzano (2009) discusses the huge limitations of Educational Research, and warns,
"educators are making at least three mistakes when using the lists of strategies presented in our books (and other books like them). Left unchecked, these mistakes can impede the development of effective teaching in classrooms across the country." (p. 30-31)
The 3 mistakes Marzano lists are:

1. Focusing on a narrow range of strategies.
2. Assuming that High Yield strategies Must be used in every class.
3. Assuming that High Yield strategies will always work.

Marzano concluded,
"Checklist approaches to providing feedback to teachers probably don’t enhance pedagogical expertise, particularly when they focus on a narrow list of instructional, management, or assessment strategies. In fact, such practice is antithetical to true reflective practice. As City and colleagues note, such behavior is profoundly anti-professional." (p. 37)
Academics Raise Doubt about the HITs

Tytler & Prain (2017), in Simplistic advice for teachers on how to teach won’t work state,
"These strategies seem plausible, but we have concerns about their narrow view of teacher practice, their unconvincing “scientific” evidence base, and their limited view of the curriculum, teacher and student roles, and the capabilities required of students this century."
Armstrong & Armstrong (2021) in their book, Educational Trends Exposed: How to be a Critical Consumer, rate HITs as "Approach With Caution".

Their rating of various trends (p. 7)


Wescott (2021) also raises some concerns,
"The new Victorian reforms, such as the common model of instruction, and 'high impact teaching strategies', introduced under a suite of new practice initiatives, seemed to demand increasing visibility, measurability and conformity, seemed to be an official cementation of an approach to teaching that many of my colleagues and I remained deeply sceptical about." (p. 119)

Glenn Savage (2021b), in an Interview with Greg Ashman, develops the argument about,

The dangers of Standards & Alignment (@17min),

The danger of top-down pedagogy like HITs (it's not working (@20min),

The need to look at classrooms as evidence creators (@26min),

Questions the evidence behind HITs (@35min), 

Gives an example of a School where HITS did not work (@36min), 

Says teachers are cogs in the machine (@43min).

Then Savage concludes, 
"we need to move beyond what I think is a damaging view, that sameness and commonality is a path to improvement. The grand design of the Education revolution, it's sense to want to homogenise and to see that as the panacea, it hasn't worked... we would have seen 15 years later, a turn around in the key indicators, but things to a large extent have gone in the other direction." (@44min)

The Evidence-Based Agenda

A number of academics comment on the problem of this "evidence-based" agenda dominating teacher professional experience/judgment and call for a reclaim in the value of these - Ladwig (2019), Qvortrup (2019), Eacott (2017), Savage (2021b) and McKnight & Whitburn (2018).

The highly respected European Academic, Gert Biesta, also writes in detail about this problem - see Teacher Agency
"This becomes deeply problematic in those cases in which it is argued that professionals should only be allowed to do those things for which there is positive research evidence available - an approach which Holmes et al. (2006) have, in my view, correctly identified as a form of totalitarianism" (2010, p. 492).
What About Other Research?

The dominance of HITs has overshadowed other important research.

E.g., in 2018, I was introduced to "Cognitive Load Theory". Dylan Wiliam stated, 'This is the single most important thing for teachers to know' yet the Victorian DOE has not promoted this as compared to the HITs.

Contradictions

The major evidence-based organisations (EEF, WWC, Marzano, etc.) list different strategies for teachers to focus on and they often contradict each other - see here.

The Victorian Education Department is now promoting "Amplify" which they define as, "Empowering students through voice, agency and leadership." Yet, according to Hattie, this has a low effect size - see "Student Control".

Educational Research is Constantly Changing

For the last 15 years, Hattie used 1 PhD study for Worked Examples (effect size of d = 0.57). In early 2018, he added a 2nd study which reported a low effect size of d = 0.16. The combination of the 2 studies now gives a much lower average of d = 0.37 which is below Hattie's threshold of 0.40.

So Worked Examples is not a HITs anymore!

Another example is, Hattie in an interview in 2019 now saying the "jigsaw" method is the most effective teaching method. Yet, this did not appear in his 2009 book or his 2012 update, or the HITs!

Teacher's Passion

In Hattie's 2012 update to VL he states, 

"Throughout Visible Learning, I constantly came across the importance of ‘passion’; as a measurement person, it bothered me that it was a difficult notion to measure – particularly when it was often so obvious" (preface).
Passion is not included in Hattie's list of influences, yet he raises it as one of the most important influences!

It was refreshing to read some peer reviews that address these issues-

Nielsen & Klitmøller (2017),
"Hattie's synthesis is problematic because it gives the impression that he has all there is to say about a particular educational phenomenon but, we try to show this is far from the case" (p. 10).
Professor Dylan Wiliam explains the problem in 'Inside the Black Box' (2001),
"Teachers will not take up attractive sounding ideas, albeit based on extensive research, if these are presented as general principles which leave entirely to them the task of translating them into everyday practice - their classroom lives are too busy and too fragile for this to be possible for all but an outstanding few. What they need is a variety of living examples of implementation, by teachers with whom they can identify and from whom they can both derive conviction and confidence that they can do better, and see concrete examples of what doing better means in practice" (p. 10).
Then commenting on research again, Wiliam says,
"despite the many and varied reports of successful innovations, they fail to give clear accounts on one or other of the important details, for example about the actual classroom methods used, or about the motivation and experience of the teachers, or about the nature of the tests used as measures of success, or about the outlooks and expectations of the pupils involved" (p. 12).
In the Scandinavian context Nilholm (2017) reports,
"...he [Hattie] does not give reasonable answers to how his theses are to be translated into practical work and I have definitely not seen any study that critically examined what happens when municipalities and schools try to base their work on Hattie's work" (p. 3).


No comments:

Post a Comment