Looking at the historical academic data (see below), the school improved around 2008-2013 then plateaued and then decreased a little in 2015 (the year Hattie was there).
The show displayed many difficult and disruptive students. Also, apparent was the passion, enthusiasm, care, patience and hard work of the teachers. The staff also displayed a desire to improve their teaching, a key mindset, according to a different expert, Dylan Wiliam.
Ironically, the qualities like passion are the invisible influences that Hattie argues AGAINST in his book Visible Learning. But, in the 2012 update, Visible Learning for Teachers, he states,
"Throughout Visible Learning, I constantly came across the importance of 'passion'; as a measurement person, it bothered me that it was a difficult notion to measure – particularly when it was often so obvious" (preface).At the same time Vaillant (2012), takes this a step further,
"Perhaps the greatest problem faced by the academic social sciences is that what is measurable is often irrelevant, and what is truly relevant often cannot be measured."I am concerned that Hattie seems to be given the credit for the improvement from 2008-2013, yet Hattie was only there in 2015, when the Year 12 and NAPLAN results decreased!
What concerns me more is Hattie’s financial conflict of interest: he promotes his own software, e‑Astle and the Visible Classroom app, as fixes to the ‘problems’ he claims exist in public education. O’Neill (2016) documents this conflict clearly.
In Episode 4, ~26 minutes, Janet Clinton demonstrates the app. The program doesn’t disclose that she earns royalties from it or that she is Hattie’s wife—both significant conflicts of interest.
The Education Endowment Foundation did a large study on the Visible Classroom app and found:
"Our trial of the Visible Classrooms intervention involved teachers of 7230 students from 86 schools. The independent evaluation found that pupils taught by teachers in intervention schools made, on average, one month less progress in KS2 reading and maths."
Also, O'Neill wrote a timely warning in 2012, about Hattie's strategy,
The "discourse seeks to portray the public sector as ‘ineffective, unresponsive, sloppy, risk-averse and innovation-resistant’ yet at the same time it promotes celebration of public sector 'heroes' of reform and new kinds of public sector 'excellence'. Relatedly, Mintrom (2000) has written persuasively in the American context, of the way in which ‘policy entrepreneurs’ position themselves politically to champion, shape and benefit from school reform discourses" (p. 2).This particularly applies to Revolution School - Hattie diagnosis:
Hattie's comments that education in Australia is "DIRE" since it is slipping down the world rankings, directly contradicts his 2005 ACER lecture where he exhorts to politicians & school leaders,
"It’s NOT about numbers, assessment literacy, national ability tests."McKnight & Whitburn (2018) in Seven reasons to question the hegemony of Visible Learning are also concerned about Hattie's portrayal in this TV series as,
"the potential saviour of public education and redeemer of recalcitrant teachers" (pp. 2 ff).They also question the financial conflict of interest of Visible Learning,
"Where are the flows of capital around Visible Learning? Where is capital and what kinds of capital are accruing for those producing “Visible Learning” as a brand? What material and financial benefits flow on to teachers and students?" (p. 6).Professor Ewald Terhardt (2011) also comments on Hattie's conflict of interest,
"A part of the criticism on Hattie condemns his close links to the New Zealand Government and is suspicious of his own economic interests in the spread of his assessment and training programme (asTTle)." (p. 434)Compare Hattie diagnosis with 2017 Australian of Year Maths teacher - Eddie Woo:
Was there a revolution?
Nilholm (2017),
"... he [Hattie] does not give reasonable answers to how his theses are to be translated into practical work and I have definitely not seen any study that critically examined what happens when municipalities and schools try to base their work on Hattie's work" (p. 3).The Maths curriculum for all Victorian schools including Kambrya has detailed the following criteria for ALL students to achieve by the end of Year 10:
"Evaluate statistical reports in the media and other places by linking claims to displays, statistics and representative data." Mathematics Statistics and Probability Levels 7-10A.We expect our students to evaluate claims but we rarely do this ourselves, particularly with regard to educational statistics and in particular to Hattie's claims.
The program displays this graphic indicating phenomenal improvement in Year 12 achievement, click here for short video clip from TV show.
Hattie's Preponderance to Exaggerate
In Hattie's 2017 presentation @researchEd in Melbourne, Hattie states,
Kambrya was in the bottom 10% of all schools in the state, now its in the top 10% in ANY measure! (@25mins here)
However, When I checked the Actual Results I got...
These results show a reasonable improvement from 2008 to 2010, but they don't indicate the phenomenal improvement implied by the graph on TV. Also, it appeared that when Hattie consulted with the school in 2015, the results slightly decreased!
Perhaps the graph is showing the change in ranking and the School got into the top 25% of Schools. But, this is not correct either as the school ranking in 2015 was 273rd out of 528 schools. Berwick College (nearby) was ranked higher at 260th - check rankings here.
Similar spectacular English results were displayed:
But these cannot be Y12 results so they must be the Y7 and Y9 NAPLAN results. When I checked the NAPLAN results (see- https://www.myschool.edu.au) I found these (Y9 are the higher line):
These NAPLAN results do not show phenomenal improvement.
Hattie Continues to Exaggerate the Improvement of Kambrya
"I was involved in the television show ‘Revolution Schools’ where a TV crew followed a high school here in Melbourne that went from the bottom 10% to top 10% in the state." (Hattie & Larsen 2020, p. 282)Hattie is completely misleading when he says the school improved to the top 10% in the State. The School did improve from 2008 to 2011 but the key NAPLAN results show a moderate decline from 2011 & the Year 12 scores show a leveling off from 2011.
Looking at the actual Year 12 scores it would be more accurate to say the School improved from the bottom 25% of schools to the bottom 35% of schools in the state.
An improvement but not a revolution.
Comparison with other local schools:
A comparison of Y12 results with other schools nearby shows a revolution is also happening, e.g., Berwick SC is 1km away from Kambrya.
Comparison with other local schools:
A comparison of Y12 results with other schools nearby shows a revolution is also happening, e.g., Berwick SC is 1km away from Kambrya.
But, if you look at the results of other private schools in the area they achieve much higher results!
Also, Nossal High (select entry) is a few km from Kambrya & Box Hill High, seems to achieve much better results. If there is any revolution, I would be looking at these 2 schools.
The 2016 results have just been released Kambrya - Median = 29 and '% of 40+' = 2.20%
Berwick SC - Median = 29 and '% of 40+' = 3.8%
The other spectacular results are the parent, student and staff surveyed perceptions of the school. I have not got access to all that data. Small segments of the surveys are available in the School's annual reports. These brief snapshots of the surveys do show improvement in the student, parent and staff attitudes. Whilst these are not the phenomenal improvements implied by the graphs, they are still worthwhile.
So what caused these improvements? Was it the leadership? Was it the passion and other qualities fostered and demonstrated by the teachers? Was it the strategies they employed? Was it due to Hattie's involvement? Or was it something else?
Hattie was very prominent on the show. He expressed some controversial views like 'public schools perform as well as private schools' and 'your postcode does not determine your VCE score'. But, the only strategy I could identify that he was directly involved with was the recording of teacher-directed instruction and determining the % of the time the teacher spoke versus students. It seemed Hattie was coaching staff to get down to 50%.
Given Hattie's position of pushing for influences with high effect sizes (d), values of d > 0.40 and his associated mantra of 'know thy impact', it seemed contradictory to me that the other strategies used by the school were strategies which Hattie ranks very lowly. Hattie has regularly described influences where d < 0.40 as 'going backward', 'distractions' or 'disasters'!
Strategies Kambrya used and their effect size (d):
Principals / school leaders: d = 0.36
Maths teacher told to make videos: audio/video, d = 0.22
Daraby (difficult) boys class reduced to 18 kids: class-size, d = 0.21 Hattie has called a focus on a class-size a disaster!
Daraby boys taken on outdoor-ed camp: co/team teaching d = 0.19, out of school d = 0.09, extra-curricula d = 0.17
Gender seperation: Daraby boys and girls: gender d = 0.12
Psychologist brought in to teach well-being, respect: Values/moral d = 0.24 (Prof. Lea Waters)
Full-time welfare worker: Welfare d = -0.12 (NEGATIVE!!!!)
Use of restorative practice: Decreasing disruptive behavior d = 0.34
Classroom management (Dr Rogers): Decreasing disruptive behaviour d = 0.34, but, Classroom behavioral d = 0.80
Reading programs: d = 0.36 (literacy consultant Di Snowball)
Debutante ball: out of school d= 0.09, extra-curricula d = 0.17
Home visitation: d = 0.29
Mentoring: d = 0.15
From Hattie's 2008 Nuthall lecture:
The School's choice to use these strategies (classified as poor by Hattie) questions Hattie's interpretation of the research. Also, Dan Haesler in his excellent commentary about the program mentions Hattie's input would cost the school about $60,000.
In Hattie's recent collaboration with 16+ experts; the head scholar, Peter Blatchford, concludes by asking for a new direction of research - the aim is move beyond the rather tired debates about whether these individual influences affect pupil performance and instead move things on by developing an understanding of the complex nature of classrooms and teaching with important practical benefits for education worldwide (p. 102).
Berwick SC - Median = 29 and '% of 40+' = 3.8%
The other spectacular results are the parent, student and staff surveyed perceptions of the school. I have not got access to all that data. Small segments of the surveys are available in the School's annual reports. These brief snapshots of the surveys do show improvement in the student, parent and staff attitudes. Whilst these are not the phenomenal improvements implied by the graphs, they are still worthwhile.
So what caused these improvements? Was it the leadership? Was it the passion and other qualities fostered and demonstrated by the teachers? Was it the strategies they employed? Was it due to Hattie's involvement? Or was it something else?
Hattie was very prominent on the show. He expressed some controversial views like 'public schools perform as well as private schools' and 'your postcode does not determine your VCE score'. But, the only strategy I could identify that he was directly involved with was the recording of teacher-directed instruction and determining the % of the time the teacher spoke versus students. It seemed Hattie was coaching staff to get down to 50%.
Given Hattie's position of pushing for influences with high effect sizes (d), values of d > 0.40 and his associated mantra of 'know thy impact', it seemed contradictory to me that the other strategies used by the school were strategies which Hattie ranks very lowly. Hattie has regularly described influences where d < 0.40 as 'going backward', 'distractions' or 'disasters'!
Strategies Kambrya used and their effect size (d):
Principals / school leaders: d = 0.36
Maths teacher told to make videos: audio/video, d = 0.22
Daraby (difficult) boys class reduced to 18 kids: class-size, d = 0.21 Hattie has called a focus on a class-size a disaster!
Daraby boys taken on outdoor-ed camp: co/team teaching d = 0.19, out of school d = 0.09, extra-curricula d = 0.17
Gender seperation: Daraby boys and girls: gender d = 0.12
Psychologist brought in to teach well-being, respect: Values/moral d = 0.24 (Prof. Lea Waters)
Full-time welfare worker: Welfare d = -0.12 (NEGATIVE!!!!)
Use of restorative practice: Decreasing disruptive behavior d = 0.34
Classroom management (Dr Rogers): Decreasing disruptive behaviour d = 0.34, but, Classroom behavioral d = 0.80
Reading programs: d = 0.36 (literacy consultant Di Snowball)
Debutante ball: out of school d= 0.09, extra-curricula d = 0.17
Home visitation: d = 0.29
Mentoring: d = 0.15
From Hattie's 2008 Nuthall lecture:
The School's choice to use these strategies (classified as poor by Hattie) questions Hattie's interpretation of the research. Also, Dan Haesler in his excellent commentary about the program mentions Hattie's input would cost the school about $60,000.
No comments:
Post a Comment