References

Published peer reviews of Visible Learning:


Allerup, Peter (2015). Hattie's use of effect size as the ranking of educational efforts. Paideia: Journal of Professional Educational Practice, No. 9, 42-51.
Bergeron, Pierre-Jérôme (2017). How to engage in pseudoscience with real data: A Criticism of John Hattie's Visible Learning from the perspective of a statistician. You can listen to Prof Bergeron's podcast here.

Blatchford et al (2016). Class Size Eastern + Western perspectives.

Blatchford, Peter (2016). Is Class Size important?


Blatchford, Peter (2016). Class Half Full, AEU News (Vol 22 - 7, Dec 2016).

Blichfeldt, Jon Frode (2011). About valid research and use of research and test results.

Cheung, A., & Slavin, R. (2016). How methodological features affect effect sizes in education
Educational Researcher, 45(5), 283-292.

Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Earlbaum Associates.

Eacott, Scott (2017). School Leadership and the cult of the guru: the neo-Taylorism of Hattie
School Leadership & Management, 37(4), 413–426.

Eacott, Scott (2018). Ranting, raving and complaining: reflections on working against orthodoxyInternational Journal of Leadership in Education. DOI: 10.1080/13603124.2018.1492025.

Glass, G. V., (1977). Integrating Findings: The Meta-Analysis of Research. Review of Research in Education, Vol. 5, 351-379.

Glass, G. V., Smith, M. L. (1979). Meta-Analysis of Research on Class Size and Achievement. Educational Evaluation and Policy Analysis, Vol. 1, No. 1, pp. 2- 16.

Glass, G. V., (2000). Meta-Analysis at 25.

Higgins and Simpson (2011). A review of Visible Learning.

Hooley, Neil (2013). Making judgments about John Hattie's effect size.


Imsen, Gunn (2011). Hattie fever in Norwegian politics.

Knudsen, Hanne (2017). John Hattie: I’m a statistician, I’m not a theoretician.

Kohn, Alfie (1994). The Truth About Self-Esteem. Phi Delta Kappan, December 1994.

Kohn, Alfie (2006). Abusing Research- The Study of Homework and Other Examples. Phi Delta Kappan, September 2006.

Lervåg, Arne &  Melby-Lervåg, Monica (2014). John Hattie’s Visible Learning: No "Holy Grail" for teaching.

Lind, Georg (2011). Pedagogy or structure? The Hattie study also knows no advice. Hessian teacher newspaper 11/2011, 26-27.

Lind, Georg (2013). Meta-analyzes as a guide? On the Reception of the study of Hattie in politics.

McKnight, L & Whitburn, B. (2018). Seven reasons to question the hegemony of Visible Learning. Discourse Studies in the Cultural Politics of Education DOI: 10.1080/01596306.2018.1480474

Myburgh, Siebert (2016). Critique of Peer reviewed Articles on John Hattie’s Use of Meta Analysis in Education.

Nepper Larsen, Steen (2014). Know thy impact – blind spots in John Hattie’s evidence credo.  https://www.journalofacademicperspectives.com/back-issues/volume-2015/volume-2015-no-1/

Nielsen, Klaus & Klitmøller, Jacob (2017). Blind spots in Visible Learning - Critical comments on "Hattie revolution". (Danish).

Nilholm, Claes (2013). It's time to critically review John Hattie.

Nilholm, Claes (2017). Is John Hattie in Blue Sword? (The Blue Sword was a fantasy novel).

O'Neill, John (2012). Letter to the NZ Minister of Education, critical of Hattie's methods.

O'Neill, John (2012). Material fallacies of education research evidence and public policy advice.


O'Neill, J., Duffy, C.  & Fernando, S. (2016). Charities, Philanthropists, Policy Entrepreneurs, International Companies and State Schooling in Aotearoa New Zealand. Final report to the New Zealand Educational Institute Te Riu Roa, New Zealand Post Primary Teachers’ Association Te Wehengarua, and New Zealand Primary Principals’ Federation Ngā Tumuaki o Aotearoa.

Pant, Hans Anand (2014). Preparation of evidence for education policy and pedagogical decisions: meta-analyzes in educational research. Journal of Educational Science, July 2014, Volume 17, Supplement 4, pp. 79-99.

Poulsen, Sten Clod (2014). John Hattie: A Revolutionary Educational Researcher? Adult Education, March 2014, No. 108, pp. 13-18.

Proulx, Jérôme (2017). Critical essay on the work of John Hattie for teaching mathematics: Entrance from the Mathematics Education.

Robinson, Daniel H (2004). An Interview with Gene V. Glass.

Rømer, Thomas (2016). Criticism of Hattie's theory about Visible learning. Retrieved February 20, 2016.

Ruiz - Primo, Maria Araceli; Shavelson, Richard J.; Hamilton, Laura; Klein, Steve (2002). On the evaluation of systemic science education reform: Searching for instructional sensitivity. Journal of Research in Science Teaching, 2002, Vol.39(5), p.369-393.

Sahlberg, Pasi (2015). You can do more with less.

Sahlberg, Pasi (2015). Finnish Lessons 2.0.

Schulmeister & Loviscach (2014). Errors in John Hattie’s “Visible Learning”.

Schulmeister & Loviscach (2014). Critical comments on the study "Making learning visible" (Visible Learning).

See, Beng Huat (2017). Evaluating the evidence in evidence-based policy and practice: Examples from systematic reviews of literature.

Shannahan (2017). Why You Need to Be Careful About Visible Learning.


Simpson, Adrian (2017). The misdirection of public policy: comparing and combining standardised effect sizes. You can listen to Prof Simpson's detailed podcast here.


Simpson, Adrian (2018). Princesses are bigger than elephants: Effect size as a category error in evidence-based education. British Educational Research Journal. Vol 44, Issue 5.

Sjøberg, Svein (2012). Hattie's Visible learning in perspective: Critical comments Educational Association's School Leadership Conference, Web Publishing: Oslo, Oct. 23, 2012.


Slavin, Robert (2018). John Hattie is Wrong.

Snook, I., Clark, J., Harker, J.,  O’Neill, A., O’Neill, J. & Openshaw, R. (2009). Invisible Learnings?: A Commentary on John Hattie's Book - 'Visible Learning.

Snook, I., Clark, J., Harker, J.,  O’Neill, A. & O’Neill, J. (2010). respond to Hattie's defense in 'Critic and Conscience of Society: A Reply to John Hattie.

Terhart, Ewald (2011). Has John Hattie really found the holy grail of research on teaching?

Thibault, Mathieu (2017). Is John Hattie's Visible Learning so visible?


Wiliam, Dylan (2001). Inside the Black Box.


Wiliam, Dylan (2016). Leadership for teacher learning. Moorabbin, Victoria. Hawker Brownlow Education.

Wiliam, Dylan (2017). Getting educational research right


Wecker, Christof., Vogel, Freydis., & Hetmanek, Andreas (2016). 'Visionary and impressive - but also resilient ?'

Wrigley, Terry (2015). Bullying by numbers.

Wrigley, Terry (2018). The power of ‘evidence’: Reliable science or a set of blunt tools? British Educational Research Journal, Vol. 44, No. 3, June 2018, pp. 359–376. DOI: 10.1002/berj.3338

Yelle, et al. (2016). What is visible from learning by problematization: a critical reading of John Hattie's work.

Zyngier, David, (2014). Class size and academic results, with a focus on children from culturally, linguistically and economically disenfranchised communities.



Hattie References:


Corwin, Hattie's commercial arm of Visible learning, website here.

Hattie, John. (2010). On Being a 'Critic and Conscience of Society': The Role of the Education Academic in Public Debates. New Zealand Journal of Educational Studies, Volume 45 Issue 1, 85-96.

Hattie Interview (2015). Teacher Magazine.

Hattie, John. (2017). Educators are not uncritical believers of a cult figure, School Leadership & Management, 37:4, 427-430.


Books:


Salkind, N.  (2006). Encyclopedia of Measurement and Statistics, SAGE.

Smith, G. (2014). Standard Deviations- Flawed assumptions, Tortured data and other ways to lie with statistics, Duckworth Overlook, London.

Willingham, D. (2012). When Can You Trust the Experts: How to Tell Good Science from Bad in Education. Jossey-Bass, San Francisco, CA.


Detailed On-line Reviews:


Ashman, Greg. A critique of the Education Endowment Fund (EEF).

Ashman, Greg. A critique of Meta-analysis methodology.

Becker, Dr Jonathan. A critique of Marzano's research, but very relevant to Hattie's research.

Brown, Neil Excellent summary of issues with Visible Learning.

Cain, Tim. A review of Hattie's Teacher Subject Matter.

Class Size Matters - Join the group -here.

DiCerbo, Dr. Kristen. Detailed analysis of Self-report grades.

Didau, David. Effect size, class size, self-report grades.

Didau, David. https://learningspy.co.uk/research/john-hattie-and-the-magical-power-of-prediction/

DeWitt, Peter. Hattie is not wrong. 

Docendo Discimus. Detailed analysis of a number of influences.

Frederiksen, Mette. General comments about Visible Learning.

Haesler, Dan. details a number of issues, in particular, Class Size.

Lupton, Dr Mandy. a critique of Hattie's misrepresentation of Inquiry-based and problem-based teaching.

Moore, Darcy. posts on the 2 peer review critiques of Hattie.

Ollie Orange. Details of mistakes in Visible Learning.

Rose, Nick An excellent blog on analysing evidence in Educational research.

Talmage, Emily. Who is Dr. Marzano.

Van Dijk, Rene. A good overview of critiques.

Smythe, Kelvin. Has written extensively on Hattie.

Wiliam, Prof Dylan. Adds many relevant comments on a variety of online forums.


Teacher Unions:

Alberta Teacher's Association - Jonathan Teghtmeyer.

New Zealand Post Primary Teacher's Association reviews of Hattie.


Other Blogs:


Great summary of problems with Effect Size in education by Nick Rose.

Some dialogue with the Guru himself

The Blog that initially identified Hattie's incorrect CLE calculations


News Paper Articles:


Chris Bantick its-time-researchers-let-teachers-do-their-job


Radio/Video:


Dr. Ben Goldacre, Bad Science web site.

Dr. Pierre-JérômeBergeron, on Radio for ResearchED.

Dr. Adrian Simpson, on the ERR Podcast.


Summary Details of Peer Reviews:


Professor John O'Neill has reviewed these influences: micro-teaching, professional development, providing formative evaluation, comprehensive interventions for learning disabled students, feedback, spaced vs. massed practice, problem-solving teaching, metacognition strategies, teaching strategies, co-operative vs. individualistic learning, study skills and mastery learning.

Dr Kristen Dicerbo has analysed self-report grades.

Dr Mandy Lupton has analysed Problem-Based and Inquiry-Based Learning.

Professors Higgins and Simpson have published Hattie's calculation errors.

Professor Arne Kare Topphol also published Hattie's calculation errors (in Norwegian) summary here.

Professor Ivan Snook et al, a general critique of VL focusing on lack of quality studies, the problems of Hattie's ranks and generalisations. They use class size and homework as examples.

Professor Ewald Terhardt published a general critique of Hattie's methodology and issues of Hattie's conflict of interest.

Hattie's retort to Snook and Terhardt, which is basically a defense of meta-analysis as a methodology.

Snook, Clark, Harker, Anne-Marie O’Neill and John O’Neill, a reply to Hattie's retort.

Lind, Georg (2013)
'Hattie synthesis are shortsighted and its conclusions problematic.'
'The famous psychologist Paul Meehl once defined a good scientist - and as someone who uses his head instead of a formula. The meta-analysis should, so once its inventor, an "intellectual counterpart of original empirical research" writes his. However, when Hattie it degenerates into a mechanical implementation of computer-generated statistics in policy to the exclusion of (re) thinking. Worse, meta-analyzes suggest to the reader that they make the reading of original research unnecessary.'
Dr Myburgh analysed Hattie's retort to Snook, et al and Terhardt above. Myburgh focuses on the critique of meta-analysis as a methodology and not the specific critiques of Hattie's misrepresentations.

Professor Bergeron (2017) published Hattie's calculation errors plus other issues about correlation studies and misinterpretation.
'When taking the necessary in-depth look at Visible Learning with the eye of an expert, we find not a mighty castle but a fragile house of cards that quickly falls apart' (p1).
Pant (2014) critiques Hattie methodology showing major flaws in calculations and interpretations.
'an attempt was made to argue that the hitherto most comprehensive research findings synthesis in education, John Hattie's meta-analysis Visible Learning (2009), although in many places a commitment to the importance of differentiating considerations of the findings of meta-analyzes surrenders that his own methodological approach, however, this seems more like lip service' (p96).

Wecker et al (2016) published major issues of misrepresentation, major calculation & interpretation errors.
'the methodological claims arising from Hattie's approach, and the overall appropriateness of this approach suggest a fairly clear conclusion: a large proportion of the findings are subject to reasonable doubt' (p35).
Professor Timothy Shanahan Why You Need to Be Careful About Visible Learning (2017) shows that Hattie often counts the same studies twice resulting in unreliable effect sizes. Also, he shows that Hattie gives the same weighting to meta-analyses even though there are vast differences in their quality, which also leads to unreliable results.

Professor Scott Eacott's (2017) critique of the 'cult of Hattie'; how and why it came to be and its dangers.


Kelvin Smythe e.g., 'John Hattie: your research is now a con.'

Whilst not directly about Hattie's evidence for feedback, David Didau gives an excellent overview of the evidence for feedback here. Also, Gary Davies has an excellent blog - Is Education Research Scientific?

No comments:

Post a Comment