Complexity

I've argued strongly throughout this blog that Hattie's Meta-meta-analysis method is flawed & wrong at a number of levels. 

I have previously shown that Hattie's method of summarising 100's studies with ONE average effect size is misleading & reports the opposite conclusion to that of the individual studies themselves, e.g., Class Size.

I have quoted reputable academics re Hattie's averaging, 

Prof Gene Glass (who Hattie quotes inspired him),

"Averaging class size reduction effects over a range of reductions makes no sense to me."

Bergeron & Rivard (2017) reiterates,

"Hattie computes averages that do not make any sense."

Thibault (2017), 

"We are entitled to wonder about the representativeness of such results: by wanting to measure an overall effect for subgroups with various characteristics, this effect does not faithfully represent any of the subgroups that it encompasses! 

...by combining all the data as well as the particular context that is associated with each study, we eliminate the specificity of each context, which for many give meaning to the study itself!"

My position is consistent with the highly respected European academic, Gert Biesta (2010), who summarised,

"...we have not yet conducted sufficient research in order to be able to encapsulate all factors, aspects and dimensions that make up the reality of education."

and also consistent with Dr Ben Goldacre on Meta-analysis in Education,

"I think you'll find it's a bit more complicated than that."

The major evidence summaries are neither consistent nor conclusive - see other researchers.

Examples of Studies Acknowledging the Complexity of Education

Ironically, Hattie often credits Graham Nuthall's most famous work The Hidden Lives of Learners as the inspiration & name of his book "Visible Learning".

Nuthall used the analysis of detailed sound and video recordings of teaching and learning in the classroom and concluded learning is complex, nuanced & changing with circumstance.

David Clarke has continued Nuthall's work using more sophisticated sound and video recordings.

Professor Clarke set up The International Centre for Classroom Research which has focused on 3 projects:

1. The Learner’s Perspective Study (LPS), examined teaching and learning classroom practices of competent mathematics teachers from sixteen countries.

2. The Social Unit of Learning project is a study of student social interaction during the completion of mathematics tasks undertaken in individual, pair, small group and whole class social units. The aim of the project is to investigate the social interactions that characterise learning

3. The Learning from Lessons project, which investigates the structure and construction of mathematics teacher knowledge in Australia, China, and Germany. The project aims to understand mathematics teachers’ thinking processes during lesson planning, teaching.

Jan Tishauser has presented on Nuthall's findings here, and states,

"The outcomes of the research that Graham Nuthall conducted into the classroom experience of learners are little known, notwithstanding the far-reaching implications for our classroom practice."

Similarly, David Clarke's research is little know by teachers.

These bodies of research contradict Hattie's simplistic claims that-

1. Educational influences can be separated from context.

2. These influences can be measured in a meaningful way using the Effect Size.

3. These effect sizes can be used to determine 'What Works Best" or "Best Bets".

I will in the coming months summarise David Clarke's work here.

Teacher Appraisal

Dylan Wiliam goes through the evidence on Teacher Appraisal, particularly using class-room observation and shows the results vary enormously due to the complexity of Teaching.