Class Size - Other Issues

Hattie's Presentations on Class Size.

1. The Disasters.

Hattie started identifying & presenting influences with low ES as "disasters" from about 2005. In particular, he promoted his view that "reducing class size" was a "disaster" and a "distraction" - e.g. from Hattie's 2005 ACER lecture: 

Hattie continued deriding class size to 2015 in "What Does Not Work in Education: The Politics of Distraction", naming Class Size as one of Education's distractions.

Then in the 2016 TV series, "Revolution School" (part 3, 1min 20sec) Hattie definitively states,

"Reducing class size... does not make a difference to the quality of education!"

Worse, Hattie and his commercial partner Corwin have continued to publish since 2016 to (September 2022):

Hattie (2015) responding to numerous critiques of his work, seemed to pull back a little, lamenting:

"The main message remains, be cautious, interpret in light of the evidence, search for moderators, take care in developing stories..."

Contradicting the Corwin advert above, and at the same time in 2016, writing in a collaboration, "Class Size Eastern and Western Perspectives", Hattie concedes & cleverly changes his narrative,

"The evidence is reasonably convincing - reducing class size does enhance student achievement" (p. 113).

"Why is the (positive) effect so small?" (p. 105).

Eacott's (2017) analysis of this was insightful,

"Hattie’s work has provided school leaders with data that appeal to their administrative pursuits" (p. 3).

2. Class Size Matters, But Why is the ES small?

With more peer review critique of his work, Hattie has retreated from those earlier "disaster" presentations and cleverly changed his message to, Class Size does matter but why is the ES so small?

E.g., In Hattie's 2021 presentation "Visible Learning: The Sequel" he again emphasised the ES is small but apologises,

"Frankly I did not help by initially casting anything below 0.40 as unimportant, that should never have been the message." (@11mins)

In his interview with Lovell (2018), Hattie admits his rankings were misleading and that he does not rank anymore. Furthermore he cleverly pivots to a different "The Story" narrative,

"It’s not the numbers it’s the story... that’s why this will keep me in business to keep telling the story" (@46min)

Again with Larsen (2020), Hattie retreats from his earlier polemic,

"they look at that effect-size table and say tick, tick, tick to the top influences and no, no, no to the bottom, and this was never my message." (p. 28)

"Too many readers thought I was saying we can rank all effects one by one (whereas there are underlying themes differentiating the top and bottom influences), too many thought 0.40 was somehow magic." (p. 79)

Then again with Kraft (2021) Hattie retreats,

"But since 2009, I have tried to discourage focusing only on the top influences and ignoring the lower effects, as it may be easier to derive high effects when the outcome is narrow (e.g., vocabulary) than when it is wide (e.g., creativity). I have also tried to emphasize that while achievement is important, there are many other attributes that we need to value in schools."

So Why is the ES Small?

1. A number of researchers have identified that the ES calculated from standardised tests can be 1/4th that calculated by specific tests for the SAME intervention (Ruiz-Primo et al. (2002), Wiliam (2016), Simpson (2017) & Kraft (2019)). 

Class Size has been measured with standardised tests and aspects like Feedback have been measured with specific tests, so comparing ES from these two different types of tests is not valid.

Hattie in his interview with Lovell (2018, @45mins) admits standardised tests get lower ES.

Then, Kraft (2021) in his discussion with Hattie reminds him of this,

"We both agree that the magnitude of an effect size depends a lot on what outcome you measure. Studies can find large effect sizes when they focus on more narrow or proximal outcomes that are directly aligned with the intervention and collected soon after. It is much easier to produce large improvements in teachers' self-efficacy than in the achievement of their students. In my view, this renders universal effect size benchmarks impractical."

2. Averaging across a range of different Class Size reductions reduced the ES for a particular Class Size reduction. This is clearly the case when Hattie averages the Glass & Smith (1979) study to get a low ES of 0.09.

3. Despite being shown the above 2 reasons, Hattie uses a 3rd explanation or "story". Hattie claims the reason is because teachers do not change their methods from larger to smaller classes.

Does Teaching Change From Larger to Smaller Classes?

A number of authors who Hattie cites contradict Hattie's claim teaching does not change.

Glass & Smith (1980) Meta-analysis of Research on Class Size and Its Relationship to Attitudes and Instruction, which is a companion to their 1979 study, conclude,

"In sum, reducing class size has beneficial effects both on cognitive and affective outcomes and on the teaching process." (p. 432)

Goldstein et al. (2003) also show that a whole range of factors change in smaller classes:

Teaching changes dramatically in small classes.

More individualised learning.

More immediate feedback and questions.

Behaviour of students.
Students have more focused and sustained attention.
Class size effects are compounding.
The age of students is important.
Benefit children who are most in need academically, and who thus have most ground to make up.

Blatchford (2003), also showed significant changes,

"We found consistent relationships between class size and teaching... children in small classes were more likely to interact with their teachers, there was more teaching on a one-to-one basis, more times when children were the focus of a teacher’s attention, and more teaching overall. In short, there was more teacher task time with pupils... there was more teacher support for learning, as reflected in the amount of individual attention paid to students, and in terms of the immediate, responsive, sustained and purposeful nature of teacher interactions with children, the depth of a teacher’s knowledge about children, and sensitivity to individual children’s needs." (p. 149).

Prof David Zyngier provides some support to Hattie, that teachers don't change, but also cites research showing other differences,

"In larger class sizes teachers used class groupings, and these classes had lower achievement, while in smaller classes it was more common to teach to the whole class. There were more student questions in larger classes (usually seeking help or clarification), but more teacher follow-up of questions in smaller classes. There was a greater use of homework, assignments and oral tests for assessment purposes in smaller classes. The amount of time teachers spent directly interacting with students, and monitoring students’ work, was differently related to class size; more direct interaction occurred in smaller classes, whereas teachers lectured or explained more in larger classes. Finally, larger classes had fewer interactions overall between teachers and students, had higher noise levels, required more management than smaller classes, and the time spent in this way did not assist student learning" (2014, p. 6).

The Most Comprehensive Review of Class Size (so far).

Peter Blatchford, the lead author of:

Class Size Eastern and Western perspectives (2016) challenges Hattie's promotion that reducing class size is a "disaster",

"One reason for the prevalence of the unimportant view are several highly influential reports which have set in motion a set of messages that have generated a life of their own, separate from the research evidence, and have led to a set of taken for granted assumptions about class size effects.

Given the important influence these reports seem to be having in government and regional education policies, they need to be carefully scrutinised in order to be sure about the claims that are made" (p. 93).

Blatchford names Hattie's interpretation & summary of studies as the major source of the evidence provided by these reports.

Blatchford et al. (2016), also talk about the lack of quality evidence,

"there are in fact relatively few high-quality dedicated studies of class size and this is odd and unfortunate given the public profile of the class size debate and the need for firm evidence based on purposefully designed research fit for purpose" (p. 275).

Hattie Claims He Regularly Teaches to a 1000 Students!

In this 2019 podcast with Pedagogy Non-Grata (around 52mins)

"...in my work I deal with class sizes of 1000, I know how to do that, I’ve got quite good at it over the last 40 years, how to teach to class of 1000."

That's strange, the Melbourne Graduate School, where Hattie has been for last 11 years, does not have a room or theatre that would fit anywhere near 1000 students. Also, he mostly works 1-to-1 with PhD students.

This is another example of Hattie's propensity for exaggeration and misrepresentation.

Hattie's Interpretation Is Used by Politicians for Public Policy:

"Hattie’s work has provided school leaders with data that appeal to their administrative pursuits" Eacott (2017, p. 3).

The Australian Government used Hattie & The Grattan Institute to block significant funding to redress the socioeconomic imbalance in Australian Schools - called the Gonski Review.

In a report by Dean Ashenden, summarising and interviewing the then Education Minister, Christopher Pyne (2012), Ashenden states,

"The best single source of evidence about the relative effectiveness of class size reductions and many other educational strategies is John Hattie’s Visible Learning..."

Pyne was reported to say,

"the evidence overwhelmingly shows that investing in teacher effectiveness rather than the number of teachers is the most successful method of improving student learning and creating top performing education systems."

Professor Blatchford (2016c) comments about this,

"When Christopher Pyne [the then Australian Education Minister] talked about prioritising teacher quality, rather than reducing class sizes, he set up a false and simplistic dichotomy" (p. 16).

From New Zealand, a similar example, where Professor John O'Neill (2012a) writes a significant letter to the NZ Minister of Education on the problem of using Hattie's research for class size policy, then publishes these issues in O'Neill (2012b).

Further O'Neill (2012b) states,

"...the Minister of Education declined to rule out increases in class size. In short, this was because the ‘independent observation’ of Treasury and the research findings of an influential government adviser, Professor John Hattie, were that schooling policy should instead focus on improving the quality of teaching." (p. 1).

Also, on Hattie's class size interpretation O'Neill (2012b) warns that,

"Much of the terminology is ambiguous and inconsistently used by politicians, officials and academic advisers. The propositions are not demonstrably true – indeed, there is evidence to suggest they are false in crucial respects. The conclusion is, at best, uncertain because it does not take into account confounding evidence that larger classes do adversely affect teaching, learning and student achievement" (p. 2).

I am concerned about the unwavering confidence that Hattie displays when he talks about class size, given the caution and reservation that the scholars of each of his 3 studies discuss as well as other reputable scholars around the world. Reservations due to the lack of quality studies, the inability to control variables, the major differences in how achievement is measured, major confounding variables and benchmark effect sizes.

Hattie's Errors

Hattie used the 3 -meta-analyses listed above to get his average effect size. However, he also cites the single study, Dustmann et al., (2003) on p. 87 & also in his 2005 paper, "The paradox of reducing class size and improving learning outcomes." But, he quotes an effect size from this paper of -0.04. But, Dustmann et al., found that the larger the class, the worse the test performance, i.e., the paper's reference point is INCREASING class size NOT reducing class size (which is Hattie's reference point) so the effect size should be +0.04.

Hagemeister (2020, p. 4) and also Sundar & Agarwal (2021, p. 9) report this error and Slavin (2018) reports Hattie makes the same error in citing other studies.

Hattie also claims he is focusing on class size reductions of 25 to 15. He describes the table of class size studies on p. 87 as, "Synthesis of meta-analyses and major studies reducing class size from 25 to 15." However, from the details of the studies, shown above, this is clearly NOT the case.

Class Size Eastern and Western perspectives (2016) - more details.

Edited by Prof Blatchford et al. Note: Blatchford has a dedicated website to class size research - http://www.classsizeresearch.org.uk

The editor's state,

"there are in fact relatively few high-quality dedicated studies of class size and this is odd and unfortunate given the public profile of the class size debate and the need for firm evidence based on purposefully designed research fit for purpose" (p. 275).

"What often gets overlooked in debates about class size is that CSR is not in itself an educational initiative like other interventions with which it is often (and in a sense unfairly) compared, for example, reciprocal teaching, teaching metacognitive strategies, direct instruction and repeated reading programmes; it is just a reduction of the number of pupils in a classroom" (p. 276).

Blatchford warns again about correlation studies, 

"Essentially the problem is the familiar one of mistaking correlation for causality. We cannot conclude that a relationship between class size and academic performance means that one is causally related to the other" (p. 94).

The editors conclude, 

"the chapters in this book are only a start and much more research is needed on ways in which class size is related to other classroom processes. This has implications for research methods: we need more systematic studies, e.g. which use systematic classroom observations, but also high-quality multi-method studies, in order to capture these less easily measured factors.

There is some disagreement about which groups are involved but often studies find it is low attaining and disadvantaged students who benefit the most. Blatchford et al. (2011) found evidence that smaller classes helped low attaining students at secondary level in terms of classroom engagement. Hattie (Chapter 7) develops the view that we might expect low attaining students to benefit from small classes in terms of developing self regulation strategies" (p. 278).

Blatchford concludes, 

"The aim is move beyond the rather tired debates about whether class size affects pupil performance and instead move things on by developing an integrative framework for better understanding the relationships between class size and teaching, with important practical benefits for education world wide" (p. 102).

Hattie's contribution to the book (Chapter 7):

Hattie appears to be an outlier in this book. Of the 17 scholars who have contributed to the book ONLY Hattie myopically uses the effect size statistic to fully interpret the research. All the others use contextual and detailed features of the research to reach the conclusion that class size is important and significant.

At least the weight of scholarship has caused Hattie to retreat from his polemic on reducing class size as "a disaster" and "going backwards" and he finally concedes, 

"The evidence is reasonably convincing - reducing class size does enhance student achievement" (p. 113).

But, Hattie cleverly re-frames the issue to 

"Why is the (positive) effect so small?" (p. 105).

Given the significant amount of critique about Hattie's methodology: the lack of quality studies, the use of, disparate measures of student achievement, university students or pre-school children, correlation, the inconsistent definition of small and large class sizes, indiscriminate averaging,benchmark effect sizes, etc, etc. I was disappointed that Hattie did not address any of these issues. But rather focused on attacking Zyngier's (2014b) meta-review, 

"Zyngier's review misses the elephant in the room" (p. 106).

But if Zyngier misses the elephant in the room, then so do all the other 16 researchers contributing to the book. For example, in the following chapter (8) Finn & Shanahan, display what they believe to be significant findings (p. 124):

Hattie once again sidesteps the SIGNIFICANT issues raised by Zyngier & others - the control of variables, the differing definition of large and small classes, the differences on how to measure class size, e.g., some studies use a student/teacher ratio (STR) which includes many non-teaching staff like the principal, welfare staff, library, etc. 

"Past research has too often conflated STR with class size" (p. 4).

Blatchford et al. (2016), also comment on this STR problem, 

"they are not a valid measure of the number of pupils in a class at a given moment" (p. 95).

Hattie just re-states that meta-analyses provide a reasonably robust estimate and myopically focuses on the effect size statistic. But he provides no defence for the validity issues. However, he concedes STR and class size are different, but he does not resolve the validity issue of using these disparate measures and just fobs off the argument by using a red herring - STR and Class size are related (p. 112) but he provides no evidence for this claim. 

Given the importance of class size research, STR and Class size need to be MORE than just related.

They need to be the SAME!!!!

Hattie includes a 4th study to his effect size average, Shin and Chung (2009) - effect size d = 0.20. But he conveniently does not inform the reader that this study re-analysed the same data (the Tennessee STAR study) as the previous meta-analyses that he used.

Ironically, Shin and Chung warn against creating an effect size from repeated use of the same data, 

"If a study has multiple effect sizes, the same sample can be repeatedly used. Repeated use of the same sample is, however, a violation of the independent assumption" (p. 14).

They also warn, 

"we found too many Tennessee STAR studies... We worry about the dependence issue" (p. 15).

It seems to me Hattie's strategy is to take the focus off the scrutiny of his evidence and re-direct our attention elsewhere - a strategy for politicians, NOT for researchers!

Join the group 'Class Size Matters' - here.

Teacher Morale:

Blatchford et al. (2016), comment on the  associated issue of teacher morale and class size,

"Virtually all class size studies report that teacher morale is higher in small classes than in larger classes. The personal preference for small classes was demonstrated by STAR third-grade teachers interviewed at the end of the school year. Teachers were asked whether they would prefer a small class with 15 students or a $2,500 salary increase. Seventy percent of all teachers and 81 percent of those who had taught small classes chose the small class option over a salary increase" (p. 129).

Prof Gene Glass agrees, 

"Teacher Workload and its relationship to class size is what counts in my book."

PISA

Blatchford et al., challenge the statements of the head of PISA, Andreas Schleicher,

"there was reference to ten myths of education, as expressed by Andreas Schleicher, one of which was the myth that smaller classes benefited academic performance. The editors of this book tend to side with Berliner and Glass (2014) who address what they see as the 50 myths and lies which threaten American public schools. Myth no. 17 in their list is the belief that reducing class size will not result in more learning" (p. 275).

OECD Class Size Averages

from - https://www.oecd.org/education/skills-beyond-school/EDIF%202012--N9%20FINAL.pdf

The Literacy/Numeracy Initiative

In Hattie's jurisdiction, the state of Victoria Australia, the Education Dept is implementing the largest educational initiative in Australia, costing over $200 Million. The project aims to improve literacy and numeracy.

The program incorporates a wide range of research, e.g., The G.R.I.N Program from Monash University., which recommends,

"Prior to the normal daily mathematics lesson, trained tutors conduct GRIN sessions with small groups of students (ideally three)."

A number of teachers commented in my session that this is in conflict with Hattie's presentations the class size does not matter.

The Grattan Institute's 2023 report "Tackling under-achievement: Why Australia should embed high-quality small-group tuition in schools" confirms the GRIN project recommendations of 1 teacher to 3 students.

Other Commentary

The Australian Education Union (2014) has published a comprehensive analysis of the class size research. They summarise that reducing class size does seem to improve student outcomes. Also, they highlight the problems with Hattie's methodology:

"The critics have cited the methodological problem of synthesising a whole range of meta-studies each with their own series of primary studies. There is no quality control separating out the good research studies from the bad ones. The different assumptions, definitions, study conditions and methodologies used by these primary studies mean that Hattie’s meta-analysis of the meta-analyses is a homogenisation which may distort the evidence (comparing apples with oranges)" (p. 13).

"The 0.21 effect he claims for class size is an average so that some studies may have found a significantly higher effect than that. For example, ‘gold standard’ primary research studies (using randomised scientific methodology) such as the Tennessee STAR project recorded a range of effect sizes including some at 0.62, 0.64 and 0.66, clearly well above the ‘hinge-point’ and the same as most variables which Hattie regards as very important" (p. 14).

O'Neill (2012a) riases more complex issues using the detailed case/naturalistic study by Blatchford (2011),

"...Blatchford makes the point that class size effects are ‘multiple’. For children at the beginning of schooling, there are significant potential gains in reading and maths in smaller classes. Children from ethnic minorities and children who start behind their peers benefit most. There is also a positive effect on behaviour, engagement and achievement, particularly for low achievers, where classes are smaller in the lower secondary school" (p. 10).

Leading researcher, Professor Dylan Wiliam states that the evidence is pretty clear that if you teach smaller classes you get better results. The problem is smaller classes cost a lot more (7min into full lecture).

Also, many scholars point out the irony in Hattie's view, that class size is a distraction - because the number of students in a class limits the ability of teachers to implement the kinds of changes that Hattie shows have the biggest effect, e.g., formative evaluation, micro teaching, behavior, feedback, teacher-student relationships, etc.

For example, Zyngier (2014) in his meta-review - 

"The strongest hypothesis about why small classes work concerns students’ classroom behaviour. Evidence is mounting that students in small classes are more engaged in learning activities, and exhibit less disruptive behaviour" (p. 17).

Each of these studies also discusses their limitations. In particular, Goldstein et al. (2000) emphasise the issue, that has emerged for all of Hattie's synthesis; 

"...we have the additional problem that different achievement tests were used in each study, and this will generally introduce further, unknown, variation" (p. 403).

Goldstein et al. (2003) go into detail about the problems of comparing correlation studies with random controlled experiments;

"…correlational studies that ... examined relationships between class size and children’s achievements at one point in time, are difficult to interpret because of uncertainties over whether other factors (e.g., non-random allocation of pupils to classes) might confound the results" (p. 3).

Goldstein et al. (1998) point out another major confounding variable: 

"There is a tendency for schools to allocate lower achieving children to be in smaller classes. This bias means a considerable number of large cross-sectional studies (correlational) need to be ignored due to validity requirements" (p. 256).

Zyngier (2014) in his meta-review on class size -

"Noticeably, of the papers included in this review, only three authors supported the notion that smaller class sizes did not produce better outcomes to justify the expenditure" (p. 3).

"The highly selective nature of the research supporting current policy advice to both state and federal ministers of education in Australia is based on flawed research. The class size debate should now be more about weighing up the cost-benefit of class size reductions, and how best to achieve the desired outcomes of improved academic achievement for all children, regardless of their background. Further analysis of the cost-benefit of targeted CSR is therefore essential" (p. 16). 

"Recognised in the education research community as the most reliable and valid research on the impact of class size reductions at that time, the Tennessee STAR project was a large series of randomised studies, followed up in Wisconsin by the SAGE project. After four years, it was clear that smaller classes did produce substantial improvement in early learning and cognitive studies, and that the effect of small class size on the achievement of minority children was initially about double that observed for majority children" (p. 7).

Zyngier concludes:

"Findings suggest that smaller class sizes in the first four years of school can have an important and lasting impact on student achievement, especially for children from culturally, linguistically and economically disenfranchised communities" (p. 1).

Snook et al. (2009) in their peer review of Hattie, also comment in detail about class size. They also discuss the STAR study reporting effect sizes did reach 0.66. They conclude: 

"The point of mentioning these studies is not to 'prove' that Hattie is 'wrong' but to indicate that drawing policy conclusions about the unimportance of class size would be premature and possibly very damaging to the education of children particularly, young children and lower ability children. A much wider and in depth debate is needed" (p. 10).

Dan Haesler has a detailed look at class size and other issues.



Kelvin Smythe provides a detailed review of Hattie's research on Class size.

Why Teachers Agree Class Size Does Matter.

A comedian's look at class size - https://youtu.be/GGvo7TPN67A