Hattie's Rankings

Rank	Domain	Influence	d
1	Student	Self-report grades	1.44
2	Student	Piagetian programs	1.28
3	Teaching	Providing formative evaluation	0.90
4	Teacher	Micro teaching	0.88
5	School	Acceleration	0.88
6	School	Classroom behavioral	0.80
7	Teaching	Comprehensive interventions for learning disabled students	0.77
8	Teacher	Teacher clarity	0.75
9	Teaching	Reciprocal teaching	0.74
10	Teaching	Feedback	0.73
11	Teacher	Teacher-student relationships	0.72
12	Teaching	Spaced vs. mass practice	0.71
13	Teaching	Meta-cognitive strategies	0.69
14	Student	Prior achievement	0.67
15	Curricula	Vocabulary programs	0.67
16	Curricula	Repeated reading programs	0.67
17	Curricula	Creativity programs	0.65
18	Teaching	Self-verbalization/self-questioning	0.64
19	Teacher	Professional development	0.62
20	Teaching	Problem-solving teaching	0.61
21	Teacher	Not Labeling students	0.61
22	Curricula	Phonics instruction	0.60
23	Teaching	Teaching strategies	0.60
24	Teaching	Cooperative vs. individualistic learning	0.59
25	Teaching	Study skills	0.59
26	Teaching	Direct Instruction	0.59
27	Curricula	Tactile stimulation programs	0.58
28	Curricula	Comprehension programs	0.58
29	Teaching	Mastery learning	0.58
30	Teaching	Worked examples	0.57
31	Home	Home environment	0.57
32	Home	Socioeconomic status	0.57
33	Teaching	Concept mapping	0.57
34	Teaching	Goals	0.56
35	Curricula	Visual-perception programs	0.55
36	Teaching	Peer tutoring	0.55
37	Teaching	Cooperative vs. competitive learning	0.54
38	Student	Pre-term birth weight	0.54
39	School	Classroom cohesion	0.53
40	Teaching	Keller's PIS	0.53
41	School	Peer influences	0.53
42	School	Classroom management	0.52
43	Curricula	Outdoor/adventure Programs	0.52
44	Teaching	Interactive video methods	0.52
45	Home	Parental involvement	0.51
46	Curricula	Play programs	0.50
47	Curricula	Second/third chance programs	0.50
48	School	Small group learning	0.49
49	Student	Concentration/persistence/engagement	0.48
50	School	School effects	0.48
51	Student	Motivation	0.48
52	Student	Early intervention	0.47
53	Teaching	Questioning	0.46
54	Curricula	Mathematics	0.45
55	Student	Preschool programs	0.45
56	Teacher	Quality of Teaching	0.44
57	Curricula	Writing Programs	0.44
58	Teacher	Expectations	0.43
59	School	School size	0.43
60	Student	Self-concept	0.43
61	Teaching	Behavioral organizers/Adjunct questions	0.41
62	Teaching	Matching style of learning	0.41
63	Teaching	Cooperative learning	0.41
64	Curricula	Science	0.40
65	Curricula	Social skills programs	0.40
66	Student	Reducing anxiety	0.40
67	Curricula	Integrated Curriculum Programs	0.39
68	School	Enrichment	0.39
69	Curricula	Career Interventions	0.38
70	Teaching	Time on Task	0.38
71	Teaching	Computer assisted instruction	0.37
72	Teaching	Adjunct aids	0.37
73	Curricula	Bilingual programs	0.37
74	School	Principals/ school leaders	0.36
75	Student	Attitude to mathematics/science	0.36
76	Curricula	Exposure to reading	0.36
77	Curricula	Drama/Arts programs	0.35
78	Student	Creativity	0.35
79	Teaching	Frequent/effects of testing	0.34
80	School	Decreasing disruptive behavior	0.34
81	Student	Drugs	0.33
82	Teaching	Simulations	0.33
83	Teaching	Inductive teaching	0.33
84	Student	Positive view of own ethnicity	0.32
85	Teacher	Teacher effects	0.32
86	Teaching	Inquiry based teaching	0.31
87	School	Ability grouping for gifted Students	0.30
88	Teaching	Homework	0.29
89	Home	Home visiting	0.29
90	Student	Exercise/relaxation	0.28
91	School	Desegregation	0.28
92	School	Mainstreaming	0.28
93	Curricula	Use of calculators	0.27
94	Curricula	Values/moral education programs	0.24
95	Teaching	Programmed instruction	0.24
96	Teaching	Special college programs	0.24
97	Teaching	Competitive vs. individualistic learning	0.24
98	School	Summer school	0.23
99	School	Finances	0.23
100	Teaching	Individualized instruction	0.23
101	School	Religious Schools	0.23
102	Student	Lack of Illness	0.23
103	Teaching	Teaching test taking	0.22
104	Teaching	Visual/audio-visual methods	0.22
105	Teaching	Comprehensive teaching reforms	0.22
106	School	Class size	0.21
107	School	Charter Schools	0.20
108	Teaching	Aptitude/treatment interactions	0.19
109	Student	Personality	0.19
110	Teaching	Learning hierarchies	0.19
111	Teaching	Co-team teaching	0.19
112	Teaching	Web-based learning	0.18
113	Home	Family structure	0.17
114	Curricula	Extra-curricular programs	0.17
115	Teaching	Teacher immediacy	0.16
116	School	Within class grouping	0.16
117	Teaching	Home-school programs	0.16
118	Teaching	Problem-based learning	0.15
119	Curricula	Sentence combining programs	0.15
120	Teaching	Mentoring	0.15
121	School	Ability grouping	0.12
122	Student	Gender	0.12
123	Student	Diet	0.12
124	Teacher	Teacher training	0.11
125	Teacher	Teacher subject matter knowledge	0.09
126	Teaching	Distance Education	0.09
127	School	Out of school curricula experiences	0.09
128	Curricula	Perceptual-Motor programs	0.08
129	Curricula	Whole language	0.06
130	School	College halls of residence	0.05
131	School	Multi-grade/age classes	0.04
132	Teaching	Student control over learning	0.04
133	School	Open vs. traditional	0.01
134	School	Summer vacation	-0.09
135	Home	Welfare policies	-0.12
136	School	Retention	-0.16
137	Home	Television	-0.18
138	School	Mobility	-0.34

Note: in Hattie’s 2016 publication, Learning strategies: a synthesis and conceptual model - he argued against ranking (p. 9)!

"There is much debate about the optimal strategies of learning, and indeed we identified >400 terms used to describe these strategies. Our initial aim was to rank the various strategies in terms of their effectiveness but this soon was abandoned. There was too much variability in the effectiveness of most strategies depending on when they were used during the learning process."

Hattie confirms this in his 2018 podcast with Ollie Lovell. Hattie said,

"it worked then it got misleading so I stopped it"

Listen - here - at 1 hour 21min 44secs. 

Hattie's new way of ranking with Corwin (March 2019):

Other Peer Reviews on Hattie's Rankings:

Wecker et al. (2017, p. 34)

"Hattie's ranking suggests that the 138 influencing factors are to be seen as basically alternative options for action, from which the most effective one is to be selected, for example, during lesson design. However, this would require that the influencing factors are in the first place real options for action and that their effect sizes were determined in relation to the same "baseline", ie. that the control conditions in the primary studies in all meta-analyzes included in the first step are to be regarded as an equivalent benchmark. A look at the ranking shows that this is neither the case nor is it possible at all, since the influencing factors belong to completely different types: On the one hand, one finds not only institutional framework conditions such as school size. As well as personal traits such as the self-concept. Such as those classified under "direct instruction". Effect sizes usually relate to a comparison with some form of "traditional" instruction... 

It is doubtful that teachers would ever usefully choose between direct instruction and distributed practice rather than the optimal combination of both to decide. If this is the case, the assessment of the relative effectiveness of these options cannot be based in one case on a comparison with an instructional "standard condition" and on the other hand on a comparison with another design variant of the same instruction form."

Prof Adrian Simpson (2017) is similarly critical of rankings in his detailed analysis, 

"The numerical summaries used to develop the toolkit (or the alternative ‘barometer of influences’: Hattie 2009) are not a measure of educational impact because larger numbers produced from this process are not indicative of larger educational impact. 

Instead, areas which rank highly in Marzano (1998), Hattie (2009) and Higgins et al. (2013) are those in which researchers can design more sensitive experiments. 

As such, using these ranked meta-meta-analyses to drive educational policy is misguided."

Schulmeister & Loviscach (2014),

"If one corrects the errors mentioned above, list positions take big leaps up or down.Even more concerning is the absurd precision this ranking conveys. It only shows the averages of effect sizes but not their considerable variation within every group formed by Hattie and even more so within every individual meta-analysis...

To think that didactics can be presented as a clear ranking order of effect sizes. It is a dangerous illusion. To an extreme degree, the effect of a specific intervention depends on the circumstances. Focusing on the mean effect sizes and ignoring their considerable variations and condensing the data to a seeming exact ranking order, Hattie pulls the wool over his audience’s eyes."

Dr. Jim Thornton (2018),

"To a medical researcher, it seems bonkers that Hattie combines all studies of the same intervention into a single effect size. Why should “sitting in rows”, for example, have the same effect on primary children as on university students, on maths as on art teaching, on behaviour outcomes as on knowledge outcomes? In medicine it would be like combining trials of steroids to treat rheumatoid arthritis, effective, with trials of steroids to treat pneumonia, harmful, and concluding that steroids have no effect! I keep expecting someone to tell me I’ve misread Hattie."

Claes Nilholm (2013) used 2 detailed examples to draw his conclusion (see problem-based learning and effect size),

"...Hattie draws far too far-reaching conclusions... he presents summative measurements in tabular form, and factors are ranked according to their significance. Unfortunately, SKL (Swedish Municipalities and County Council) ,puts great emphasis on these summing measures. I would like to say that the summaries, if used to guide teachers' work, can give utter incorrect implications" (p. 3).

McKnight & Whitburn (2018),

"Teachers will pick and choose from the list of “what works” that forms Visible Learning, even as their guts tell them externally mandated, evidence-based practices will not necessarily work for them (Biesta, 2007)" (p. 16).