Other Researchers

The key researchers differ significantly when their top strategies are compared.

1. Hattie, Marzano, and Dylan Wiliam:

The most influential researchers differ and contradict each other in so many areas although formative assessment and feedback are common.

Hattie's Top Strategies 2009Hattie's Top Strategies 2017MarzanoProf Dylan William
1. Self-Reported GradesCollective Teacher EfficacyIdentify Similarities+DiffsFormative Assessment
2. Piagetian programsSelf-Reported GradesSumm + Note-takingFeedback
3. Formative evaluationTeacher estimates of achReinforcing Effort+RecognitionPeer Tutoring
4. Micro-teachingCognitive task analysisHomework and PracticeSelf-regulated learning
5. AccelerationResponse to interventionNonlinguistic RepresentationsDeliberate practice
6. Classroom behavioralPiagetian programsCooperative LearningTeacher Mindset
7. Intervent-learning disabledJigsaw methodSetting Objectives+FeedbackStaff as critical friends
8. Teacher clarityConceptual changeGenerating and Testing Hypoth's
9. Reciprocal teachingPrior AbilityCues+Advanced Orgs
10. FeedbackIntegrate prior knowledge

Hattie now admits his rankings are misleading! Listen to Ollie Lovell interview with Hattie in June 2018 (1hr 21min 45sec) here.

Marzano has been criticised by Dr J BeckerEllis Whyatt and Matthew Fitzpatrick for similar reasons to Hattie - misrepresentation, poor research and conflicts of interest.

I also think Professor Wiliam's strategy of influencing teacher's mindsets is important. Teachers must want to improve or everything else is secondary.

2. The U.S Dept of Education Recommendations:

The U.S Education Department fund a research group consisting of 8+ distinguished professors and 15+ PhD research scientists. They review research with a focus on the quality of research. So far nearly 12,000 individual educational research studies have been reviewed. Their detailed recommendations are here.

As already stated, most of Hattie's studies would fail the U.S. Educational Department quality standards.

It is significant that the U.S. Educational Department provide some caution in their recommendations with the 'Level of Evidence' rating. This is at odds with Hattie's confidence about his own conclusions. Also, one of the strong features of these practice guides is that they are subjected to rigorous external peer review (pvii). Hattie rarely responds to the critique from numerous peer reviews of his work.

U.S. Ed Dept Recommendations (p2):

U.S. Ed Dept Recommendation versus HattieLevel of EvidenceHattie's Rank
Space learning over time.Moderate70?
Interleave worked example with problem-solving exercises.Moderate20,30,118?
Combine graphics with verbal descriptions.Moderate104
Connect and integrate abstract and concrete representations.Moderate82?
Use pre-questions to introduce a new topic.Minimalnot ranked
Use quizzes to re-expose students to key content.Strongnot ranked
use delayed judgments of learning to identify.Minimalnot ranked
Tests and quizzes to identify content that needs to be learned.Minimal3?
Ask deep explanatory questions.Strong53?

A colleague of mine, Oliver Lovell, has used strategies closely related to the U.S recommendations to improve his Year 12 Maths class score by 1 standard deviation; quite an improvement! Details of what Ollie did here.

The U.S Dept of Education Maths specific Recommendations:

3. The Finnish System:

The Director-General Pasi Sahlberg outlines in Finnish Lessons 2.0, the priorities of the Finnish system, Teacher Training both in subject knowledge and didactics (p77).

Mathematics teaching is strongly embedded in curriculum design and teacher education in Finnish primary schools (p77).

A focus on welfare policies, e.g free healthy lunch for all children (p62).

Student's have choice, 
'Today, students build their own personalized learning schedules from a menu of courses offered in their school or by other education institutions. Studying in upper-secondary school is therefore flexible, and selected courses can be completed at a different pace depending on students’ abilities and life situations' (p87).
Systematic counselling and career guidance (p87).

School Autonomy,
'little interference by the central education administration in schools’ everyday lives' (p88).
Less instructional time and more time for play and recreation (p91).

Less teacher workload,
'Lower-secondary teachers’ total weekly working time in Finland was 31.6 hours; that is significantly less than in Australia (42.7 hr), the United States (44.8), England (45.9), Singapore (47.6), Alberta (48.2), or in the surveyed 34 countries on average (38.3)' (p91).
Time for staff collaboration,
'... teaching is a holistic profession that combines work with students in the classroom and collaboration with colleagues in the staff room' (p93).
Teacher-designed curricula (p99). 

Systematic care for students with diverse special needs (p99).

4. The Education Endowment Foundation (EEF):

Is an independent English organisation with a funding of around $250 Million. It has 5 full-time researchers and has published an evidence guide mostly based on good quality Randomised Controlled Studies (RCTs) similar to the US Education Dept, but adds a helpful costing feature.

It has a very different time scale to Hattie. An effect size of about d = 0.1 equivalent to a months progress. In contrast, Hattie has an effect size of 0.4 equivalent to 12 months progress.

Their ranking of influences is very different to Hattie's - click here for full list:

The Australian version of the EEF can be found here.

5. The Sutton Trust:

With a focus on good quality research, the two factors with the strongest evidence of improving pupil attainment are:

a. Teachers’ content knowledge, including their ability to understand how students think about a subject and identify common misconceptions - a direct contradiction to Hattie's work where this is ranked #125 with a small effect size = 0.09).

b. The quality of instruction, which includes using strategies like effective questioning and the use of assessment.

6. The Deans For Impact:

They produced a document The Science of Learning which details the key strategies for effective teaching, e.g.,

a. Students learn new ideas by reference to ideas they already know.

b. Information is often withdrawn from memory just as it went in. We usually want students to remember what information means and why it is important, so they should think about meaning when they encounter to-be-remembered material.

c. Practice is essential to learning new facts, but not all practice is equivalent.

d. Self-determined motivation (a consequence of values or pure interest) leads to better long-term
outcomes than controlled motivation (a consequence of reward/punishment or perceptions of self-worth).

e. Effective feedback is often essential to acquiring new knowledge and skills.

7. Professor Jo Boaler:

Jo Boaler's focus is on visual representations of abstract ideas which seems to directly contradict Hattie's rankings where 'simulations' and 'visual representations' are ranked very lowly.

And Jo Boaler's example of simulations:

Although Boaler's use of the mindset research has been criticised see here.

8. Professor Michael Fullan:

In ACEL Monograph 52 Fullan states,
'there was one finding that stood out as twice as powerful as any other factor in 'effect size' - principals who participated as learners working with teachers to make improvements had twice the impact on school-wide student achievement compared to any other factor' (p3).
Note that Fullan is a leadership consultant with a focus on Leadership.

9. The Grattan Institute:

Professor Wiliam also provides some extra useful strategies on the teacher. He suggests teachers must want to improve and teachers need to act as critical friends. The Grattan Institute analysed the high performing international educational systems and concluded that one of the reforms responsible for improving student achievement across the four high-performing education systems in East Asia was teachers acting as critical friends. Note the amount of time devoted to feedback, lesson planning, etc. Although, one caveat in these systems is that class sizes are higher (Grattan Report p14).

10. PISA 2015:

David Didau in his excellent blog on feedback pointed out that even though most researchers have feedback as an important teaching strategy, PISA has feedback NEGATIVELY correlated with Science performance.

The Negative Influences

From PISA 2015 Volume 2 (page 228)

11. The Victorian Education Dept High Impact Strategies (HITS):

These are largely based on Hattie and Marzano and can be found here.

12. Student Research on What makes a great teacher:

Azul Terronez surveyed 26,000 students and found great teachers:

1. build positive relationships.
2. are chilled.
3. are good listeners.
4. love to learn.
5. knows kids have a life outside of school.
6. notice if kids struggle.
7. sings!
8. are humble and take risks.

See his TEDx talk here.

The Australian Government Productivity Commission.

In their latest report (2017) they make a number of recommendations regarding educational research.

Firstly, a focus on the quality of research:

'... the gold standard techniques are meta-analyses of randomised controlled trials and individual trials. Such approaches are widely used in health research, but are not routinely used in Australian education research' (p17).

Verifying the quality of the research (p19):

'A range of processes can be used to ensure the findings from completed research are robust. These include independent validation of the findings, peer review of research, publication of all outputs to enable scrutiny and debate (irrespective of findings), and the provision of project data for secondary analysis.'

Their recommendations (p28):

In assessing whether to improve the quality of existing education data, governments should examine on a case by case basis whether:
the existing quality of the data is fit for purpose
data quality improvements are feasible given the context of data collection
other options are available
the benefits of improving data quality exceed the costs.

The Australian Government should request and sufficiently fund the agencies that conduct the Longitudinal Study of Australian Children to establish new cohorts at regular intervals. 

U.S. Ed Dept Levels of Evidence guide:

The US Education department state, 'In classifying levels of empirical support for the effectiveness of our recommendations, we have been mindful not only to the issue of whether a study meets the “gold-standard” of a randomized trial but also to the question “Effective as compared to what?” Virtually any educational manipulation that involves exposing students to subject content, regardless of how this exposure is provided, is likely to provide some benefit when compared against no exposure at all. To recommend it, however, the question becomes “Is it more effective than the alternative it would likely replace?” In laboratory studies, the nature of instruction in the control group is usually quite well defined, but in classroom studies, it is often much less clear. In assessing classroom studies, we have placed the most value on studies that involve a baseline that seems reasonably likely to approximate what might be the ordinary practice default' (p3).

Full Guide here https://www2.ed.gov/policy/elsec/leg/essa/guidanceuseseinvestment.pdf

No comments:

Post a Comment