Welfare & Diet

WELFARE POLICIES

Effect Size d= -0.12  (Hattie's Rank = 135)

Hattie relied on a singular meta-analysis, characterizing it with a negative effect size of -0.12, yet refrained from providing specific details about this study. Additionally, he presented the CLE probability statistic as -8%. It is noteworthy that peer reviews by Topphol (2011), Higgins & Simpson (2011), and Bergeron & Rivard (2017) have shown this is a significant mistake as probabilities cannot be represented as negative values.

Hattie and his commercial partner Corwin continue to ONLY use this ONE study with an effect size = -0.12, but have re-labeled "Welfare Policies" to "Family on welfare/state aid" (as of Jan 2024),

The change in category name, without explanation, is concerning given Hattie's presentations on Welfare Policies about ALL the different sorts of welfare strategies schools use. Many schools have taken Hattie's claims seriously and removed the focus or funding from welfare strategies, e.g. Noel Pearson (see below).

In addition, this ONE study (details below) has nothing to do with the sorts of welfare strategies schools use, so Hattie's polemic about Welfare Policies in his presentations is Irresponsible!

Examples of Hattie's Presentations

In Hattie's 2008 Nuthall lecture he posted this slide claiming Welfare Policies in schools has been a DISASTER!

Hattie in his 2013 TEDx talk,

"I can't find a single structural effect that's greater than 0.4 the majority of things we debate in Education don't matter much!" (@6 minutes)

Schools Use a Wide Range of Welfare Strategies-

Money given to families to help buy books, food, etc.

Pastoral Care Programs.

1-1 social worker and psychologist help.

Curriculum Design, e.g., the PERMA model and social/emotional learning.

Buddies & mentors.

Breakfasts & Lunches.

Scholarships.

Home Visits.

The ONE Study Hattie Used - Gennetian et al. (2004)

The ONLY policies they investigated were,

"designed to increase parental employment" (p. 404).

The key policies considered in this literature were (p. 401):

(1) work mandates, which require parents to work or to participate in education and training programs designed to enhance their employability;

(2) financial incentives, such as enhanced earned income disregards that allow working welfare recipients to keep more of their welfare benefits than under Aid to Families with Dependent Children (AFDC) or earnings supplements that are contingent on 30 hr or more of work per week; and

(3) time limits, which limit the length of time families can receive cash assistance. These policies are designed to increase maternal employment and family income, and reduce receipt of public assistance.

How Was Student Achievement Measured?

The achievement was measured at the 60-month follow-up (p. 405).

"This outcome was measured using maternal responses to the following question: 'Based on your knowledge of your child's schoolwork, including report cards, how has your child been doing in school overall?' Parents expressed their responses on a 5-point scale ranging from 1 (not well at all) to 5 (very well)" (p. 406).

The authors concede,

"Maternal reports of children's school performance are less than ideal" (p. 407).

RESULTS

In their discussion, the authors try to explain why the effect size is negative. Two issues seem to emerge: there is less parental supervision of homework and adolescents take on the parenting role of younger siblings leaving less time for studies. (p. 413)

So, this meta-analysis DOES NOT appear to be relevant to the many varied and different welfare policies schools use.

Hattie does not give any details of this study and continues with the misrepresentation and polemic as shown above. In VL he concluded,

"...it seems that there are other more powerful effects on achievement than the welfare status of the family." (p. 65)

Contradictions

The inclusion of Welfare is one of the many contradicting elements to Hattie's aim in VL where he stated,

"It is not a book about what cannot be influenced in schools- thus critical discussions about class, poverty, resources in families, health in families, and nutrition are not included-but this is NOT because they are unimportant, indeed they may be more important than many of the influences discussed in this book. It is just that I have not included these topics in my orbit." (VL, preface)

Contrast With Finland

The high-performing Finnish system as outlined by Pasi Sahlberg, Finnish Lessons 2.0,

"basic structures of the Finnish welfare state play a crucial role in providing all children and their families with equitable conditions for starting a successful educational path at the age of 7. Extended parental leave, comprehensive and preventive health care for all infants and their mothers, and systematic monitoring of children’s physical and mental development are accessible to everybody regardless of life circumstances or wealth. 

Early childhood education, voluntary free preschool that is attended by some 98% of six-year-olds, comprehensive health services, and preventive measures to identify possible learning and development difficulties before children start schooling are accessible to everyone. Finnish schools also provide each child with a free and healthy lunch every day' (p. 73).

'The two types of higher education institutions offer a place of study for about two-thirds of the age cohort. Because studying in Finnish universities and polytechnics is tuition-fee free, higher education is an equal opportunity for all those who have successfully completed upper-secondary education. The current challenge in Finnish higher education is to encourage students to complete their studies faster than they did before and thereby enter the labor market sooner. The government of Finland is introducing new conditions for financial aid for students that are encouraging students to graduate on time" (p. 69).

The Australian Education Union (AEU)

The AEU has negotiated and lobbied for increased funding for welfare programs. Victorian Vice President, Marino D'Ortenzio, in the 2019 February bulletin, outlined some achievements:

The expansion of the breakfast clubs from 500 to 1000 schools.

The inclusion of lunch in these programs.

Dental checkups at school, are now available for ALL Students.

Youth mental health workers present in secondary school at least 1 day/ week.

Regarding the welfare of the Principal and staff and the effect of increased workloads, the AEU has negotiated common templates for policies that all schools can use to decrease administrative load.

Increased access to mental health professionals for staff and students.

Australian Indigenous Leader Noel Pearson

An example of how seriously many people in Australia take Hattie's research is Indigenous Leader Noel Pearson, who wrote (The Australian -April 2011),

"the evidence of the impact of welfare reform policies is negative (-0.12). Yet, in our experience in Cape York, policies aimed at ensuring children are sitting in their seats are crucial in communities where school absenteeism is chronic. However, attendance is a necessary but insufficient condition for learning. If it is not accompanied by the factors crucial to learning then welfare reforms by themselves understandably will have no positive effect."

Pearson details his focus on Explicit Teaching and Phonics, much of which has been motivated by his reading of Hattie's VL and Engelmann's scripted lessons.

I have tried to contact Mr Pearson and implored him to check the study that Hattie cites and not simply rely on Hattie's interpretation. Mr Pearson has not replied.

Lives May be Damaged and Opportunities Missed.

Beng Huat See, after examining many of the studies cited by Hattie and the EEF in, Evaluating the evidence in evidence-based policy and practice: Examples from systematic reviews of literature, is very relevant here,

"This paper evaluates the quality of evidence behind some well-known education programmes using examples from previous reviews of over 5,000 studies on a range of topics. It shows that much of the evidence is weak, and fundamental flaws in research are not uncommon. This is a serious problem if teaching practices and important policy decisions are made based on such flawed evidence.

Lives may be damaged and opportunities missed."

A Leading Australian Teacher's View

Sarah Collins, insight-fully observes,

"He (Hattie) doesn't recognise that student learning will not happen if their well-being is adversely impacted. That welfare programs do not necessarily translate into improved well-being is well documented and without improved student well-being there will be no improvement in student engagement and learning. 

In other words he is not measuring what he thinks he is measuring.

It's sloppy in the extreme and quite clearly indicates he has never engaged with the research on well-being or welfare in any meaningful way.

But that's what he does with everything. Uses I'll defined terms and strips the context from nuanced and complex relationships and boils them down to a number."

DIET (now removed from Visible Learning, 2022)

Effect Size d = 0.12  (Hattie's Rank = 123)

One meta-analysis was used:
Karvale, K. & Forness, S., (1983) Hyperactivity and diet treatment. Journal of Learning Disabilities. 16(6), 324-330.

This paper is not measuring diet as it relates to improving student achievement. But rather diet modification as a treatment of hyperactivity. In addition, the research is only about a particular type of diet modification based on the hypothesis that, 

"the ingestion of artificial (synthetic) food additives (colours and flavours) and naturally occurring salicylates in foods results in hyperactivity and learning disabilities in children... It was suggested that treatment be based on the Feingold Kaiser-Permanente (K-P) diet which is designed to eliminate all foods containing natural salicylates and artificial food additives from the diet" (p. 324).

Hyperactivity was measured by a number of different tests: e.g., Connors' Scale parents, Connors' Scale teachers, attention, disruptive behaviour, impulsiveness, global improvement, learning ability and hyperkinesis. The researchers report (p. 327) a summary of effect sizes as listed below:

Hattie seems to have averaged all the categories except for 'Impulsivity' and reports an average, d = 0.12. There is no explanation about this in his commentary on the diet on pages 52-53 of VL.

So once again Hattie includes items that are not measuring achievement. So significant doubts are raised about the validity of this analysis in the context of VL - i.e., improving student achievement.

Dodiscimus (2014d) agrees,

"There is one meta-analysis, Kavale and Forness (1983). I can only access the abstract but it’s clear that despite the missing clause in Hattie’s summary, the meaning that I had assumed he intended does match this meta-analysis. Equally, it is clear that this is very specifically looking at ADHD and not children without this diagnosis. Essentially this paper states that the studies analysed do not provide evidence to support the earlier hypothesis that dietary changes could have a positive effect on ADHD symptoms. I’m guessing that the outcome measure was not academic achievement, but more likely some behavioural measure, which reminds me again that Hattie seems rather blasé about what his meta-analyses are measuring."