Diet & Welfare Policies


Effect Size d= -0.12  (Hattie's Rank = 135)

Hattie used 1 meta-analysis which seemed to give a negative effect size of -0.12. Note the CLE probability statistic of -8% is wrong as probabilities cannot be negative.

Hattie and his commercial partner Corwin continue to publish this result (downloaded Feb 2020):

In Hattie's 2008 Nuthall lecture he posted this slide claiming a focus on Welfare in schools has been a DISASTER!

Hattie in his 2013 TEDx talk,
"I can't find a single structural effect that's greater than 0.4 the majority of things we debate in Education don't matter much!" (@6 minutes)

Yet in Hattie's conclusion in his 2015 defense (p. 8), he is a lot more cautious,
"The main message remains, be cautious, interpret in light of the evidence, search for moderators, take care in developing stories, welcome critique, ..."
Sarah Collins, insight-fully observes,
"He is quite clearly conflating well-being with welfare. He doesn't recognise that student learning will not happen if their well-being is adversely impacted. That welfare programs do not necessarily translate into improved well-being is well documented and without improved student well-being there will be no improvement in student engagement and learning. 
In other words he is not measuring what he thinks he is measuring.

It's sloppy in the extreme and quite clearly indicates he has never engaged with the research on well-being or welfare in any meaningful way.

But that's what he does with everything. Uses I'll defined terms and strips the context from nuanced and complex relationships and boils them down to a number."
Schools use a wide range of welfare policies:

From, money given to families to help buy books, food, etc,

1-1 social worker and psychologist help

Curriculum Design, e.g., the PERMA model and social/emotional learning, 

Buddies, mentors, etc.

Many schools interpret the above result as welfare policies, IN GENERAL, do not improve student achievement. Once again this needs to be looked at in greater detail.

Hattie's only comment (p. 65) on this influence is that the 1- meta-analysis used found,
"close to zero effects from students in families who received welfare compared to those not receiving welfare. While they make much of an effect size of d = –0.10, by claiming that the effects on adolescents were “significantly worse”, it is difficult to imagine the visible effects of findings such as about four percent fewer mothers in the welfare program group reporting that their child performed above average, and only about two percent more of this group of mothers indicating that their child repeated a grade. There are certainly many other effects of welfare programs for these families that are beneficial, but it seems that there are other more powerful effects on achievement than the welfare status of the family."
What Were the Policies in this Research?

The policies investigated were ones which were 
"designed to increase parental employment" (p. 404).
The key policies considered in this literature were (p. 401):
(1) work mandates, which require parents to work or to participate in education and training programs designed to enhance their employability;
(2) financial incentives, such as enhanced earned income disregards that allow working welfare recipients to keep more of their welfare benefits than under Aid to Families with Dependent Children (AFDC) or earnings supplements that are contingent on 30 hr or more of work per week; and
(3) time limits, which limit the length of time families can receive cash assistance. These policies are designed to increase maternal employment and family income, and reduce receipt of public assistance.

The achievement was measured at around the 60 months follow up (p. 405).
"This outcome was measured using maternal responses to the following question: 'Based on your knowledge of your child's schoolwork, including report cards, how has your child been doing in school overall?' Mothers expressed their responses on a 5-point scale ranging from 1 (not well at all) to 5 (very well)" (p. 406).
The authors concede,
"Maternal reports of children's school performance are less than ideal" (p. 407).


In their discussion (p413), the authors try to explain why the results are negative. Two issues seem to emerge: there is less parental supervision of homework, etc. and adolescents take on the parenting role of younger siblings leaving less time for studies.

In any case, this meta-analysis does not appear to be relevant to the many varied and different welfare policies schools use.

Contrast the so-called welfare policies above with the high performing Finnish system as outlined by Director General Pasi Sahlberg, Finnish Lessons 2.0,
"basic structures of the Finnish welfare state play a crucial role in providing all children and their families with equitable conditions for starting a successful educational path at the age of 7. Extended parental leave, comprehensive and preventive health care for all infants and their mothers, and systematic monitoring of children’s physical and mental development are accessible to everybody regardless of life circumstances or wealth. 
Early childhood education, voluntary free preschool that is attended by some 98% of six-year-olds, comprehensive health services, and preventive measures to identify possible learning and development difficulties before children start schooling are accessible to everyone. Finnish schools also provide each child with a free and healthy lunch every day' (p. 73).
'The two types of higher education institutions offer a place of study for about two-thirds of the age cohort. Because studying in Finnish universities and polytechnics is tuition-fee free, higher education is an equal opportunity for all those who have successfully completed upper-secondary education. The current challenge in Finnish higher education is to encourage students to complete their studies faster than they did before and thereby enter the labor market sooner. The government of Finland is introducing new conditions for financial aid for students that are encouraging students to graduate on time" (p. 69).
The Australian Education Union (AEU) has negotiated and lobbied for increased funding for welfare programs. Victorian President Marino D'Ortenzio, in the 2019 February bulletin, outlined some achievements:

The expansion of the breakfast clubs from 500 to 1000 schools.

The inclusion of lunch to these programs.

Dental checkups at school, now available for ALL Students.

Youth mental health workers present in secondary school at least 1 day/ week.

Regarding the welfare of Principal's and staff and the effect of increased workloads the AEU has negotiated common templates for policies that all schools can use to decrease administrative load.

Increased access to mental health professionals for staff and students.

For Hattie to rank the wide range of welfare policies lowly, based on this 1 irrelevant meta-analysis, and directly say that the focus of welfare policies in Schools has been a 'DISASTER' is extremely disappointing.

Beng Huat See conclusion in, ‘Evaluating the evidence in evidence-based policy and practice: Examples from systematic reviews of literature', is very relevant here,
"This paper evaluates the quality of evidence behind some well-known education programmes using examples from previous reviews of over 5,000 studies on a range of topics. It shows that much of the evidence is weak, and fundamental flaws in research are not uncommon. This is a serious problem if teaching practices and important policy decisions are made based on such flawed evidence.
Lives may be damaged and opportunities missed."
An example of how seriously many people in Australia take Hattie's research is Indigenous Leader, Noel Pearson, who wrote (The Australian -April 2011),
"the evidence of the impact of welfare reform policies is negative (-0.12). Yet, in our experience in Cape York, policies aimed at ensuring children are sitting in their seats are crucial in communities where school absenteeism is chronic. However, attendance is a necessary but insufficient condition for learning. If it is not accompanied by the factors crucial to learning then welfare reforms by themselves understandably will have no positive effect."


Effect Size d = 0.12  (Hattie's Rank = 123)

One meta-analysis was used:
Karvale, K. & Forness, S., (1983) Hyperactivity and diet treatment. Journal of Learning Disabilities. 16(6), 324-330.

This paper is not measuring diet as it relates to improving student achievement. But rather diet modification as a treatment of hyperactivity. In addition, the research is only about a particular type of diet modification based on the hypothesis that, 
'the ingestion of artificial (synthetic) food additives (colours and flavours) and naturally occurring salicylates in foods results in hyperactivity and learning disabilities in children... It was suggested that treatment be based on the Feingold Kaiser-Permanente (K-P) diet which is designed to eliminate all foods containing natural salicylates and artificial food additives from the diet' (p. 324).
Hyperactivity was measured by a number of different tests: e.g., Connors' Scale parentsConnors' Scale teachers, attention, disruptive behaviour, impulsiveness, global improvement, learning ability and hyperkinesis. The researchers report (p. 327) a summary of effect sizes as listed below:

Hattie seems to have averaged all the categories except for 'Impulsivity' and reports an average, d = 0.12. There is no explanation about this in his commentary on the diet on pages 52-53 of VL.

So once again Hattie includes items that are not measuring achievement. So significant doubts are raised about the validity of this analysis in the context of VL - i.e., improving student achievement.

From docendo:
'There is one meta-analysis, Kavale and Forness (1983). I can only access the abstract but it’s clear that despite the missing clause in Hattie’s summary, the meaning that I had assumed he intended does match this meta-analysis. Equally, it is clear that this is very specifically looking at ADHD and not children without this diagnosis. Essentially this paper states that the studies analysed do not provide evidence to support the earlier hypothesis that dietary changes could have a positive effect on ADHD symptoms. I’m guessing that the outcome measure was not academic achievement, but more likely some behavioural measure, which reminds me again that Hattie seems rather blasé about what his meta-analyses are measuring.'

No comments:

Post a comment