Hattie has omitted a category termed "Teacher Certification" from his compilation of influences in Visible Learning. Despite referencing several studies related to this topic under various categories, such as placing Hacke (2010) under "Teacher Training," this absence raises the question: why?
This omission is notable, especially considering Hattie's advocacy for the USA's National Board Certified Teachers (NBC) and Australia's Highly Accomplished Lead Teacher (HALT) programs, which he aligns with Teacher Expertise and connects to performance-based pay. Thus, understanding the rationale behind Hattie's decision becomes pertinent for educators seeking insight into the background of these initiatives.
Hattie's Background Regarding Certification
Hattie stated in his 2016 HALT address that he and others had set up the assessments for the USA's NBC with funding of US $80 million per year - (@2mins)
Harris and Sass (2007) report that the NBPTS who administer the NBC generates around $600 million in fees from teachers each year (p. 4)
During the recent 2022 debate about teacher shortages in Australia, Hattie's HALT certification has been presented as a solution by the Federal Government working party & Hattie has also been contracted by the NSW Government to provide solutions to the teacher shortage problem.
Update February 2023: Angelo Gavrielatos, head of The NSW Teacher Federation, tweeted NSW teachers with HALT certification will get a $4,000 cash bonus - details here.
However, later in May 2023, the new Education minister Prue Car, confirmed the new government would dump the previous Coalition’s rewarding excellence program, led by education academic John Hattie, which planned to give a selection of top teachers super-salaries.
"We are not about paying a very small percentage of certain teachers significantly more than others. We will be negotiating to provide greater opportunities for all teachers across the board in salaries, conditions and workload," Car said. Report here.
Juxtapose Hattie's Narrative About Teacher Training in Visible Learning With His Narrative About Certification (NBC)
The certification studies Hattie cites have very low ES, e.g., Hacke (2010) who compared NBC vs Non-NBC teachers using Student Achievement gets a very low ES = 0.09. However, as previously stated, Hattie misrepresents this study by placing it in his Teacher Training category.
Hattie's usual narrative is polemic when categories have low ES, e.g., "bankrupt", "don't matter much", "trivial", "distractions" & "disasters" (see a range of Hattie's presentations here).
However, in VL Chapter 11 regarding Expert vs Experienced Teachers using NBC certification, Hattie removes his point of reference, Student Achievement, from the ES calculations.
Hattie used his small NBC study - Bond, Smith, Baker, Hattie, Jaeger & Strahan (2000) Preliminary Analysis Report: Construct Validity Study of the National Board for Professional Teaching Standards.
This validation study analyzed 65 teachers, who appear to have volunteered or have been coerced. The study also mentions another 80 are telephone interviewed, but these were not included in the analysis and no reason was given as to why.
Yet, in his 2022 address promoting the Australian certificate equivalent - HALT, Hattie exaggerated that 400-500 teachers had been thoroughly analyzed in this study - see here.
Also, around the same time that Visible Learning was published (2008), Hattie & Ingvarson edited a book promoting the NBC:
Ingvarson & Hattie (2008). Assessing Teachers for Professional Certification : The First Decade of the National Board for Professional Teaching Standards, Vol 1st Ed, Advances in Program Evaluation, JAI Press Inc, Amsterdam.
Hattie & Ingvarson also reprinted a revised version of his earlier validation study of 2000, as well as a chapter with Hattie's wife Janet Clinton, Identifying accomplished teachers: A validation study.
So these publications make it clear that Hattie was well aware of the Certification in USA & the difference between this and University Degrees.
Then a reasonable question is why were these studies reported in his Teacher Training/ Education category when they have nothing to do with the University Degrees that are required for teaching?
In addition, he has often made vitriolic statements, based on the low ES of these studies, e.g.,
"Teacher Education, as we most commonly know it, is the most bankrupt institution I know." (2011 Melbourne Graduate School address @22mins)
Note: Hattie has recently removed all the studies on Certification from his references.
Student Achievement - To Use or Not to Use That is the Question!
Contradicting Hattie's focus in Visible Learning, Hattie & colleagues in their validation study in 2000 argue NOT to use Student Achievement as a measure of certification and provide the following rationale,
'We appreciate the intense interest in the question "Do students who have been instructed by National Board Certified teachers obtain higher scores on standardized measures of academic achievement than students who have not been taught by NBCTs?" Nevertheless, a simple comparison of mean scores on standardized achievement tests of students who have been instructed over the past academic year by NBCTs versus those who were not is misguided. First, such a comparison ignores other powerful influences on standardized measures such as socio-economic status. Even if one could control for such effects, a simple mean comparison would still be inappropriate. It is simply unrealistic to expect student performance on off-the-shelf, multiple-choice achievement tests to be measurably sensitive to differences between teachers in a single academic year. This is especially so since the relationship between what teachers are teaching and the content of various standardized tests varies widely from state to state, district to district, school to school, and indeed from classroom to classroom.' (p. 7-8)
Then in the chapter with his wife, Janet Clinton, they also argue not to use Student Achievement,
"An obvious and simple method would be to investigate the effects of passing and failing NBPTS teachers on student test scores. Despite the simplicity of this notion, we do not support this approach... student test scores depend on multiple factors, many of which are out of the control of the teacher." (p. 319)
Yet, Hattie in Visible Learning states a contrary focus,
"This book is based on a synthesis (a method referred to by some as meta-meta-analysis) of more than 800 meta-analyses about influences on learning that have now been completed... the various innovations in these meta-analyses can be ranked from very positive to very negative effects on student achievement." (VL, p.3)"This book focuses on student achievement, and that is a limitation of this review." (VL, p.6)
Hattie magnifies this focus by constantly blaming the teacher regarding lack of Student Achievement,
"You had him for a year and You failed." Hattie (2017). ResearchEd, Melbourne @31mins.
Yet, Hattie & his colleague's in their validation study in 2000, argue for 14 other criteria, instead of Student Achievement.
Their 1st criteria is Content Knowledge (p. 7) which directly contradicts Hattie's result in Visible Learning where Subject Matter Knowledge has a very low ES that Hattie often belittles, e.g., Hattie 2011 AITSL address,
Dimensions like "passion" were measured with a Likert Scale (p. 10).
Predictably this has come under much critique, e.g. Podgursky (2001) describes these as,
"Nebulous Standards."
Further their comparison of student work samples contradict the fundamental method of Visible Learning in comparing average effect sizes of VERY different forms of assessment,
"The approximately 260 student work samples produced in response to assignments made during the observed unit of instruction will be scored using the SOLO taxonomy. Unfortunately, a straightforward comparison of mean scores on the SOLO taxonomy for students taught by NBCTs versus those taught by non-NBCTs would be misleading. This is so because the nature of student products depends critically upon the nature of the assignment given by the teacher. " (p. 14)
USA - Teacher Experience with NBC
A number of teachers document their experience with NBC and it sounds out a warning to Australian Teachers, e.g., Michael Pershan -This is my post critiquing National Board Certification for Teaching summarises,
"If a teacher tells me that they are NBCT, I think I know something about that teacher. They’re hard-working, because NBCT is a lot of work. They are likely ambitious, probably not on their way out of the profession.All this I know because NBCT was a ton of work. I can’t imagine a teacher going through this without something pushing them — either a financial incentive or something internal.So I know they’re hard-working and committed to teaching, but that’s pretty much all that I know. Nothing about the NBCT process gives me any confidence that it was assessing the quality of my teaching in any sense at all."
The Australian Equivalent of the NBC - the HALT
Hattie for over 10 years, has headed The Australian Institute for Teaching and School Leadership (AITSL), and has promoted its certificate, the HALT.
Eacott, reports the low uptake of this certificate of only around 1000 out of >300,000 teachers since 2012.
Taylor (2021) confirms Eacott's analysis and adds,
"Perhaps what is most remarkable in this evolution of the profession in Australia is, firstly, how seduced the profession is by the Standards and secondly, how little use is made of them at the higher levels...Why are we so seduced by things that do not really seem to make much of a difference?" (abstract)
My experience is the cost and time for this certificate is not worth it, so the uptake by teachers is small - see Hacke (2010) in Teacher Training for the NBC experience.
The 2022 NSW Parliamentary Inquiry also mentioned the problems with HALT,
"Highly Accomplished and Lead Teacher (HALT) accreditation has been inadequate, with only a small number of teachers applying and qualifying each year. The additional data collection and paperwork and the requirement to self-advocate serve to disincentivise teachers from pursuing this higher accreditation." (p. 17)
Although, "the committee divided" was reported over 90 times in their many proposals.
Hattie's Arbitrary Interpretations of NBC Research (VL, Ch 11):
As already stated, Hattie used:
Smith, Wanda, Baker, Hattie & Bond (2008). Chapter 12 A validity study of the certification system of the National Board for Professional Teaching Standards
Bond et al. (2000) make the arbitrary judgement that NBC certified teachers are 'Experienced Experts' while Non-NBC teachers are 'Experienced Non Experts' (VL, p. 259).
Pool, Ellett, Schiavone &Carey-Lewis (2001) in How Valid are the National Board of Professional Teaching Standards..., investigate the claims made by Bond et al. (2000) and conclude,
"...there appear to be no assurances that attaining these credentials guarantees quality teaching and learning environments in everyday practice." (p. 46)
Podgursky (2001) in his critique of (Bond et al. (2000)) verifies this,
Greg Ashman wrote about the meaningless Teacher Standards in Australia - here.
'No study, however, has ever shown that National Board-certified teachers are any better than other teachers at raising student achievement. Nothing has changed with the release of this report. The National Board’s researchers rejected the use of student test scores as a measure of teacher performance, claiming, “It is not too much of an exaggeration to state that such measures have been cited as a cause of all of the nation’s considerable problems in educating our youth. . . . It is in their uses as measures of individual teacher effectiveness and quality that such measures are particularly inappropriate.”' (p. 2).
Cunningham & Stone (2005) in Value-added assessment of teacher quality as an alternative to the National Board for Professional Teaching Standards: What recent studies say, conclude,
"...questions have persisted regarding the worth of NBPTS certification. Thus far, studies have shown that it is more clearly an indicator of teacher commitment to NBPTS’s Five Core Propositions (National Board for Professional Teaching Standards, 2005) than it is a predictor of improved student achievement.The underlying problem seems to be that NBPTS’s favored teaching style not well suited to the realization of the public’s primary policy objective: improvement in objectively measured student achievement." (p. 13)
Hattie & colleagues in Bond et al. (2000) conclude that expert teachers (NBC) outperform Non-NBC teachers on almost every criterion (VL, p. 260).
Harris and Sass (2007) The Effects of NBPTS-Certified Teachers on Student Achievement was another study just investigating NBC vs Non-NBC.
Harris and Sass (2007) conclude, similar to Hacke (2010),
"we find relatively little support for NBPTS certification as a signal of teacher effectiveness. In general we find that prior to certification, future NBCTs are no more effective in raising student test scores than are other teachers who are never observed to become NBCTs." (p. 25)
Yet, Hattie & Ingvarson (2008) claims the opposite conclusion from Harris & Sass (p. 10),
The What works Clearing House (WWC)
The largest educational evidence organisation, the USA's WWC, also produced an intervention report in 2018: The National Board for Professional Teaching Standards (NBPTS) certification.
The WWC has the highest quality standards and concluded,
"NBPTS-certified teachers had mixed effects on mathematics achievement and no discernible effects on English language arts achievement for students in grades 3 through 8." (p. 1)
Dylan Wiliam: Teacher quality: What it is, why it matters and how to get more of it.
Wiliam makes 2 important claims & provides significant evidence to support these:
1. We can’t identify good teachers by observing them.
2. We can’t identify good teachers from changes in test scores.
Wiliam concludes,
"a counterargument is often raised: how can we improve teachers if we don’t know how good they are? The answer is this: if we think of teacher quality as a continuum, we now know that we are unable to locate a particular teacher along that continuum with any accuracy, but we do know which way is better. If we direct our energies not on evaluating teachers but on improving them, we are far more likely to improve the quality of education that our pupils receive."
No comments:
Post a Comment