Since around 2000, Hattie has been involved in designing and advocating for Teacher Certification Programs, such as the USA's National Board Certified Teachers (NBC) and Australia's Highly Accomplished Lead Teacher (HALT). Interestingly, despite referencing several studies on this topic, he did not include "Teacher Certification" as a category in his compilation of influences in Visible Learning. Instead, these studies were categorized under "Teacher Training/Education."
As of January 2025, category names have evolved. "Teacher Training" has changed to "Initial Teacher Education," and a separate category of "Teacher Qualifications" has been added, but "Teacher Certification" is still missing.
The nuance in these category names is significant, as many certification studies are included in the "Initial Teacher Education" category.
Given Hattie's advocacy for NBC and HALT programs, it would be logical to create a distinct "Teacher Certification" category. This would allow for a consistent comparison of effect sizes for certification with other educational influences, adhering to Hattie's methodology.
This is the focus of my discussion below.
Hattie's Background Regarding Certification
Hattie stated in his 2016 HALT address that he and others had set up the assessments for the USA's NBC with funding of US $80 million per year - (@2mins)
Harris and Sass (2007) report that the NBPTS who administer the NBC generates around $600 million in fees from teachers each year (p. 4)
During the recent 2022 debate about teacher shortages in Australia, Hattie's HALT certification has been presented as a solution by the Federal Government working party & Hattie has also been contracted by the NSW Government to provide solutions to the teacher shortage problem.
Update February 2023: Angelo Gavrielatos, head of The NSW Teacher Federation, tweeted NSW teachers with HALT certification will get a $4,000 cash bonus - details here.
However, later in May 2023, the new Education minister Prue Car, confirmed the new government would dump the previous Coalition’s rewarding excellence program, led by education academic John Hattie, which planned to give a selection of top teachers super-salaries.
"We are not about paying a very small percentage of certain teachers significantly more than others. We will be negotiating to provide greater opportunities for all teachers across the board in salaries, conditions and workload," Car said. Report here.
Juxtapose Hattie's Narrative About Teacher Training in Visible Learning With His Narrative About Certification (NBC)
The certification studies that Hattie cites have very low effect sizes (ES). For example, Hacke (2010) compared NBC vs. Non-NBC teachers using Student Achievement and found an ES of just 0.09. Hattie placed this study in his "Teacher Training" category, but I believe it would be more consistent to place it in a separate "Teacher Certification" category.
Hattie's usual narrative is polemic when categories have low ES, e.g., "bankrupt", "don't matter much", "trivial", "distractions" & "disasters" (see a range of Hattie's presentations here).
However, in VL Chapter 11 regarding Expert vs Experienced Teachers using NBC certification, Hattie removes his point of reference, Student Achievement, from the ES calculations.
Hattie used his small NBC study - Bond, Smith, Baker, Hattie, Jaeger & Strahan (2000) Preliminary Analysis Report: Construct Validity Study of the National Board for Professional Teaching Standards.
This validation study analyzed 65 teachers, who appear to have volunteered. The study also mentions another 80 telephone interviews, but these were not included in the analysis and no reason was given as to why.
However, in his 2022 address promoting the Australian equivalent certificate, HALT, Hattie seems to overstate that 400-500 teachers were thoroughly analyzed in this study - here.
Additionally, in 2008, the same year that Visible Learning was published, Hattie and Ingvarson edited a book advocating for the NBC.
Ingvarson & Hattie (2008). Assessing Teachers for Professional Certification : The First Decade of the National Board for Professional Teaching Standards, Vol 1st Ed, Advances in Program Evaluation, JAI Press Inc, Amsterdam.
Hattie & Ingvarson also reprinted a revised version of Hattie's earlier validation study of 2000, as well as a chapter with Hattie's wife Janet Clinton, Identifying accomplished teachers: A validation study.
So these publications make it clear that Hattie was well aware of the Certification in USA & the difference between this and University Degrees & Initial Teacher Training or Education.
A reasonable question to ask is why these studies were included in his "Teacher Training/Education" category when they are unrelated to the university degrees required for teaching.
In addition, he has often made vitriolic statements, based on the low ES of these studies, e.g.,
"Teacher Education, as we most commonly know it, is the most bankrupt institution I know." (2011 Melbourne Graduate School address @22mins)
Student Achievement - To Use or Not to Use That is the Question!
Contradicting Hattie's focus on Student Achievement in Visible Learning, Hattie & colleagues in their validation study in 2000 argue NOT to use Student Achievement as a measure of certification and provide the following rationale,
'We appreciate the intense interest in the question "Do students who have been instructed by National Board Certified teachers obtain higher scores on standardized measures of academic achievement than students who have not been taught by NBCTs?" Nevertheless, a simple comparison of mean scores on standardized achievement tests of students who have been instructed over the past academic year by NBCTs versus those who were not is misguided. First, such a comparison ignores other powerful influences on standardized measures such as socio-economic status. Even if one could control for such effects, a simple mean comparison would still be inappropriate. It is simply unrealistic to expect student performance on off-the-shelf, multiple-choice achievement tests to be measurably sensitive to differences between teachers in a single academic year. This is especially so since the relationship between what teachers are teaching and the content of various standardized tests varies widely from state to state, district to district, school to school, and indeed from classroom to classroom.' (p. 7-8)
Then in the chapter with his wife, Janet Clinton, they also argue not to use Student Achievement,
"An obvious and simple method would be to investigate the effects of passing and failing NBPTS teachers on student test scores. Despite the simplicity of this notion, we do not support this approach... student test scores depend on multiple factors, many of which are out of the control of the teacher." (p. 319)
Yet, Hattie in Visible Learning states a contrary focus,
"This book is based on a synthesis (a method referred to by some as meta-meta-analysis) of more than 800 meta-analyses about influences on learning that have now been completed... the various innovations in these meta-analyses can be ranked from very positive to very negative effects on student achievement." (VL, p. 3)"This book focuses on student achievement, and that is a limitation of this review." (VL, p. 6)
Hattie magnifies this focus by constantly blaming the teacher for the lack of Student Achievement,
"You had him for a year and You failed." Hattie (2017). ResearchEd, Melbourne @31mins.
Yet, Hattie & colleague's in their validation study in 2000, argue for 14 other criteria, instead of Student Achievement.
Their 1st criterion is Content Knowledge (p. 7) which directly contradicts Hattie's result in Visible Learning where Subject Matter Knowledge has a very low ES that Hattie often belittles, e.g., Hattie 2011 AITSL address,
Dimensions like "passion" were measured with a Likert Scale (p. 10).
Predictably this has many critiques, e.g., Podgursky (2001) describes these as,
"Nebulous Standards."
Further, their comparison of student work samples contradicts the fundamental method of Visible Learning in comparing average effect sizes of VERY different forms of assessment,
"The approximately 260 student work samples produced in response to assignments made during the observed unit of instruction will be scored using the SOLO taxonomy. Unfortunately, a straightforward comparison of mean scores on the SOLO taxonomy for students taught by NBCTs versus those taught by non-NBCTs would be misleading. This is so because the nature of student products depends critically upon the nature of the assignment given by the teacher. " (p. 14)
USA - Teacher Experience with NBC
Several teachers document their experience with NBC and it sounds like a warning to Australian Teachers, e.g., Michael Pershan -This is my post critiquing National Board Certification for Teaching summarises,
"If a teacher tells me that they are NBCT, I think I know something about that teacher. They’re hard-working, because NBCT is a lot of work. They are likely ambitious, probably not on their way out of the profession.All this I know because NBCT was a ton of work. I can’t imagine a teacher going through this without something pushing them — either a financial incentive or something internal.So I know they’re hard-working and committed to teaching, but that’s pretty much all that I know. Nothing about the NBCT process gives me any confidence that it was assessing the quality of my teaching in any sense at all."
The Australian Equivalent of the NBC - the HALT
Hattie for over 10 years, has headed The Australian Institute for Teaching and School Leadership (AITSL) and has promoted its HALT certificate.
Eacott, reports the low uptake of this certificate of only around 1000 out of >300,000 teachers since 2012.
Taylor (2021) confirms Eacott's analysis and adds,
"Perhaps what is most remarkable in this evolution of the profession in Australia is, firstly, how seduced the profession is by the Standards and secondly, how little use is made of them at the higher levels...Why are we so seduced by things that do not really seem to make much of a difference?" (abstract)
My experience is the cost and time for this certificate are not worth it, so the uptake by teachers is small - see Hacke (2010) in Teacher Training for the NBC experience.
The 2022 NSW Parliamentary Inquiry also mentioned the problems with HALT,
"Highly Accomplished and Lead Teacher (HALT) accreditation has been inadequate, with only a small number of teachers applying and qualifying each year. The additional data collection and paperwork and the requirement to self-advocate serve to disincentivise teachers from pursuing this higher accreditation." (p. 17)
Although, "the committee divided" was reported over 90 times in their many proposals.
Hattie's Arbitrary Interpretations of NBC Research (VL, Ch 11):
As already stated, Hattie used:
Smith, Wanda, Baker, Hattie & Bond (2008). Chapter 12 A validity study of the certification system of the National Board for Professional Teaching Standards
Bond et al. (2000) make the arbitrary judgement that NBC-certified teachers are 'Experienced Experts' while Non-NBC teachers are 'Experienced Non Experts' (VL, p. 259).
Pool, Ellett, Schiavone &Carey-Lewis (2001) in How Valid are the National Board of Professional Teaching Standards..., investigate the claims made by Bond et al. (2000) and conclude,
"...there appear to be no assurances that attaining these credentials guarantees quality teaching and learning environments in everyday practice." (p. 46)
Podgursky (2001) in his critique of (Bond et al. (2000)) verifies this,
Greg Ashman wrote about the meaningless Teacher Standards in Australia - here.
'No study, however, has ever shown that National Board-certified teachers are any better than other teachers at raising student achievement. Nothing has changed with the release of this report. The National Board’s researchers rejected the use of student test scores as a measure of teacher performance, claiming, “It is not too much of an exaggeration to state that such measures have been cited as a cause of all of the nation’s considerable problems in educating our youth. . . . It is in their uses as measures of individual teacher effectiveness and quality that such measures are particularly inappropriate.”' (p. 2).
Cunningham & Stone (2005) in Value-added assessment of teacher quality as an alternative to the National Board for Professional Teaching Standards: What recent studies say conclude,
"...questions have persisted regarding the worth of NBPTS certification. Thus far, studies have shown that it is more clearly an indicator of teacher commitment to NBPTS’s Five Core Propositions (National Board for Professional Teaching Standards, 2005) than it is a predictor of improved student achievement.The underlying problem seems to be that NBPTS’s favored teaching style not well suited to the realization of the public’s primary policy objective: improvement in objectively measured student achievement." (p. 13)
Hattie & colleagues in Bond et al. (2000) conclude that expert teachers (NBC) outperform non-NBC teachers on almost every criterion (VL, p. 260).
Harris and Sass (2007) The Effects of NBPTS-Certified Teachers on Student Achievement was another study just investigating NBC vs Non-NBC.
Harris and Sass (2007) conclude, similar to Hacke (2010),
"we find relatively little support for NBPTS certification as a signal of teacher effectiveness. In general we find that prior to certification, future NBCTs are no more effective in raising student test scores than are other teachers who are never observed to become NBCTs." (p. 25)
Yet, Hattie & Ingvarson (2008) claim the opposite conclusion from Harris & Sass (p. 10),
The What Works Clearing House (WWC)
The largest educational evidence organization, the USA's WWC, also produced an intervention report in 2018: The National Board for Professional Teaching Standards (NBPTS) certification.
The WWC has the highest quality standards and concluded,
"NBPTS-certified teachers had mixed effects on mathematics achievement and no discernible effects on English language arts achievement for students in grades 3 through 8." (p. 1)
Dylan Wiliam: Teacher quality: What it is, why it matters, and how to get more of it.
Wiliam makes 2 important claims & provides significant evidence to support these:
1. We can’t identify good teachers by observing them.
2. We can’t identify good teachers from changes in test scores.
Wiliam concludes,
"a counterargument is often raised: how can we improve teachers if we don’t know how good they are? The answer is this: if we think of teacher quality as a continuum, we now know that we are unable to locate a particular teacher along that continuum with any accuracy, but we do know which way is better. If we direct our energies not on evaluating teachers but on improving them, we are far more likely to improve the quality of education that our pupils receive."
No comments:
Post a Comment