Defrocking the National Board

Education Next Issue Cover

Will the imprimatur of "board certification" professionalize teaching?



By MICHAEL PODGURSKY

12 Comments | Print | PDF |

Summer 2001 / Vol. 1, No. 2


The Certification System of the National Board for Professional Teaching Standards: A Construct and Consequential Validity Study

by Lloyd Bond, Richard Jaeger, Tracy Smith, & John Hattie

Center for Educational Research and Evaluation, University of North Carolina at Greensboro, October 2000

The National Board for Professional Teaching Standards, through its series of comprehensive performance assessments of teaching proficiency, is identifying and certifying teachers who are producing students who “differ in profound and important ways from those taught by less proficient teachers.” So concludes this National Board-selected group of researchers in their study comparing National Board-certified teachers with teachers who were unsuccessful in their bid for certification.
Although the federal government, states, school districts, and private foundations already have invested nearly $200 million in producing and rewarding National Board-certified teachers, this is the first study assessing whether the National Board has actually succeeded in identifying “expert” or “master” teachers who perform better than their uncertified peers. The National Board and its supporters greeted the results with pleasure. “This study tells parents and the community, educators and policymakers that National Board Certification is a distinction that really matters,” National Board president Betty Castor declared in a press release. Lee Shulman, president of the Carnegie Foundation for the Advancement of Teaching, claims, “In no other profession will you find an equivalent study with these kinds of rigorous assessments . . . no comparison can be found.”

Unfortunately, there is much less in this report than the press releases imply. In effect, the report really tells us only that teachers who were certified by the National Board were more likely to display the types of behaviors the National Board favors. Such a circular exercise does not necessarily prove that National Board-certified teachers do a better job of raising student achievement. After $500,000 (the cost of this U.S. Department of Education-funded study) and three years of research, the fundamental question remains unanswered: Is the National Board’s certification process a valid and cost-effective way of identifying the nation’s best teachers?

A Rising Tab


The National Board for Professional Teaching Standards was created in 1987 on the recommendation of the Carnegie Corporation’s Task Force on Teaching as a Profession. The board’s mandate was to raise the professional status of teachers (and the quality of teaching) by creating a means to identify and certify the most accomplished teachers. The National Board likes to compare itself to the medical specialty boards. All doctors are licensed by their states, but most go on to obtain advanced training and voluntary certification from one of the 24 medical specialty boards. The National Board sees itself as providing a similar form of advanced certification, a signal of expertise and excellence.
The Carnegie Foundation provided the National Board’s start-up funds, but, beginning in the early 1990s, the U.S. Department of Education became the National Board’s primary source of support. In recent years the National Board has received roughly $20 million annually from the U.S. Department of Education. Currently there are roughly 9,500 nationally certified teachers, with many more in the pipeline. Substantial pay increases now accompany board certification in many states and districts. The Los Angeles unified school district recently signed a contract with its American Federation of Teachers local that gives board-certified teachers a 15 percent bonus for the ten-year duration of a National Board certificate. Florida offers a 10 percent bonus for ten years plus an additional 10 percent if board-certified teachers agree to mentor other teachers. Ohio provides an annual bonus of $2,500 for ten years. The Cincinnati teachers union negotiated an additional $1,000 bonus, plus an additional $4,500 if board-certified teachers serve as lead teachers.

In short, states and districts are beginning to incur substantial long-term costs in rewarding National Board teachers. If the National Board reaches its goal of having 105,000 certified teachers by 2006, states and districts may be spending nearly $1 billion annually in additional compensation alone (not counting the $2,300 National Board assessment fee and related costs). Moreover, with National Board teachers acting as mentors for new teachers, their influence will extend well beyond their numbers.

(Not) Measuring Achievement


No study, however, has ever shown that National Board-certified teachers are any better than other teachers at raising student achievement. Nothing has changed with the release of this report. The National Board’s researchers rejected the use of student test scores as a measure of teacher performance, claiming, “It is not too much of an exaggeration to state that such measures have been cited as a cause of all of the nation’s considerable problems in educating our youth. . . . It is in their uses as measures of individual teacher effectiveness and quality that such measures are particularly inappropriate.”

To measure teacher quality, the authors began by trying to determine what good teachers know and do. This process of creating what the authors consider appropriate standards involved “a massive synthesis of meta-analyses that encompass over 200,000 research studies.” These 200,000 studies included ethnographic along with conventional statistical studies. From this synthesis, the researchers claim to have distilled 13 principles of good teaching. Examples include:
• “Experienced expert teachers adopt a problem-solving stance to their work.”
• “Experienced expert teachers aim at creating an optimal classroom climate for learning.”
• “Experienced expert teachers are passionate about teaching and learning.”
The researchers then developed methods for measuring and scoring such attributes in their sample of 65 teachers (31 who passed and 34 who failed their National Board assessment). For example, in order to assess the “multidimensional perception” of teachers (i.e., “Expert teachers develop a high level of-withitness,’ that is, they show that they are aware of events that occur simultaneously”), the researchers used a survey of each teacher’s students and observed teachers during a three-hour, prearranged classroom visit. The goal of creating an “optimal classroom climate” was measured in a similar manner. On all 13 dimensions of teaching practice, National Board teachers outscored those who did not make the cut. In 11 of 13 cases, the differences were statistically significant.

Although nearly $200 million has been invested in producing and rewarding National Board-certified teachers, this is the first study assessing whether “board-certified” teachers perform better than their uncertified peers.

The researchers also examined two measures of student performance: an “internal” example of student work provided by the teacher based on the lesson observed by the research team and an “external, developmentally appropriate measure of writing proficiency.” The “internal” work samples obviously varied from teacher to teacher, but were graded according to a standard rubric. The students of the board-certified teachers scored significantly higher on the “internal” measure. No statistically significant differences were found on the “external” writing assignment. (Ten of the 65 teachers refused to provide student work, and a number of the classroom-based assignments were unscorable.)

There are numerous problems with this methodology. Let’s begin with the authors’ using 13 “dimensions of teaching expertise” as their evaluation criteria instead of student test scores. Even if we accept the dubious proposition that 200,000 studies provide a scientific basis for the authors’ 13 nebulous standards of good teacher practice, we can’t be sure that the ways in which the authors have chosen to measure these standards necessarily replicate those of the underlying studies. Exactly how did thousands of different studies, of varying methodological rigor, measure “an optimal classroom climate for learning”? The authors’ 13 dimensions of teaching practice are valid measurement criteria only if the authors can demonstrate that their measures exactly replicate those of the literature they cite. And if the underlying measure of student achievement in these studies was standardized tests, as was surely the case in many of them, why are such tests acceptable as measures of teacher quality in studies that are meta-analyzed and used indirectly, but unacceptable when they are used directly to assess teacher quality in a structured research design? Readers of this study simply have no way of knowing whether the researchers’ 13 measures of teacher expertise actually correlate with improved student achievement.

Lack of Controls


When the authors actually did examine student performance, albeit with rather vague measures, they neglected to collect data that would have permitted them to adjust the performance data for students’ socioeconomic status, demographic characteristics, and previous levels of achievement. If the socioeconomic status and demographic characteristics of the classrooms taught by National Board teachers differ from those of noncertified teachers, measures of teacher quality that rely on student performance may be biased. Students of National Board teachers who exhibited superior academic performance may already have been performing at a high level when they entered class in the fall.

The authors acknowledge this limitation concerning their measures of student performance, but the issues of socioeconomic status and previous student achievement are problems for all of the researchers’ measures of teacher quality. For example, several of the 13 dimensions of teaching expertise were measured using student surveys, with questions such as, “An important reason why I do homework is because I like to learn new things,” or, “I do my schoolwork because I’m interested in it.” Students’ family backgrounds and previous educational achievement are likely to influence their responses to such questions.

Students’ family backgrounds are also likely to affect researchers’ evaluations of teachers’ classroom practice. Imagine two classrooms: one with well-behaved, highly motivated students from well-to-do families, the other with poorly behaved, unmotivated students from poor families. Now consider the scoring criteria. Under “Preventive and Reactive Classroom Environment,” teachers receive the top score if they “provide effective management procedures with a comprehensive focus on student learning,” but receive the lowest score if they “react to disciplinary incidents after the fact rather than trying to prevent them.” On “Multidimensional Perception,” teachers receive the top score if they “identif[y] events occurring simultaneously while maintaining a focus on instruction.” A teacher who is “often overwhelmed by the complexity of classroom events” receives the lowest score. It is easy to imagine the bias introduced by differences in students across classrooms. If we took high-scoring, “multidimensionally perceptive” teachers out of their well-to-do classrooms and put them in tough, low-income classrooms, they too might be “overwhelmed by the complexity of classroom events.”

In fact, there is evidence of significant socioeconomic differences between the classrooms of National Board teachers and those of unsuccessful certification candidates. In response to queries about this matter, the National Board provided some unpublished data. The 65 teachers were asked the following question: “Approximately what percent of the students in your class come from the following types of families?” Among the board-certified teachers, 44 percent reported that the largest share of their students came from “well-to-do families with few if any financial problems,” while only 8 percent reported that the largest share came from “families who cannot afford the basic necessities of food, clothing, and shelter.” Among unsuccessful candidates, however, the corresponding percentages were 21 percent in each category. By this measure, board-certified teachers were twice as likely to have children from wealthy families and less than half as likely to have poor children. This suggests that there may have been a major socioeconomic gap between the students of the two groups of teachers.

If the National Board reaches its goal of having 105,000 certified teachers by 2006, states and districts may be spending nearly $1 billion
annually in additional compensation alone.

Finally, there is the question of sampling. To determine whether the average National Board teacher is better than the average unsuccessful candidate, we need to draw random samples of both groups. The National Board’s researchers, however, chose a peculiar sampling scheme that oversampled particularly high-scoring National Board teachers and particularly low-scoring teachers who were unsuccessful. If we think of teachers who pass the National Board’s assessments as earning a C or better, this procedure amounted to oversampling teachers who earned As and Fs. The researchers are well aware of the effect of such sampling: “These groups were defined to ensure that dependable differences between National Board Teachers and non-Board Certified teachers could be detected.” In other words, they structured the sampling so as to increase the likelihood of finding an effect of National Board certification. This type of sampling is justified for some of the more complicated statistical analysis conducted in one section of the report. However, by sampling in this manner, the authors have rendered meaningless any simple comparisons of averages-the kinds of simple comparisons that are prominently displayed in the press release and the executive summary of their report. It is simply incorrect to claim that “NB teachers are superior in 11 of 13 dimensions” when the researchers have sampled in this manner. We do not know whether simple random sampling would have yielded significantly higher means for National Board-certified teachers on 11 of 13 dimensions.

Readers of this study simply have no way of knowing whether the researchers’ 13 measures of teacher expertise actually correlate with improved student achievement.

The resources available for this study ($500,000, or roughly $8,000 per teacher) would certainly have been more than enough to perform a rigorous analysis of the performance of National Board teachers vis-à-vis unsuccessful candidates, using a random sample of the two groups and adjusting for students’ socioeconomic status and previous achievement levels. In fact, these resources probably would have been adequate to increase the sample to several hundred teachers. Such a study would ask not only whether the achievement scores of students of National Board teachers improved more than the scores of students of unsuccessful candidates, but also whether National Board certification was a cost-efficient way to identify excellent teachers. For example, would principal or parental evaluations have worked just as well? Or were less costly components of the teachers’ National Board scores, such as the one-day assessment at a Sylvan Learning Center, just as effective as the costly, time-consuming (and coaching- or cheating-prone) portfolio in predicting student performance? The shortcomings of this study, the paucity of independent research on the National Board, and the large investments being made by states in rewarding National Board-certified teachers highlight the need for a rigorous and arm’s-length cost-benefit study of National Board certification.




Comment on this article
  • Kathleen Carrick says:

    Dear MICHAEL PODGURSKY:
    What I am searching for is information on how to pass the NBPTS. What I have discovered so far is that it is very caprious. Teachers in my group re-do one of the standards putting hours of work into the project only to receive the same score or a lower score. Every time that a teacher needs to re-take a standard, they are charged $275 for each standard. It is to the advantage of the NBPTS to fail teachers. I want to know if there is any information on how to pass the NBPTS.
    Thanks!
    Kathleen Carrick

  • Wendy says:

    Dear Ms. Carrick,

    I am currently working on the earning National Board Certification. My county offers support in the way of monthly meetings and a list of mentors who are willing to read and answer questions about the process and entries.This process has been helpful to know what the assesors are looking for. Our county has a higher passing rate then the national average.

    Good luck,

    Wendy Butz

  • Jonathan Keith says:

    Dear Ms. Carrick,
    You absolutely can pass the NBPTS, but you will need support. I had two awesome mentors I couldn’t have done without. And I do believe the process made me a better teacher, the content test is another story. You can’t prepare for it – you either know it or don’t, but if you apply basic test taking strategies you’ll be ok.

    Jonathan Keith, NBCT

  • Maria Hirst says:

    Ohio’s legislature reneged on the $2,500 stipend several years ago. Despite being disillusioned by my state’s encouragement to enter the process and then not honoring the deal for stipends, it was a great process. Learning to evaluate and reflect on one’s teaching practice is invaluable, and National Board Certification is a worthwhile project. Unlike the tests we measure student success by in our state, the National Boards exams do not rely on trickery – it is knowledge and practice-based.

  • Valerie says:

    What I find interesting was finding an help wanted ad on Craigslist under education employment opportunities a few summers ago. NBPTS was looking for scorers and the pay $18.00/hour. So this is how they find qualified people to score our materials? Yikes!

  • Sal says:

    Just days ago I found out that I was unsuccessful in my sole bid for National Board certification. Disappointment aside, I was left perplexed as a result of the feedback I received with my scores, which I’d like to share here for any teacher considering this certification.

    I knew beforehand about the generic type of feedback used in this process, and that there would be no specific constructive feedback to point out where exactly I went wrong or could improve. But some of the statements on my score report were baffling to me, as they implied that I provided no evidence whatsoever of reflecting on instructional decisions (most of my written commentary was an exercise in doing just that), setting worthwhile goals for learning (the objectives for each lesson were taken directly from the state standards, and the lessons were formulated in collaboration with accomplished teachers from my content area department), or facilitating individual participation (despite my video entries showing just about all of my students participating as appropriate for each lesson).

    For the amount of money each candidate is paying, NBPTS ought to have the means to give feedback that is more helpful. They certainly have enough time to do so. An NBPTS customer service rep told me that providing more specific feedback would be like telling candidates what exactly to do to pass the next time around; however, teachers know that commenting on a student’s rough draft of an essay is not the same as writing their final draft for them, nor does it guarantee a better outcome. And in the case of NB certification, a follow-up attempt must rely on entirely new student work, new videos, etc. and not simply a rewriting of the original portfolio entries. In light of this, it’s ironic that NBPTS appraises teachers on how well they give “appropriate feedback and next steps to students,” yet they won’t do the same for the teacher candidates they’re evaluating. Moreover a $75 re-evaluation fee for each portfolio section only serves to confirm their initial scoring without additional clarification, while an appeal process is advertised by NBPTS as costly and highly unlikely to yield desired results.

    As perplexing as the above is to me, the worst part in my case deals with the assessment center exercises (i.e., exam questions). The correct responses to these are more objectively verifiable. After taking the exam for my subject area (Social Studies-History), I immediately went home and checked the relevance and accuracy of my responses to the exam prompts and verified them to be correct almost in their entirety; and, I had much to write for each prompt. My scores did not reflect this at all. One cannot help but note that the greater the number of sections a candidate must reattempt for success, the more NBPTS stands to profit financially therefrom.

    Finally, it’s worth mentioning that I attempted this on my own out of my entire school district. I had no cohort and no mentors to assist me even though I tried to incentivize my time-strained NBCT friends to do so. To anyone in this same position considering National Board certification, know that there are no “extra credit” points to compensate for this disadvantage. If I ever try it again, it will be when I have the opportunity to do it with a cohort of teachers or similar support mechanism. I hope these observations will help potential NB candidates make a more informed decision about pursuing this.

  • Meryle Hiurotsu says:

    Sal,
    I encourage you to try again. Don’t give up. When I certified in 1999 and recertified (Young Adulthood Mathematics) in 2008, I joined a group of teachers. I found our meetings a waste of time and discouraging. Also sometimes misleading…

    However, I watched a video by State Farm Insurance and found it helpful. It gave points on how to write my paper. On each question, I would address specifically what was asked. In self-analysis type questions, I would answer “How” and “Why” in detail.

    I don’t know what state you are from, but in California where I did my NBC certification in 1999, there are several support groups. I live in Hawaii now and did my recertification by myself.

    Whatever you decide to do, I hope you won’t give up. The NBC process is time-consuming and stressful, but I have the rewards to surpass all other accomplishments. Good luck! Meryle

  • Lucy says:

    I just recently received my results from NBC and didn’t pass. Only one of my entries received acceptable points. I was mentored by a double NBC teacher (she renewed her certification and passed). She was noted as being the best. Well, I too worked endless hours towards becomig an accomplished NB teacher. I not only took this really hard but the person who was the best mentor in our school district, was disappointed too. I don’t take failure too well, so I am appealing my entries. I have another accomplished NB score person to re-read my entries I didn’t pass and compare them to the standards and their “showed no evidence…” comments. This person is carefully pulling my entries apart to see if I have a case to appeal. So far, I have case to appeal.
    I am at the end of my teaching career and my retake fee was over $2,000 in order to do it again. I know I am a good teacher and I struggle with this definition of a national board certified teacher identifying a good teacher or a good teacher being a good teacher due to years of experience with top evaluations, degrees, lots of professional development, leadership positions and other credentials.

  • Linda says:

    Lucy don’t let it define you. I did pass my boards but it is not that certificate, my evals, my leadership roles in my school, my state exam scores, or anything of the such that makes me think I’m a good teacher. It’s my students. They are in the trenches with us every day and at the end of the day they will be the ones who allow us to go home feeling either accomplished or like we need a do-over the next day. And since you have been at this a while you know as well as I do…certificate or not…we just hope our accomplished days out-number the do-over days. Thanks for your dedication to our profession!

  • Sally Jones says:

    Florida does not offer the 10% bonus money or the mentoring opportunity. That was taken away many years ago. We receive nothing for it. Please get the facts correct as many teachers find no incentive to go out for National Board Certification anymore. Additionally, for what it costs to renew, it seems to me it’s big business and quite a sham. I was promised 80K over 10 years if I mentored. I mentored my first year and got 3K. After that, I got nothing. I also became quite a laughing stock with a “who do you think you are?” attitude from those who did not have NBC. I don’t know who is making the money off this, but it is definately NOT the teachers. Spend the money and go out for a Master’s Degree or a Doctorate, both which WILL pay for many years to come.

  • J says:

    Board Certification is a for-profit business defended with pseudo-science…this author is absolutely right–there is no way to validate their “standards”, and their refusal to incorporate student scores in teacher evaluation is highly suspect. I know a number of average-below average teachers who have achieved certification–it cheapens a profession that is already struggling to regain it’s esteem.

  • Tom says:

    The National Board is a wonderful program which improves teachers skills in the classroom dramatically. I was fortunate enough to achieve on my first attempt, but even if I did not, I would have learned so much from the National Board process that it would have made it worth it. This private organization does wonderful things for teachers who are looking to improve their impacts on students in the classroom, so let’s support National Board instead of criticizing it.

  • Comment on this Article

    Name ()


    *

         12 Comments
    Sponsored Results
    Sponsors

    The Hoover Institution at Stanford University - Ideas Defining a Free Society

    Harvard Kennedy School Program on Educational Policy and Governance

    Thomas Fordham Institute - Advancing Educational Excellence and Education Reform

    Sponsors