The Effectiveness of Traditional Teacher Certification
John T. Wenders*

Summary. The evidence clearly shows that traditional teacher training and certification has almost no influence on student performance. The high quality studies provide ambiguous results, and where a positive and statistically significant relationship between certification and student performance is found, it is small and therefore an unimportant determinant of student performance. The only teacher characteristics that have been clearly shown to improve student performance are: teacher verbal cognitive ability, a few years teaching experience, and, for secondary teachers, a Masters degree or major in the teachers subject taught. There is also evidence that focused, in-service, subject-matter training improves student performance.

Studies that show significant or important effects of traditional teacher training and certification on student performance are usually scientifically flawed and of low quality, such as the often quoted study of Linda Darling-Hammond (1999), whose results are light years out of the range of effects that have been reported by all other studies of this topic. (Whitehurst, 2002)

This evidence suggests that what makes an effective teacher is idiosyncratic and not capable of being easily identified beforehand. There is wide variance in the effectiveness of teachers with the same measurable characteristics. Requiring traditional teacher training and certification as a condition of entry into the teaching profession therefore acts as an effective bar to many potentially effective teachers. In this situation, the most effective way to attract and keep high quality teachers is to hire bright teachers with subject matter training (for secondary teaching), and rely mainly on post-employment mentoring, experience and in-service subject matter training to develop effective teachers. To the maximum extent, this will allow intelligent teachers to prove themselves as measured by their students performance. Those who prove themselves should be nurtured and rewarded, and those who do not should be terminated.

The Evidence on the Effectiveness of Traditional Teacher Training. This literature is too vast to cover in detail. The following brief discussion summarize what the research shows.

The Abell Foundation compiled a report that summarized the extant literature on teacher quality as properly measured by student performance (Walsh 2001). This research has been carried out primarily by economists, statisticians, and other social scientists outside the education establishment. The findings of the Abell report (pp. 5-7) are:

1. Teacher quality is a strong determinant of student achievement, regardless of the demographics of the students. Some teachers are clearly better than others.

2. Experienced teachers are more effective than new teachers. However, the positive effect of experience seems to peter out after a few years, and may in fact decline in later years.

3. Matching teachers and students race does not consistently improve student performance.

4. The most consistent finding is that effective teachers have higher scores on tests of verbal and other measures of cognitive ability. This is apparently the single best predictor of teacher effectiveness. Simply put, a teachers brains matter.

5. Teachers who have attended more selective colleges produce higher student achievement. One suspects that this is why private schools tend to hire teachers from such colleges. Selective colleges are probably simply another measure of the general verbal intelligence of their student bodies.

6. At the secondary level, science and mathematics teachers with better knowledge of subject matter are generally more effective. Again, another surrogate measure of general intelligence. There has been little research done of the importance of subject matter knowledge in English and the social sciences.

7. At the elementary level of teaching, there is no research that finds any particular college course work to be more effective.

8. Teachers with masters degrees are not significantly more effective than those without, unless the teacher is at the secondary level and the degree is in the subject matter being taught.

On another front, two economists, Dale Ballou and Michael Podgursky (1997), have done extensive research on the relationship between teacher pay and teacher quality. The results of their research are interesting for what they reveal about the workings of the market for teachers. Their measures of teacher quality as it affects student performance are (p. 8):
1. The quality of the college or university that awarded the teachers bachelor degree as measured by selectiveness.
2. A degree in an academic subject rather than a degree in education.
3. For secondary teachers, an undergraduate major in mathematics or science.
4. Undergraduate GPA.
5. SAT scores of prospective education majors.

Cognitive ability is the underlying feature of most of these measures.
As the authors note:

. . . The link between teachers cognitive abilities and student learning stands out in a literature that frequently fails to find significant relationships between other teacher attributes and student achievement. (p. 11)

Further, the authors say:

There are two widely used indicators of teacher quality that we do not employ--the percentage of teachers who hold advanced degrees and teachers experience. Although teacher compensation is generally based on these factors, their demonstrated relationship to teacher quality is slight or nonexistent. Advanced degrees have not been found to improve teacher effectiveness, while the contribution of experience appears weak, at best, and limited to the first few years of teaching. (References omitted)

Economist Eric Hanushek (1986) concluded after extensive meta-analysis on teacher attributes and student performance:

The only reasonably consistent findings seem to be that smarter teachers do better in terms of student achievement.

Lewis C. Solomon (in Wenglinsky 2000) summarized the research on teacher quality in the following way:

Perhaps the most enduring findings (about teacher quality) have been that student achievement is higher, ceteris paribus, when teachers have high verbal skills, and when they have strong knowledge of the discipline they teach (usually indicated by a major or minor in that field). (References omitted)

Again, in the definitive work summarizing the evidence of class size on student performance, Ehrenberg, et al, (2001) concluded that

. . . Certain teacher characteristics, do matter. . . . (T)he evidence suggests that teachers with higher verbal ability and (at the secondary level) with greater subject matter knowledge are associated with greater student learning. However, in spite of this evidence, school districts do not appear to systematically choose from their pools of teacher applicants those applicants who have the strongest academic backgrounds, who come from the better academic institutions, or who score the highest on tests of academic aptitude.

In a recent article reviewing the conflict between those who endorse traditional teacher education and certification as measures of teacher quality and those who say these matter little, if at all, Jeff Archer (2002) listed the following points that seem to be incontrovertible.

What do most scholars agree on? Perhaps the least contested studies on teacher quality point to the importance of teachers' basic skills.

Some of the most cited research on that topic was carried out by Harvard's Ferguson, who has analyzed data from Arkansas and Texas, two states that had, at one point, required all teachers to take a basic-skills test as part of their relicensure requirements. In Texas, for example, Ferguson (1991) found that teachers' scores on the skills exams had the strongest link to student achievement of four different variables he examined. The other variables, in descending order of effects, were teacher experience, class size, and whether teachers held master's degrees.

Other comparatively uncontroversial findings show a link between student improvement and teachers' knowledge of the subjects they teach. In a study of science and mathematics at the secondary school level, education professor David H. Monk of Pennsylvania State University (1994, 1995) found that the number of college-level courses that educators took in those content areas correlated positively with student achievement.

But his analysisbased on two federal databases of information on teachers and studentsalso hinted at a point of diminishing returns. For example, it seemed to make a big difference whether 11th graders' teachers had taken at least five math courses, but each additional course thereafter did not translate into as big a student-achievement gain. Monk also found that teachers' course-taking was more important than their attainment of a master's or doctorate.

Kirk Johnson (2000) looked into the issue of teacher training in education as compared to training in subject matter.

This report (studies) the test scores of fourth and eighth grade students who took the 1998 National Assessment of Educational Progress (NAEP) reading test and the 1996 NAEP math test. This analysis compared students who were taught by teachers holding advanced degrees in education with those whose teachers did not. The data showed that:

" In reading, eighth grade students of teachers who hold advanced degrees in the field of education perform worse on the NAEP exam than those whose teachers have advanced degrees in English.
" In math, eighth grade students of teachers who hold advanced degrees in education perform worse on the NAEP exam than those whose teachers hold any degree in math or science (bachelors or advanced degrees).
" Among fourth grade students, there is no significant difference in achievement between those whose teachers hold a bachelors degree in reading or math and those whose teachers have advanced degrees in education.
" A teachers education may be less important for achievement than the parents education. This research indicates that both math and reading scores rise if at least one parent holds a bachelors or postgraduate degree.
" Teachers with subject degrees, rather than education degrees, have students who perform better in math and reading, especially as students age. (p. 2-3)

Public elementary school administrators have an interest in hiring the best teachers for their schools, especially since Americans increasingly demand results and accountability for public education spending. As the findings of this analysis indicate, hiring teachers who hold subject degrees in math or English, rather than education degrees, is more likely to result in higher math or reading achievement among older (eighth grade) students.(p. 12)

A recent review of the literature by Grover J. Whitehurst (2002) concluded with the following graphical summary of what the high quality studies show about the effects of teacher characteristics on student achievement.

A paper by Betts, Zau, and Rice (2003) shows that in general these weak results for the effect of traditional certification on student achievement are applicable in California, at least in its second largest district, San Diego.

We certainly found many instances in which the achievement of students responded positively to higher teacher qualifications. But in most cases, we found no significant difference between less than fully credentialed, relatively inexperienced teachers and teachers in our comparison group. Overall, teacher qualifications appear to affect gains in student achievement sporadically.

(A) few measures of teacher credentials/teacher experience are similarly, and in a few cases much more strongly, related to student learning. But these results are sporadicmost of our measures of teacher credentials and teacher experience are not statistically significant and the results vary between reading and math. Math achievement seems to suffer when elementary school students are taught by interns with 0-1 years of experience. It is puzzling to note that student gains in both math and reading are predicted to be higher when students are taught by a teacher with an emergency credential and 0-1 years of experience instead of by a fully credentialed teacher with ten or more years of experience.

In some respects, administrators should be reassured to learn that a less than fully credentialed teacher sometimes appears to be as effective as a fully credentialed teacher. California spends roughly $100 million a year on the Beginning Teacher Support and Assessment (BTSA) program, which aims to provide assistance to teachers in their first and second years of teaching. This and related programs might successfully integrate inexperienced teachers into the classroom. In addition, SDUSD has adopted a peer coach program to train teachers in the latest instructional techniques, which may be particularly helpful for novice teachers.

Similarly, the result that middle and high school English and math teachers with less than a full subject authorization often are just as effective as fully authorized teachers should come as reassuring news, given that it is difficult for a district to ensure that all of its teachers have exactly the right mix of college courses as mandated by the CCTC. The one major exception to this rule was high school math teachers, in which case subject authorization level appears to matter tremendously. The evidence that teacher experience and credentials have less effect on gains in student achievement than some may think is particularly important given the grim new financial reality facing most California school districts as a result of Californias large budget deficits. In San Diego, the district tackled its budget problem in early 2003 by offering early retirement incentives. These incentives led approximately one in ten teachers to opt for retirement. The district plans to replace these teachers with less highly experienced teachers who will be paid less. It seems likely that the short-run effect of this mass retirement will be to make schools less effective simply because of the loss of institutional memory. However, our results suggest that after one or two years, many of the relatively inexperienced recruits may be far more effective teachers than some would believe.

However, our findings indicate that the traditional measures of teacher qualifications, such as education, credentials, experience, and subject authorizations, are not as strongly or as consistently related to student learning as some might think. The general concept that districts should look outside the box for additional ways to help teachers improve their teaching receives strong support from our findings.

Technical Issues. It is useful to briefly discuss why many of the studies that find traditional teacher training to be effective in producing good student performance are of low scientific validity. Michael J. Podgursky (2004) summarized the defects this way:

Large longitudinal data files have formed the basis for the most sophisticated current research on teachers and teacher effects on student achievement. . . . Studies that do not have a rigorous study design, i.e., with randomization or controls for prior student achievement, are likely to produce seriously biased estimates of the effect of teacher certification or other teacher characteristics on student achievement. The reason is that they do not adequately control for the socioeconomic background of students in classrooms and student SES is correlated with teacher credentials and strongly correlated with student achievement. In the language of econometrics, cross-section studies of the effect of teacher credentials on student achievement suffer from omitted variable bias. . . . Since SES has a very powerful effect on student achievement levels and gains, unless the researcher has very good controls for prior achievement and SES in a study of certification and student achievement, the resulting study is likely to yield an upward-biased estimate of the effect of certification.

Susanna Loeb (2004) also noted the selection bias in many studies that find a (false) connection between teacher training and certification and student performance.

The evidence as to the effect of certification is mixed, perhaps because there is very little variation in whether teachers are certified. Almost all teachers are certified and those who are not are far more likely to teach in schools with low-performing students. This selection bias produces a strong positive correlation between test scores and the percent of certified teachers in a school, but makes it difficult to show causation.

The result of selection bias is that the causation between certification and student performance may be exactly the reverse of what is often inferred: since high performing students are more likely to be taught by certified teachers, the line of causation is from students to certified teachers, not the other way around.

Another issue is the aggregation bias present in some studies that find strong teacher certification effects on student performance. Aggregation bias has been summarized by Grover J. Whitehurst in the context of part of the famous Coleman study of student performance:

Coleman's methodology is now understood to have been seriously flawed. All of his analyses were conducted on data that had been aggregated to the school level. For example, the average vocabulary score for all teachers in a school was related to the average test score for all children in a school. Researchers now understand that aggregating data in this way can distort findings. I am reminded of the man who had his head in the oven and his feet in the freezer but whose temperature, on average, was just right. If you average together the effective teachers with the ineffective teachers, and the high performing students with the low performing students, you don't get to see the cold and hot spots where teacher characteristics might make a difference.

Applying this bias issue to the work of Linda Darling-Hammond (1999), Whitehurst concluded (emphasis added):

A study by Darling-Hammond (1999) stands in contrast to the many studies that find no effects or very small effects for teacher certification. She related scores on the National Assessment of Educational Progress at the state level to the percentage of well qualified teachers in each state. "Well qualified" was defined as a teacher who was fully certified and held the equivalent of a major in the field being taught. For generalist elementary teachers, the major had to be in elementary education; for elementary specialists, the major had to be in content areas such as reading, mathematics, or special education. Darling-Hammond reported that teacher qualifications accounted for approximately 40 to 60 percent of the variance across states in average student achievement levels on the NAEP 4th and 8th grade reading and mathematics assessment, after taking into account student poverty and language background.

Although this study is frequently cited, the approach of aggregating data at the level of the state is seriously problematic. It goes backwards in terms of aggregation from the work of Coleman whose findings are considered suspect because the analyses were of data at the school level. Students do not experience a teacher with the average level of certification in a state; they experience a teacher who is or is not certified. The aggregation bias may account for Darling-Hammond's estimates of the effects of certification being light years out of the range of effects that have been reported by all other studies of this topic.

In addition, Darling-Hammonds study has several of the deficiencies note above--it is cross section, does not measure student achievement by value-added, and has very weak controls for student socio-economic status (SES).

After reviewing the technically high-quality studies, Podgursky concluded

The number of studies of teacher certification that meet the minimum methodological standards outlined above is very small. A recent survey of the literature by Wayne and Youngs in the Spring 2003 Review of Education Research found only two studies of teacher certification that were peer-reviewed, used longitudinal student-level achievement data, and controlled for student SES. The results of these studies (both by Goldhaber and Brewer and both using the National Longitudinal Educational Survey of 1988) had mixed results. They did find a small positive effect of math teacher certification on math achievement, but no statistically-significant effect of science teacher certification on science achievement. Recent surveys of the literature by Hanushek and Rivkin (2003) focusing on high quality studies that meet the standards described above find little evidence linking teacher credentials to student achievement. For example, of nine estimates of the effect of teacher test scores on student achievement, six found no statistically significant effect. Of the three finding a significant effect, two were positive and one was negative.

A survey of teacher quality by Michael B. Allen (2003) for the Education Commission of the States set a much lower standard for consideration of the studies to be included in his survey. Yet Allen found only limited support that pedagogical training had a positive effect on teacher quality. He also found that the effect of tightening entrance requirements into teacher training programs on student achievement was inconclusive. He also commented: It is not clear from the research reviewed for this report, however, whether [pedagogical] knowledge and skills are best acquired through coursework, field experience (especially student teaching) or on the job. In other words, the superiority of the common practice of requiring ex ante field experience (student teaching) as part of traditional teacher training, over careful mentoring while on the job, is problematical.

Significance vs. Importance. A few studies find traditional teacher training and certification to have statistically significant impact on student performance. However, the effects are universally very small and therefore unimportant as they relate to policy matters.
For example, a high-quality study by Aaronson, Barrow and Sander (2003) found that well over 90% of the variance in teacher quality (as measured by student performance) could not be explained by any measurable teacher characteristic. Yet the same study also found a few statistically significant relationships between some measures of certification and student performance.

This final section relates our estimates of [teacher quality] to measurable characteristics of the instructors available in the CPS administrative records. Observable characteristics include common demographic and human capital characteristics such as teachers gender, race, potential experience, tenure at CPS, advanced degrees (Masters or Ph.D.), undergraduate major, undergraduate college attended, and teaching certifications.

First and foremost, the vast majority of the total variation in teacher quality is
unexplained by observable teacher characteristics. At one extreme, a cubic in tenure and
indicators for advanced degrees and teaching certifications explains at most 3 percent of the total variation, adjusting for the share of total variation due to sampling error. That is, the characteristics on which compensation is based have extremely little power in explaining teacher quality dispersion. Including other teacher characteristics, changing the specifications for computing the teacher effects, and altering the minimum student-semester threshold have little impact on this inference. In all cases, the R2 barely exceeds 0.05.

Given a lack of compelling explanatory power, it is of little surprise that few human
capital regressors are associated with teacher quality. A notable exception is math or science
undergraduate degrees, which are associated with teacher quality of 0.06 to 0.08 grade
equivalents higher. The majority of standard education background characteristics, including
certifications, advanced degrees, and graduating from a top university, are loosely, if at all,
related to (teacher quality) .

Traditional Teacher Training and Certification Keeps Many Excellent Teachers out of Teaching. Thus, the research shows that the variance in teacher quality is wide, and over 90 percent of this variance cannot be explained by any measurable teacher characteristic. At best, traditional teacher training and certification produces teachers that are, on average, only slightly better than non-certified teachers. Thus, the failure to even consider non-traditionally trained and certified teachers automatically excludes large numbers of teachers who would turn out to be above-average in quality.

For example, consider quality distributions of two pools of potential teachers, one traditionally trained and certified and the other non-traditionally trained and certified, where the mean quality of the certified teachers is two-tenths of a standard deviation higher than the mean quality of the non-certified teachers. (This exaggerates what even the best research finds to be the quality difference between these pools, and therefore under-estimates the conclusion drawn below.) Clearly, with wide variance and small differences in their means, these quality distributions overlap greatly.

In this situation, under some common statistical assumptions, about 47.5% of the non-certified teachers will be of higher quality than the average of the certified teachers. Given that the graduates of teachers colleges are a minority of university graduates, and are universally of the lowest quality graduates, this means that confining teacher recruitment to the certified pool undoubtedly excludes many more higher-than-average teachers in the non-certified pool than it includes in the certified pool. Indeed, given that the non-certified potential teachers probably greatly outnumber those certified, it is highly probable that the number of higher-than-average non-certified teachers arbitrarily excluded from teaching is larger than all of the potential certified teachers.

Concluding Remarks. What produces good teachers is idiosyncratic: we simply do not know, before hand, what makes some teachers better than others, even though research has conclusively shown that some teachers are, indeed, much better. The only things that come through loud and clear are teachers verbal cognitive ability, some experience, and, at the secondary level, subject matter training. Good teachers are very important, but we dont know much about where they come from, and they certainly dont come exclusively, or even mainly, out of traditional teacher training and certification programs.

Thus, from a policy standpoint, the evidence argues against spending vast resources on these programs. Resources could be better deployed elsewhere. Traditional teacher training and certification act primarily as barriers for many who would be good teachers but who are not traditionally trained and certified.

Since we don't know beforehand which teachers will turn out to be good, policy should be directed at giving teachers a chance to reveal their talent, as measured by the value added to their students achievement, during a probationary period, and, once identified, rewarded accordingly. Policy makers should concentrate on what they can measure, student achievement, and ignore what they cannot measure, ex ante teacher quality. The common practice of requiring traditional teacher training and certification, and then awarding teachers tenure after only a year or two, cannot be justified on evidentiary grounds.

* Professor of Economics, Emeritus, University of Idaho.

References

Aaronson, D., L. Barrow, and W. Sander. 2003. Teachers and student achievement in Chicago public high schools. Federal Reserve Bank of Chicago Working Paper.

Allen, M. B. 2003. Eight questions on teacher pay: What does the research say? Denver: Education Commission of the States.

Archer, Jeff. 2002. Focusing In on Teachers. Education Week. April 3, 2002

Ballou, Dale, and Michael Podgursky. 1997. Teacher Pay and Teacher Quality. Kalamazoo, MI: W.E. Upjohn Institute for Employment Research.

Betts, J. R., A. C. Zau, and L. A. Rice. 2003. Determinants of student achievement: New evidence from San Diego. San Francisco: Public Policy Institute of California.

Darling-Hammond, Linda. 1999. Teaching and Knowledge: Policy issues posited by alternate certification for teachers. Seattle, Washington: Center for the Study of Teaching and Policy, University of Washington.

Ehrenberg, Ronald G., Dominic J. Brewer, Adam Gamoran and J. Douglas Willms, 2001. The Class Size Controversy,Psychological Science in the Public Interest, vol. 2, no. 1 (May).

Hanushek, Eric A. 1986. The Economics of Schooling: Production and Efficiency in Public Schools Journal of Economic Literature , 24 3(September): 1141-1177.

Johnson, Kirk A. 2000. The Effects of Advanced Teacher Training in Education on Student Achievement. Washington, DC: The Heritage Foundation. CDA00-09

Loeb, Susanna, and Michelle Reininger. 2004. Public Policy and Teacher Labor Markets: What We Know and Why it Matters. Lansing, MI: The Education Policy Center at Michigan State University.

Podgursky, Michael J. 2004. Improving Academic Performance in U. S.
Public Schools: Why Teacher Licensing is (Almost) Irrelevant. Chapter Ten in: Hess, Frederick M., Andrew J. Rotherham, and Kate Walsh, A Qualified Teacher in Every Classroom? Cambridge, MA: Harvard Education Press. 2004.

Walsh, Kate. 2001. Teacher Certification Reconsidered: Stumbling for Quality. Baltimore, MD: The Abell Foundation.

Wenglinsky, Harold. 2000. How Teaching Matters. Princeton, N.J.: Educational Testing Service and The Milken Family Trust.

Whitehurst, Grover J. 2002. Research on Teacher Preparation and Professional Development White House Conference on Preparing Tomorrows Teachers. April 4, 2002.

Draft: 8/9/04