|
Research on Alternative Teacher Certification
Michael Podgursky
Department of Economics
University of Missouri - Columbia
Prepared for “Alternative Routes into Teaching: Defining Challenges, Devising Strategies,” First Annual Conference of the National Center for Alternative Certification
San Antonio, Texas Feb. 1-3, 2004.
Introduction: What is “Alternative Certification?”
Alternative certification (AC) typically refers to programs that permit career changers or non-traditional teaching candidates to earn teaching licenses. The traditional route to teaching, which still accounts for the majority of new teachers, is for students to earn undergraduate degrees in education from a state-approved teacher training program. However, in response to concern about teacher quality, difficulties in recruiting qualified minority teachers, and shortages of teachers in academic fields such as science or math, many states have created alternative programs for entering the teaching profession. These programs are designed for individuals who already have baccalaureate degrees in areas other than education, and who often have experience in careers other than teaching. Such individuals are generally unwilling to make major investments of time and resources in returning to a school of education for a year or two of coursework required for a traditional license before they earn a paycheck as a teacher. Thus, common features of these programs are:
- Minimum pre-service training
- On-going professional development coursework and mentoring while on the job, leading to
- Regular teaching license after a probationary period of 2-3 years
Although 43 states have some sort of AC program in place, AC route teachers account for a substantial share of teaching recruits in only a few states (e.g., New Jersey, Texas, and California).
A standard reference on state programs is the annual volume by Feistritzer and Chesser (2002). A recent survey of the literature by Mathematica Policy Research (MPR) provides a useful taxonomy and overview of state programs (Mayer, et.al, 2003). The MPR survey also makes the point that there is considerable variation even within states in the character of AC programs, which mean that evaluations for any particular program may not generalize across a state, much less between states. For this reason, an MPR evaluation currently under way is focusing on particular features of AC programs that more readily generalize.
Alternative Certification and Student Achievement
How does the performance of alternative versus traditionally certified teachers compare? Do AC teachers produce larger or smaller student achievement gains as compared to traditionally-trained teachers?
Unfortunately we do not at present have research that can answer that question. While there have been many articles published about AC, few meet the standards of scientific rigor that would permit us to draw conclusions about the effect of AC on student achievement.
Scientific evaluation of the effect of educational policies, including teacher preparation, on student achievement requires either: a) randomized experimental study design, or b) non-experimental longitudinal data on student achievement. Unfortunately, little research on teacher testing or licensing meets either standard and the research that does is tentative and inconclusive.
Randomized experimental design usually provides the best data for education policy research. With respect to alternative certification, this would involve estimating the effect of teachers with AC and traditional preparation on student achievement through random assignment of students to classrooms with the two types of teachers, otherwise comparable (e.g., similar experience), within a school. Unfortunately, at present there is no research on teacher credentials or training that meets this standard, although the Institute for Education Sciences of U.S. Department of Education is promoting such studies (Moesteller and Boruch, 2002; U.S. Department of Education, undated). Mathematica Policy Research (MPR) has two random assignment studies of teachers under way. The first compares the performance of Teach for America teachers to conventionally prepared teachers ((http://www.mathematica-mpr.com/3rdLevel/teach4amer.htm). A second study compares AC with traditionally trained teachers at several sites across the nation (Mayer, et.al. 2003).[1]
If randomization is not feasible, and often it is not, then one must rely on non-experimental data to evaluate education policy. If we are to measure the contribution of a classroom teacher to student achievement, it is necessary to control for prior achievement of the student before he or she enters the classroom. Ideally, researchers would pretest the students in the fall and test them again in the spring. The difference in these scores, averaged over the classroom, would be a measure of a teacher’s “value-added.” If students are not pre-tested in the fall, then it is also possible to use test scores the previous spring, or for more than one previous year (longitudinal achievement data). Large longitudinal data files have formed the basis for the most sophisticated current research on teachers and teacher effects on student achievement (Sanders and Horn, 1994; Hanushek and Rivkin, 2004; Rivkin, Hanushek, and Kain, 2001; Aronson, Barnow, and Sanders, 2003; Betts, Zau, and Rice, 2003).
Studies that do not have a rigorous study design, i.e., with randomization or controls for prior student achievement, are likely to produce seriously biased estimates of the effect of teacher certification or other teacher characteristics on student achievement. The reason is that they do not adequately control for the socioeconomic background of students in classrooms and these omitted student SES factors are correlated with teacher credentials. In the language of econometrics, we say that these cross-section studies of teacher credentials suffer from “omitted variable bias.”[2]
The number of studies of teacher certification that meet these minimum methodological standards outlined above is very small. A survey of the literature in the Spring 2003 Review of Education Research (Wayne and Youngs, 2003) found only two studies of teacher certification that were peer-reviewed, used longitudinal student-level achievement data, and controlled for student SES. The results of these studies (both by Goldhaber and Brewer and both using the National Longitudinal Educational Survey of 1988) had mixed results. They did find a small positive effect of math teacher certification on math achievement, but no statistically-significant effect of science teacher certification on science achievement. Recent surveys of the literature by Hanushek and Rivkin (2003) focusing on “high quality” studies that meet the standards described above find little evidence linking teacher credentials to student achievement.
Three recent surveys of teacher quality research take up the question of the teaching performance of AC teachers (Allen, 2003; Wilson, Floden, and Ferrini-Mundy, 2001, Mayer, et.al. 2003). All three set a lower standard for inclusion of studies that what I have argued for above. Specifically, they include in their surveys descriptive and correlational studies that do not control for prior student achievement. Allen (2003) concludes that literature to date provides “limited” support for the proposition that AC programs can produce teachers who are as effective as traditionally-trained teachers. Wilson, Floden, and Ferrini-Mundy, 2001 take up a slightly different question and ask “what are the components and characteristics of high-quality alternative certification programs?” They conclude that the existing literature is “… limited and has produced decidedly mixed findings.” (Wilson, Floden, and Ferrini-Mundy, 2001). Of course, answering the question they pose for AC programs is a tall order since they also note that the existing literature cannot answer these same questions for traditional programs either. Unlike the earlier papers, which survey the more general literature on teacher quality, Decker, et.al, 2003 focus only on the question of AC and student achievement. They conclude that the exiting literature is not sufficiently rigorous (for reasons I have noted) to draw any conclusions about the AC and student achievement.
Academic Ability and Content Knowledge
Since there is no direct evidence on the relative performance of AC teachers, it is important to gather indirect evidence on their quality. The slender research that does exist on teacher quality suggests that teachers with better general academic skills, and, for teachers in specialty areas such as science and math, better specific content knowledge or coursework are associated with larger student achievement gains (Whitehurst, 2003). Thus it is useful to have studies that measure the academic skills of AC and traditional teachers. While states have published anecdotal or fragmentary information on the academic skills of teachers, there has been little systematic compilation of such data. Particularly useful are pass rates or scores on teacher licensing examinations, college GPA’s and coursework in teaching fields, and the selectivity of colleges attended by AC and traditional route candidates.
Retention of AC Teachers
How does the turnover of AC and traditionally-prepared teachers compare? A simple comparison of turnover rates by years of experience of ACP and traditionally trained teachers may be a misleading indicator of AC teacher’s commitment to teaching. It is well established in the research literature that schools with high concentrations of minority and poor students have higher teacher turnover rates. Now suppose the propensity to quit is the same for traditional and AC teachers in similar circumstances but AC teachers are disproportionately concentrated in poorer or high poverty schools. Then we will tend to observe higher turnover rates of AC teachers, but this is a biased estimate of the true difference in turnover propensity of the two types of teachers.
I am aware of no careful statistical study of teacher turnover comparing the two types of teachers. The Texas State Board for Educator Certification (SBEC) reports turnover rates of teachers with different types of certification for high and low poverty school districts. These SBEC data are posted on their web site (http://www.sbec.state.tx.us) and provides interesting comparisons of AC and traditional teaching candidates. An example is in Figure 1 below. However, a multivariate statistical analysis would provide a more convincing demonstration of differences in the propensity to quit for teachers with different types of certificates. In this regard, it is important to have good controls for working conditions in schools as well as teacher compensation for AC and traditionally trained teachers.
The survey discussed above by Allen (2003) also addresses the question of whether the retention of AC teachers matches that of traditionally trained teachers. He finds “limited” support for the proposition that the short-term turnover of AC teachers is comparable to that of traditionally prepared teachers, but “inconclusive” evidence about the long-term differences. However, this literature is very thin, and largely focuses on small samples of teachers in particular AC programs. It is possible that these results would not generalize to a larger universe of AC programs.
Figure 1
High Poverty / High Minority Middle Schools
http://www.sbec.state.tx.us/SBECOnline/reprtdatarsrch/tchrattremploy/Attrition%20of%20Teachers%20in%20High-Poverty,%20High-Minority%20Schools%20by%20Cert.%20Route%20(Class%20of%201995).pdf
Cost-Effectiveness of AC
Suppose that AC and traditional teachers are, on average, equally effective, however, suppose that the turnover rate of AC teachers is higher. Does this mean that the benefit-cost ratio of the former is actually lower? In fact it does not. In order to establish that, we need to compare the training costs of the two types of teachers. Here is where AC may have a considerable advantage. The reason is that AC training, by its very nature, is targeted to a person filling an actual teaching vacancy, whereas traditional university-based programs are not. University-based programs have two problems. First, there is a persistent mismatch between the degrees awarded by field and vacancies. University programs typically produce large numbers of elementary education majors – far in excess of need – whereas they produce relatively few graduates in areas such as math and science education, or special education. Table 1 illustrates this problem in Missouri. The number of newly certified elementary school teachers (predominantly graduates recommended for certification by schools of education) far exceeds the number of elementary school vacancies. By contrast the number of new certs in math was is well below the number of vacancies.
| |
First-Time
Certs |
Public School Vacancies |
Ratio (1) (2) |
| Elementary Education |
2000
|
1396
|
1.43
|
| Science |
300
|
262
|
1.15
|
| Mathematics |
193
|
270
|
.72
|
Source: Podgursky, et.al. 1999.
First-time time certs = Did not teach in a Missouri public school and did not previously hold a Missouri teaching certificate in another field.
This suggests that $10,000 spent on producing an additional education major is less likely to meet a public school vacancy than $10,000 spent on an AC candidate, who is actually filling a vacancy.
In addition to an imbalance between supply and demand by field, a second problem is that many education school graduates never teach in a public school, or do so only briefly. In this case, the public investment in their specialized pedagogical training has been wasted. In study of Missouri public higher education graduates cited above, we found that of 2239 1994-95 graduates with baccalaureate degrees in education, only 55 percent were teaching in a Missouri public school in 1998-99. (Podgursky, et. al. 1999). Again, because AC training is targeted to an existing vacancy, much less training is wasted.
Return to the Texas statistics in Figure 1 above. Let’s suppose that the cost of training for an ACP and a traditional teacher are $10,000 but only 55 percent of traditional candidates teach in a public school classroom as compared to 100 percent of ACP candidates. Then after six years, on average only $3476 of the training for a traditional teacher remains in a public school classroom as compared to $6910 for an AC candidate.
These are back-of-the-envelope calculations. However, a comparison of cost-efficiency between AC and traditional teachers must take account not only of differences in attrition after initial teaching employment, but also between the delivery of training and initial employment. I have seen no careful study making such cost comparisons.
Conclusion
At present there is little reliable research to inform most policy with regard to teacher training and licensing. However, there is now a strong emphasis at the new Institute for Education Sciences in developing such a research base in teaching. In addition, researchers are beginning to exploit large data files linking student achievement records longitudinally to examine how teacher characteristics affect student achievement gains. These data files hold great promise for providing data sets to monitor student achievement gains in schools and classrooms.
To date, studies that have estimated teacher effects on student achievement gains using these large longitudinal data files find that teacher performance is highly idiosyncratic, in that there is considerable variation in teacher performance across classrooms but these differences are not explained by differences in measured teacher characteristics such as type of license, experience, MA degrees, test scores, etc (Goldhaber, 2003; Podgursky, 2004). This suggests that AC programs that combine a search over a larger applicant pool with careful screening and mentoring hold promise has a mean to identify and cultivate superior teachers. States should continue their experimentation with AC programs, but make better use of their testing and administrative data to monitor their effectiveness.
References
Allen, Michael B. 2003. Eight Questions on Teacher Preparation: What Does the Research Say? Denver: Education Commission of the States. (July)
http://www.ecs.org/ecsmain.asp?page=/html/publications/home_publications.asp?am=5
Aaronson, Daniel, Lisa Barrow, and William Sander. 2003. “Teachers and Student Achievement in the Chicago Public High Schools” Working Paper. Research Department. Federal Reserve Bank of Chicago.
Betts, Julian R., Andrew C. Zau, and Lorien A. Rice. 2003. Determinants of Student Achievement: New Evidence from San Diego. Sacramento, CA: Public Policy Institute of California. http://www.ppic.org/main/publication.asp?i=321
Feistritzer, Emily C and David C. Chester. 2002. Alternative Teacher Certification: A State-by-State Analysis: 2002. Washington DC: National Center for Education Information.
Goldhaber, Dan. 2002. “The Mystery of Good Teaching.” Education Next Vol. 2 (Spring). pp. 50-55. http://www.educationnext.net
Hanushek, Eric. A. 2003. “The Failure of Input-Based Resource Policies.” The Economic Journal. Vol. 113 No. 485 (February). pp. F64-F98.
Hanushek, Erik A. and Steven G. Rivkin. 2004. “How to Improve the Supply of High Quality Teachers” Brookings Papers in Education Policy: 2004. Washington, DC: Brookings Institution. Forthcoming.
Hoxby, Caroline. 2001. “If Families Matter Most, Where Do Schools Come In?” in Terry M. Moe (ed.) A Primer on America’s Schools. Stanford University: Hoover Institute Press.
Mayer, Daniel P., Paul T. Decker, Steven Glazerman, Timothy W. Silva. 2003. “Identifying Alternative Certification Program for an Impact Evaluation of Teacher Preparation.” 8940-400. Washington, DC: Mathematica Policy Research, Inc. (April).
http://www.mathematica-mpr.com/PDFs/identify.pdf
Michael McKibbon. 2002. “Implementing Alternaitve Routes to Teacher Preparation and Certification in California.” Presentation to the California School Boards Association. (November).
Mosteller, Frederick and Robert Boruch (eds.) 2002. Randomized Trials in Education Research. Washington DC: Brookings Institution.
Podgursky, Michael, Donald Watson, Mark Ehlert, Michael Walker, William Foster. 1999. A Statistical Analysis of the Labor Market for Missouri Public School Teachers, 1994-95 to 1998-99. http://web.missouri.edu/~econ4mp/Downloadable_Papers.htm
Podgursky, Michael. 2004. “Improving Academic Performance in U.S. Public Schools: Why Teacher Licensing is (Almost) Irrelevant.” in Richard Hess, Andrew Rotterham, and Kate Walsh (eds.) A Qualified Teacher in Every Classroom? Appraising Old Answers and New Ideas. Cambridge, MA: Harvard Education Press.
Rivkin, Steven G., Eric A. Hanushek, and John F. Kain. 2001. “Teachers, Schools, and Academic Achievement.” Cambridge, MA: National Bureau of Economic Research.
Sanders, William L. and Sandra P. Horn. 1994. “The Tennessee Value-Added Assessment System (TVAAS): Mixed Model Methodology in Educational Assessment.” Journal of Personnel Evaluation in Education. Vol. 8, pp. 299-311.
U.S. Department of Education. Institute of Education Sciences. Undated. Random Assignment in Program Evaluation and Intervention Research: Questions and Answers http://www.mathematica-mpr.com/PDFs/randomassign.pdf
Wayne, Andrew J. and Peter Youngs. 2003 “Teacher Characteristics and Student Achievement Gains: A Review.” Review of Education Research. Vol. 73 No. 1 (Spring) pp. 89-122.
Whitehurst, Grover. 2003. “Scientifically Based Research on Teacher Quality: Research on Teacher Preparation and Professional Development.” in U.S. Department of Education. Meeting the Highly Qualified Teachers Challenge: The Secretary’s Second Annual Report on Teacher Quality. Washington D.C. http://www.ed.gov/offices/OPE/News/teacherprep/index.html
Wilson, Suzanne M., Robert E. Floden, Joan Ferrini-Mundy. 2001. Teacher Preparation Research: Current Knowledge, Gaps, and Recommendations. A Research Report Prepared for the U.S. Department of Education. Seattle, WA: Center for the Study of Teaching and Policy. (February).
[1] Unfortunately, in both cases, only elementary teachers are assessed, which may limit the generalizability of the study, since AC programs are often focused on recruiting secondary school teachers with strong content knowledge in subject areas. For example, McKibbon (2002) notes that one of the original motivations for the California Internship program was to develop a fast track entry program for professionals laid off in the aerospace industry.
[2] A recent study by Hoxby (2001) highlights the importance of these socioeconomic variables and their potential for producing bias in teacher effects research. Hoxby analyzed the effect of family, neighborhood, and school input variables on student achievement and educational attainment using two large nationally representative longitudinal studies of students (the National Educational Longitudinal Survey, NELS88, and the National Longitudinal Survey of Youth, which began in 1979). The list of variables included in each of the areas is extensive. Family variables include parent's education, family income, student race and ethnicity, books at home, etc. The school input variables include per-pupil spending, average class size, average teacher salary, maximum teacher salary, percent of teachers with MA's, average experience of the teacher, teacher certification status, and other information on school resources. Community variables include income and demographic data on households in the school district and city. Hoxby compared the percent of the variation in student achievement on various field tests (math, reading) explained by each of these sets of factors. For every test, the percent of the variation explained by the family variables far exceeded the school input variables. The family variables explained from 34 to 105 times as much variation in student achievement test scores as the school input variables. She also examined years of schooling completed at age 33. Family variables explained 19 times as much variation in student educational attainment as did school inputs.
|
