Eight Questions on Teacher Preparation: What Does the Research Say?

Author:

Michael B. Allen

Objective:

This Education Commission of the States (ECS) 2003 report reviews the body of research on teacher preparation to ascertain what evidence the research truly provides and what the implications are for policy.

Methodology:

The report is based on a review of 92 studies selected, using rigorous criteria, from more than 500 originally considered. These studies were used to answer eight questions about teacher preparation that are of particular importance to policy and education leaders.

The Criteria Used to Select Research for Review in the Report

• Directly relevant to the questions under consideration
• Published in a scientific journal (waived for the consideration of additional literature requested by ECS)
• Published within the past two decades
• Studies of U.S. teacher education
• Empirical (offering evidence – quantitative, qualitative or both – for conclusions, rather than offering opinion, theory or principles)
• Rigorous (meeting generally accepted standards in relevant research traditions).

Background:

This report is the first in a series of reports on teaching quality that ECS to be released in 2003 and 2004. Subsequent reports will focus on what the research says about teacher recruitment and retention, licensure and certification and professional development.

The Eight Questions:

Question 1. To what extent does subject knowledge contribute to the effectiveness of a teacher? (Related Questions: Is there a significant advantage to having an undergraduate major, as opposed to a minor, in the subject taught? To having a graduate degree, as opposed to an undergraduate major, in the subject?)

Question 2. To what extent does pedagogical coursework contribute to a teacher's effectiveness?

Question 3. To what extent does high-quality field-based experience prior to certification contribute to a teacher's effectiveness? (Related Questions: Are professional development schools more effective than other kinds of field experience? Are five-year preparation programs more effective than four-year programs? What are the general characteristics of high-quality field experience?)

Question 4. Are there "alternative route" programs that graduate high percentages of effective new teachers with average or higher-than-average rates of teacher retention? (Related Question: What are the important characteristics of successful alternative route programs?)

Question 5. Are there any teacher preparation strategies that are likely to increase the effectiveness of new teachers in hard-to-staff or low-performing schools? (Related Question: What about in urban or remote rural schools?)

Question 6. Is setting more-stringent teacher preparation program entrance requirements, or conducting more-selective screening of program candidates, likely to ensure that prospective teachers will be more effective?

Question 7. Does the accreditation of teacher preparation programs contribute significantly to the likelihood their graduates will be effective and will remain in the classroom? (Related Question: What accreditation measures are likely to be most effective?)

Question 8. Do institutional warranties for new teachers contribute to the likelihood that recent graduates of those institutions will be effective? (Related Question: Do teachers given remediation under those warranties demonstrate increased classroom effectiveness?)

Where Did the Questions Come From?

ECS synthesized the questions from: (1) five focus groups conducted with policymakers in 2000-01, (2) an ECS review of legislation introduced in all 50 states during the 2000 and 2001 sessions, and (3) the experience and insight of ECS staff.

How Are the Questions Answered?

The response to each question consists of four sections:

-A Quick Answer provides the briefest, most direct answer possible to the question posed.

-A Significance of the Question section outlines the debates or controversies that surround the question, provides a sense of what is at stake and gives examples of the kinds of policies that hinge on the answer.

-A review of the pertinent research (What the Research Says) gives a detailed examination of the research on the question (and, in some cases, related questions) posed.

-A section on What It Means for Policy discusses the extent to which the research findings provide guidance for developing policies or programs.

The report concludes with a discussion of some of the major shortcomings of research on teacher preparation (and education research, in general), and with a set of recommendations for strengthening education research – both overall and specifically in the area of teacher preparation.

Recommendations:

The report contains 20 detailed recommendations for education researchers,
policy-makers, educators and funders.

Availability:

The full report may be read online or purchased at

http://www.ecs.org/html/educationIssues/teachingquality/tpreport/home/index.asp

RECOMMENDATIONS

Challenges to Overcome
The Role of Policymakers, Educators and Others
What Education Researchers Can Do
What Policymakers Can Do
What Practitioners, Including Teacher Educators, Can Do
What Funders Can Do
What Researchers, Policymakers and Practitioners Can Do Together

Challenges to Overcome

If the research on teacher preparation is to be improved, fundamental obstacles to the progress of education research, in general, must be addressed. The important role research plays in policy and practice decisions in other fields, most notably health and medicine, serves as an ideal to which education can aspire. When an article appears in the New England Journal of Medicine, a bulletin is released by the Centers for Disease Control and Prevention, or a report is published by the National Institutes of Health, both professionals in the field and the larger public take notice, accord it immediate credibility and respect, and may even make appropriate changes in their lifestyle or professional practice. How far that is from the reality of research in education!

There are a number of reasons for the lack of respect for and impact of research education, not the least of which are the many inadequacies of the research itself. In addition to the methodological problems already noted, there are a number of sociological factors at work:

1. The standards for good education research are relatively unformed.
* There is no good system within education research that gives special recognition to truly outstanding and significant research.
* Peer review is of uneven quality.
* Although there is considerable consensus within the various research traditions (ethnography, experimental research, etc.) about what constitutes high-quality research, there is not always mutual recognition of the value of different research traditions.

2. There is poor understanding within the research community, as well as the policymaker and practitioner communities, about what issues it does or does not make sense to study through experimental research.
* As the National Research Council publication Scientific Research in Education makes clear, the nature of the question should dictate the nature of the research appropriate to address it. Experimental research is not the only valuable kind of research and is not the "gold standard" for every kind of question. Both experimental and descriptive methods have an appropriate role.
* The difficulties in conducting true experiments in education make it unreasonable to expect that every study seeking to identify important relationships must be experimental to be informative.

3. The reward and incentive structures in the teaching profession and higher education do not support improvement in research.
* The promotion and tenure system in higher education encourages the proliferation of research to meet the expectation for publication; any research will do. There are, therefore, too many poor-quality journals with too much poor-quality research.
* There is inadequate release time for education faculty to do good research.
* Researchers are too often interested only in conducting studies that gain them the recognition of their peers, even if such studies have little practical significance.

These problems surrounding education research reflect something of a vicious cycle. The lack of respect and attention it gets from potential supporters outside the education research community leads to inadequate resources for conducting good research and a research agenda that fails to address outsiders' key concerns, which in turn deepens their disaffection and inattention.

Clearly, funding for education research is inadequate. A 1997 report by the National Science and Technology Council includes a revealing analysis by RAND of the nation's 1995 fiscal year expenditures on research, both public and private. The analysis found that while research in the fields of health, transportation and energy accounted for 2-3% of total expenditures in each of those areas, research on children and adolescents (which includes research on education) accounted for less than three-tenths of 1% of total expenditures.

The low level of funding is reflected in the character of education research in general, and certainly in the research on teacher preparation. Much of the research on teacher preparation involves studies of individual programs, is restricted to small sample sizes and cannot afford to employ more-powerful research instruments and methods. This makes it difficult to draw generalizations that are valid for a broad range of teacher preparation programs. The lack of funding also makes it more difficult to conduct the kind of fine-grained research that looks at more specific inputs and impacts. Research literature capable of assessing the impacts of courses in child development, learning theory, classroom management, student assessment and the like – particularly on specific populations of students – would have been extremely valuable to review for this report.

The realities of funding for education research make their impact felt in other ways, as well. Government and foundation priorities frequently shift, making sustained and consistently supported research agendas difficult to carry out. The overriding focus in private and public funding for education is on program impact, and restricted budgets for the evaluation of programs often means that high-quality evaluations that could be useful to educators and policymakers simply cannot be carried out.

The problems facing education research are also a product of obstacles in the larger environment in which such research takes place. The quality of data available to researchers on the state and local level is often poor. Demographic data are inconsistent and unreliable, and test data often suffer from problems inherent in the nature or administration of the test. Although the data available in some states (e.g., Texas and North Carolina) are superb, this is hardly the case in most others. Moreover, even when the data are of reasonable quality, states and districts frequently restrict researchers' access to it out of fear of violating rules of confidentiality. In fact, most of the time such concerns about disclosure can be eliminated by assigning unique identifiers anonymously to the subjects involved.

The Role of Policymakers, Educators and Others: 20 Recommendations

It is clear that sweeping structural changes are needed if research in education is to live up to its potential. Providing a prescription for such changes lies beyond the scope of this report, however. Besides, any recommendations offered here would doubtless echo those of other reports, notably the National Research Council's (NRC) Improving Student Learning: A Strategic Plan for Education Research and Its Utilization (1999), Scientific Research in Education (2002), and its recently released Strategic Education Research Partnership (2003).

Even short of such sweeping changes, it is clear that policymakers and practitioners themselves have a role to play in making education research more responsive to their needs. Otherwise, there is not likely to be substantial progress in the ability of education research to address the most important questions that policymakers and practitioners have concerning education. In particular:

* Policymakers and practitioners need to create a demand for better, more targeted research.
* Policymakers and practitioners have to be willing to provide the funding necessary for improved research and to make available the data and other resources researchers need.
* Above all, policymakers and practitioners have to know how to use education research effectively.
Indeed, many of the issues facing education research are not so much problems with the nature or quality of the research itself or with inadequate funding, but rather with research utilization. Both education researchers, on the one side, and policymakers and practitioners, on the other, bear responsibility for addressing significant differences in the cultures of research, practice and policy that promote mutual suspicion and impede fruitful communication. Among the problems:

* The lengthy time cycle necessary for many research studies creates a barrier because it does not allow research to be responsive to the immediate needs of policymakers.
* Policymakers have an erroneous expectation that research will bring forth definitive answers as needed, and don't understand that research is a matter of successive approximations – the accumulation of evidence over time.
* Policymakers must consider not only the likely success but also the costs and feasibility of implementing particular programs and strategies. Research in education rarely addresses those issues.
* Given the compromises necessary in the political process, the implementation of programs along the lines suggested by research is rarely carried out with fidelity. On the one hand, this makes careful research seem that much less valuable and important to policymakers. On the other hand, it makes researchers doubtful that policymakers are interested in good research or are an audience whose needs for information merit the researchers' attention.

Nevertheless, in spite of the differing needs and cultures of researchers, practitioners and policymakers, there are possibilities for bridging the gaps to bring about both more effective research utilization and greater research support. Education researchers, policymakers, practitioners and others can take steps both on their own and together to help enhance the quality of education research and its significance for practice and policy.

This report concludes, then, by offering 20 recommendations for action that various stakeholders in education might take.

What Education Researchers Can Do

1. Make a systematic and concerted effort to disseminate important research findings. Education research often remains inaccessible to policymakers and practitioners, either because it has not been made readily available to them or because, though available, it is not readily comprehensible.

2. Assume greater responsibility for educating the public about the characteristics of good research, and developing ways to help the public readily distinguish good research studies from inferior ones.

3. Work with practitioners, teacher educators, policymakers and funders to help them improve the quality and significance of nonacademic research and evaluation of specific classroom practices or local programs. Increased quality, together with increased accessibility, of such local studies could make them a significant addition to the larger body of existing research in education.

4. Do a better job of addressing the most urgent policymaker and practitioner needs. The agenda in education research is frequently set without any input from policymakers and practitioners.

What Policymakers Can Do

5. Improve their own ability to appreciate both the importance of making evidence-based decisions and the value, complexities and requirements of good research.
* The existence of term limits in many states unfortunately makes it difficult for policymakers to develop the expertise they need on basic issues, let alone on the research.
* Professional staff and the various organizations that serve policymakers can play a critical role in strengthening a policymaker's understanding and appreciation of research, and helping them use research in making policy decisions.

6. Provide access for researchers to high-quality data without compromising the confidentiality of students and teachers. For their own self-interest, policymakers ought to commit to improving the capacity of their states and districts to generate and collect good data.

7. Support high-quality evaluation studies of the policies and programs they enact and provide adequate funding to carry those out.
* Particularly in times when money is tight, it is difficult to fund research instead of direct services. Policymakers have an obligation, however, to ensure the programs and services they fund are effective, and only sound research is truly capable of providing that information.
* It also would be helpful for policymakers to make the research studies they fund widely available, adding to the existing body of education research.

8. Work to align the education research agenda with policy priorities by coming to agreement on the relative importance of various issues for which research evidence would be helpful.

What Practitioners, Including Teacher Educators, Can Do

9. Ensure that graduates of teacher education programs are given a basic understanding of research and research methods, and that they are committed to evidence-based practice.

10. Ensure that professional development for teachers and administrators strengthens their basic understanding of research and research methods, and their commitment to evidence-based practice.

11. Work with researchers to increase the quality of the studies and evaluations they conduct on classroom practices and local programs. It also would be helpful for practitioners, perhaps working with researchers, to make significant studies and evaluations widely available.

12. Work with local education leaders to help create the conditions in which high-quality education research can take place. They can make possible, for example, the random assignment of students to different teachers or different kinds of programs, carry out controlled pilot studies of a new program or curriculum, and keep careful records of program details and impacts that can provide researchers with invaluable data.

What Funders Can Do

13. Commit to relying more heavily on solid research in developing funding priorities and evaluating requests for funding.

14. Ensure that they and their colleagues receive the training necessary for a basic understanding of education research and research methods.

15. Support high-quality evaluation studies of the policies and programs they enact and provide adequate funding to carry them out.

* Funders, like policymakers, have an obligation to ensure that the programs and services they fund are effective, and only sound research is truly capable of providing that information.
* It would be helpful for funders to make high-quality program evaluation studies widely accessible so those studies can be added to the body of available research.

16. Increase commitment to fund education research, in addition to basic programs and services. Solid research provides a scaffold upon which more-effective programs and services can be built.

What Researchers, Policymakers and Practitioners Can Do Together

17. Discuss differences and better appreciate one another's mutual priorities and needs. Consistent and ongoing communication might yield several significant results, including (a) a greater respect, and ultimately greater support, for research among policymakers and practitioners and (b) the development of avenues – and perhaps a more formal structure – for disseminating research in a more timely fashion.

18. Develop and continually fine-tune an education research agenda that both provides decisionmaking guidance for policymakers and respects and fulfills the professional aspirations of researchers. Once research priorities are identified, foundations and other funding agencies can play a key role in supporting the necessary research, ensuring broad dissemination of the findings and using the findings to guide funding of education programs.

19. Establish several education research journals – targeted at education professionals but accessible to the broader community – featuring high-quality research studies addressing issues of particular interest and importance. Publications that play a role in education similar to that played in medicine by the New England Journal of Medicine or the Journal of the American Medical Association could go a long way toward heightening the visibility of and respect for education research.

20. Regularly bring together leading education researchers, practitioners, policymakers and others to review the status of knowledge about key issues facing American education and identify research that is needed to further illuminate the issues.
* The What Works Clearinghouse, established by the U.S. Department of Education's Institute of Education Sciences, accomplishes some of this purpose. Its primary focus is on reviews of research related to specific interventions, such as preventing high school dropout or teaching beginning reading. Visit the What Works Clearinghouse at www.w-w-c.org.
* Another model to consider is the Consensus Development Conferences of the National Institutes of Health (NIH). The conferences involve an independent panel of experts that develops a consensus statement on the issues under discussion; for example, whether acupuncture is effective. The panel relies on a review of the relevant research literature, and listens to both scientific data presented by invited experts and to audience comments in open hearings. To learn more about the conferences, visit the NIH Web site at consensus.nih.gov.

© 2003 ECS

Early Learning Issues K-12 Issues Postsecondary Issues Issue Site Sponsors NCLB (ECS Issue Site)
Information & Links State of the States 2004
Upcoming Meetings The National Forum on Education Policy
News Releases ECS e-Clips ECS e-Connection ECS Governance Notes TQ Update Citizenship Matters Media Links
Publications by Topic How to Order
What We Do Officers & President Staff Distinguished Senior Fellows History Employment Opportunities ECS Awards ECS Commissioners' Page
Main Projects Page ECS Chairman's Initiative Accountability Community Colleges Early Learning Finance Governance Postsecondary Service-Learning/Community Service Leadership Reading/Literacy Teaching Quality
Federal Agencies Regional Education Labs National Organizations Research Centers Media
Search

| BACK TO RESEARCH |