Research-Based Recommendations for Improving Peer Review
PHILOSOPHICAL CONSIDERATIONS...
- Choose a goal for the process: is it summative, or is it formative? (Keig & Waggoner, 1994; Centra, 1993)
"Instructors," reports Centra, "will not be as open to discussing weaknesses or seeking advice from people who will judge them." - Provide training for peer reviewers. (Millis & Kaplan, 1997; Keig & Waggoner, 1994)
Training of peer reviewers has been found to increase validity and reliability of ratings. - Have mutually agreed upon criteria and points of observation. (Millis & Kaplan, 1997; Weimer, Kerns & Parrett, 1988)
Without an agreement of criteria, reviewees are more likely to view a summative evaluation as "unfair," and a formative evaluation is less likely to have a positive effect. - Peer review should involve review of the class session and associated materials (e.g., syllabus, handouts, assignments). (Malik, 1996; Webb & McEnerney, 1995)
Course materials give context to the in-class observations and, invariably, an important part of what happens within the classroom. - Provide a small renumeration to peer reviewers. (Millis & Kaplan, 1997)
Peer reviewers who have received a stipend, even as little as $50, report the experience to be more satisfying.
PERSONNEL CONSIDERATIONS...
- "Reciprocal" peer review is more effective than "expert" peer review in changing teaching behavior. (Skinner and Welch, 1996; Brightwell, 1993; Weimer, 1990)
In the words of one faculty member, "I want to be evaluated by people who know they are still learning how to teach, rather than by those who think they already know." - If your goal is to assess only pedagogical effectiveness (not content prowess), use peer reviewers from outside the discipline. (Skinner & Welch 1996; Webb & McEnerney, 1995; Millis, 1989; Weimer, Kerns & Parrett, 1988)
Reviewers within the discipline have been found to be more likely to fixate on evaluating content knowledge. Non-discipline reviewers share a perspective, frequently, more akin to the student. - Use peer reviewers who are volunteers. (Millis & Kaplan, 1997; Webb & McEnerney, 1995; Weimer, Kerns & Parrett, 1988)
Predictably, peer reviewers who volunteer to participate are more likely to adhere to the process and provide feedback likely to yield improvement in formative situations.
PROCESS CONSIDERATIONS...
- Combine peer review with self-review. (Millis & Kaplan, 1997; Kumaravadivelu, 1995; Brinko, 1993)
Self-reflection prompted by self-review often gives clarity to the observations of the peer reviewer in summative evaluation, and is a catalyst for improvement in formative evaluations. - Peer reviewers should visit a class at least three times in one semester. (Millis & Kaplan, 1997; Stoner & Martin, 1993; Centra, 1993)
This practice increases the validity and reliability of the evaluation. Millis and Kaplan (1997) suggest multiple visits, with some visits devoted to formative evaluation and others to summative. - It should be interactive, with the reviewer and reviewee meeting for discussion prior to and after each classroom visit. (Millis & Kaplan, 1997; Kumaravadivelu, 1995; Webb & McEnerney, 1995; Keig & Waggoner, 1994)
A process approach, with pre-meetings and debriefings—increase participant satisfaction and increase the likelihood of change in instructional behavior. Debriefing sessions should occur no later than three days after the observation. - Peer reviewer should interview students of the classes observed. (Morehead & Shedd, 1996; Kumaravadivelu, 1995)
This procedure provides the opportunity to validate many of the reviewer's observations about the events of the classroom. - Videotape classroom sessions that are peer observed. (Millis & Kaplan, 1997; Brinko, 1993)
This practices not only informs self-evaluation, but it helps focus the dialogue on specific instructional behaviors—a factor key in effective formative evaluation. - Have faculty being reviewed evaluate the process. (Skinner & Welch, 1996)
This practice increases reviewee satisfaction with the process and increases the potential for change in instructional behavior in formative evaluation.
INFORMATION & SOURCES ON PEER EVALUATION/REVIEW
Ackland, R. (1991). A review of the peer coaching literature. Journal of Staff Development 12(1): 22-27.
Bell, Thomas L., and McClam, Tricia. (Spring 1992). Peer review of teaching at UTK: An assessment. Teaching/Learning Issues; n70.
Brinko, K. (1993). The practice of giving feedback to improve teaching: What is effective? Journal of Higher Education. 64(5): 574-594.
Bronowski, C., Toms-Bronowski, S., & Bearden, K.J. (1993). Teacher observation forms: A new look at an old technique. NASSP Bulletin, 77, 30-38.
Centra, J.A. (1993). Reflective faculty evaluation. San Francisco: Jossey-Bass.
Collaborative Peer Review Report. (March 1996). Peer reviews affect the teaching culture of an institution. Journal of Instructional Psychology, 23(1): 8.
Cox, M.D., & Richlin, L. (1993). Emerging trends in college teaching for the 21st century. Journal on Excellence in College Teaching, 4, 1-7.
Fink. L. (1991). Evaluating your own teaching. In P. Seldin & Associates (Ed.), Improving College Teaching, (pp. 191-204). Boston, MA: Anker Publishing Company, Inc.
Hamilton, Neil W. (May-June, 1997). Peer review: The linchpin of academic freedom and tenure. Academe; v83 n3 p15-19.
Hutchings, P. (1994). Peer review of teaching. AAHE Bulletin 47(3): 3-7.
Hutchings, P. (1996). Making reaching community property: A menu for peer collaboration and peer review. Washington, D.C. AAHE Teaching Initiative. (ERIC Document Reproduction # ED406955)
Keig, Larry, & Waggoner, Michael D. (1994). Collaborative peer review: The role of faculty in improving college teaching. Washington, D.C. ASHE-ERIC Higher Education Report No. 2.
Malik, David J. (1996). Peer review of teaching: External review of course content. Innovative Higher Education 20 (4): 277-86.
McIntosh, Thomas H., & Van Koevering, Thomas E. (1986). Six-year case study of faculty peer reviews, merit rating, and pay awards in a multidisciplinary department. Journal of the College and University Personnel Associations; v37 n1: 5-14.
Menges. R.J. (1987). Colleagues as catalysts for change in teaching. To Improve the Academy. 6: 83-93.
Menges, Robert J. (1991). Why hasn't peer evaluation of college teaching caught on? Presentation at the meeting of the American Educational Research Association, Chicago, IL.
Millis, B.J. (1989). Colleagues helping colleagues: A peer observation program model. The Journal of Staff, Program, and Organization Development 7(1): 15-21.
Millis, B.J. (1992). Conducting effective peer classroom observations. To Improve the Academy, 11, 189-201.
Millis, B.J. (1994). Forging the ties that bind: Peer mentoring part-time faculty. In M.A. Wunsch (Ed.), New Directions for Teaching and Learning: No. 57. Mentoring revisited: Making an impact on individuals and institutions. San Francisco, CA: Jossey-Bass.
Millis, B.,& Kaplan, B. (1997). Enhancing teaching through peer classroom observations. In P. Selding & Associates (Ed.), Improving College Teaching, (pp. 137-151). Bolton, MA: Anker Publishing Company, Inc.
Morehead, Jere W. & Shedd, Peter J. (1996). Student interviews: A vital role in the scholarship of teaching. Innovative Higher Education, 20(4): 261-69.
Nyquist (Eds.),: Resources for faculty, instructional, & organizational development.. To Improve the Academy. Stillwater, OK: POD/New Forums Press.
Osborne, W. Larry, & Purkey, William W. (1995). A model faculty peer review process for counselor education programs. Journal of Counseling & Development. 73(6): 654-658.
Romberg, Elaine. (1985). Evaluating teaching in the health professions: Peers as evaluators of teaching. Paper presented at the Annual Meeting of the American Educational Research Association (69th, Chicago, IL).
Roworth, Wendy Wassyng. (May-June 1997). Pandora's Dilemma: Some reflections on peer review. Academe; 83(3): 35-38.
Seldin & Associates (Ed.), Improving College Teaching, (pp. 137-151). Bolton, MA: Anker Publishing Company, Inc.
Sheppard, Sherry L.; and others. (1996). Commentary on student interviews. Innovative Higher Education; 20(4),: 271-76.
Skinner, M. & Welch, F. (1996). Peer coaching for better teaching. College Teaching. 44:4, pp. 153-156.
Stoner, M., & Martin, L. (1993, November). Talking about teaching across the disciplines: How cognitive peer coaching makes it happen. Paper presented at the 79th Annual Conference of the Speech Communication Association, Miami, FL.
Walen, E., & DeRose, M. (1993). The power of peer appraisals. Educational leadership, 51: 45-48.
Webb, J. L., & McEnerney, K. (1994). The view from the back of the classroom: Teacher observation/peer support program (TOPS). Arizona State University, Tempe: International Society for Exploring Teaching Alternatives (ISETA) Conference.
Weimer, Maryellen Gleason, Kerns, Mary-Margaret &. Parrett, Joan L. (1988). Instructional observation: Caveats, concerns, and ways to compensate. Studies in Higher Education, 13( 3): 285-293.
Weinback, Robert W.& Randolph, Jerry L. (1984). Peer Review for Tenure and Promotion in Professional Schools. Improving College and University Teaching, 32(2): 81-86.
