Empirically-Identified Problems &
Advantages of Peer Review
Problems
- If practiced properly, it is a very time-consuming.
- Faculty widely hold the belief that "teaching is a private, personal undertaking—very different than research."
- Its practice posses a threat to collegial relationships.
- Its practice is a risk to academic freedom.
- The findings are not reliable and valid.
- If it is used "summatively," it serves no value "formatively," and visa-versa.
- Mandated guidelines, even those that are simple and broadly defined, are rarely adhered to (faculty subterfuge).
- The least favored form of review among women and minority faculty.
(Roworth, 1997; Centra, 1993; Bell & McClam, 1992)
Advantages
- Faculty can provide formative and/or summative information not available from any other constituency.
- The peer review has been found to be the essential component in formative evaluation—the source most likely to prompt positive changes in teaching behavior.
- Faculty engaged in the peer review process report high levels of satisfaction with the process.
- Faculty engaged in the process report higher levels of perceived collegiality.
- Faculty engaged in the peer review process report higher levels of professional motivation.
- Faculty functioning as peer reviewers report benefits to their teaching.
(Millis & Kaplan, 1997; Skinner & Welch 1996; Keig & Waggoner, 1994; Centra, 1993)
PHILOSOPHICAL CONSIDERATIONS…
- Choose a goal for the process: is it summative, or is it formative? (Keig & Waggoner, 1994; Centra, 1993)
"Instructors," reports Centra, "will not be as open to discussing weaknesses or seeking advice from people who will judge them."
- Provide training for peer reviewers. (Millis & Kaplan, 1997; Keig & Waggoner, 1994)
Training of peer reviewers has been found to increase validity and reliability of ratings.
- Have mutually agreed upon criteria and points of observation. (Millis & Kaplan, 1997; Weimer, Kerns & Parrett, 1988)
Without an agreement of criteria, reviewees are more likely to view a summative evaluation as "unfair," and a formative evaluation is less likely to have a positive effect.
- Peer review should involve review of the class session and associated materials (e.g., syllabus, handouts, assignments). (Malik, 1996; Webb & McEnerney, 1995)
Course materials give context to the in-class observations and, invariably, an important part of what happens within the classroom.
- Provide a small renumeration to peer reviewers. (Millis & Kaplan, 1997)
Peer reviewers who have received a stipend, even as little as $50, report the experience to be more satisfying.
PERSONNEL CONSIDERATIONS…
- "Reciprocal" peer review is more effective than "expert" peer review in changing teaching behavior. (Skinner and Welch, 1996; Brightwell, 1993; Weimer, 1990)
In the words of one faculty member, "I want to be evaluated by people who know they are still learning how to teach, rather than by those who think they already know."
- If your goal is to assess only pedagogical effectiveness (not content prowess), use peer reviewers from outside the discipline. (Skinner & Welch 1996; Webb & McEnerney, 1995; Millis, 1989; Weimer, Kerns & Parrett, 1988)
Reviewers within the discipline have been found to be more likely to fixate on evaluating content knowledge. Non-discipline reviewers share a perspective, frequently, more akin to the student.
- Use peer reviewers who are volunteers. (Millis & Kaplan, 1997; Webb & McEnerney, 1995; Weimer, Kerns & Parrett, 1988)
Predictably, peer reviewers who volunteer to participate are more likely to adhere to the process and provide feedback likely to yield improvement in formative situations.
PROCESS CONSIDERATIONS…
- Combine peer review with self-review. (Millis & Kaplan, 1997; Kumaravadivelu, 1995; Brinko, 1993)
Self-reflection prompted by self-review often gives clarity to the observations of the peer reviewer in summative evaluation, and is a catalyst for improvement in formative evaluations.
- Peer reviewers should visit a class at least three times in one semester. (Millis & Kaplan, 1997; Stoner & Martin, 1993; Centra, 1993)
This practice increases the validity and reliability of the evaluation. Millis and Kaplan (1997) suggest multiple visits, with some visits devoted to formative evaluation and others to summative.
- It should be interactive, with the reviewer and reviewee meeting for discussion prior to and after each classroom visit. (Millis & Kaplan, 1997; Kumaravadivelu, 1995; Webb & McEnerney, 1995; Keig & Waggoner, 1994)
A process approach, with pre-meetings and debriefings—increase participant satisfaction and increase the likelihood of change in instructional behavior. Debriefing sessions should occur no later than three days after the observation.
- Peer reviewer should interview students of the classes observed. (Morehead & Shedd, 1996; Kumaravadivelu, 1995)
This procedure provides the opportunity to validate many of the reviewer's observations about the events of the classroom.
- Videotape classroom sessions that are peer observed. (Millis & Kaplan, 1997; Brinko, 1993)
This practices not only informs self-evaluation, but it helps focus the dialogue on specific instructional behaviors—a factor key in effective formative evaluation.
- Have faculty being reviewed evaluate the process. (Skinner & Welch, 1996)
This practice increases reviewee satisfaction with the process and increases the potential for change in instructional behavior in formative evaluation.
