Implementing calibrated peer review to enhance technical critiquing skills in a bioengineering laboratory
Developed at UCLA, Calibrated Peer Review™ (CPR) is a web-based tool developed to help students improve their technical writing and critiquing skills. In 2006 and 2007 we used CPR in an upper-level tissue culture laboratory course in which students conduct viability, attachment, and proliferation assays using fibroblast cells. After completing their experiments, students use PowerPoint to construct a technical poster that illustrates their experimental methods, results, and conclusions. For the CPR component of the assignment, students first evaluate three sample posters supplied by the instructor to calibrate their critiquing skills. After this step, students conduct a blind review of three peers' posters and then evaluate their own. During the calibration, peer critiquing and self evaluation stages, students respond to 15 statements about the quality of the posters. Eleven statements cover technical content, including succinct summary of objectives, clear experimental methods, quality of graphs, and key results interpreted in words. Three statements probe the poster's visual appeal, including appropriate size and style of font. One final statement requires a holistic evaluation of the poster. Following CPR, students turn in a revised copy of their technical poster. In 2006, students had difficulty during the calibration phase. Following a major revision of the calibration phase in 2007, 79% of students passed all three calibration posters. Instructor, peer, and self evaluations were compared. There was a strong linear correlation between instructor evaluation and peer evaluation (r = 0.60, regression model ANOVA P<0.0002). In contrast, there were poor linear correlations between instructor and self evaluations and between peer and self evaluations (r < 0.25, regression model ANOVA P>0.2). These results suggest that students may be better able to technically evaluate others' work, rather than their own. Students perceived the peer evaluation process as generally helpful, although they noted that their peers' comments were less specific and occasionally inconsistent with their instructor's feedback. Students reported on surveys that peer evaluation was effective in helping them to recognize many facets of technical poster design, such as errors and omissions, data presentation, and technical argument. 97% of the students claimed their technical critiquing skills improved as a result of this experience. We feel that using CPR to facilitate the peer evaluation process is an effective way to enhance undergraduate engineering students' technical critiquing skills. © American Society for Engineering Education, 2008.
