Skip to main content
Journal cover image

Early findings and strategies for successful implementation of SIMPL workplace-based assessments within vascular surgery residency and fellowship programs.

Publication ,  Journal Article
Cox, ML; Weaver, ML; Johnson, C; Chen, X; Carter, T; Yee, CC; Coleman, DM; Sgroi, MD; George, BC; Smith, BK
Published in: J Vasc Surg
September 2023

OBJECTIVE: As medical education systems increasingly move toward competency-based training, it is important to understand the tools available to assess competency and how these tools are utilized. The Society for Improving Medical Professional Learning (SIMPL) offers a smart phone-based assessment system that supports workplace-based assessment of residents' and fellows' operative autonomy, performance, and case complexity. The purpose of this study was to characterize implementation of the SIMPL app within vascular surgery integrated residency (0+5) and fellowship (5+2) training programs. METHODS: SIMPL operative ratings recorded between 2018 and 2022 were collected from all participating vascular surgery training institutions (n = 9 institutions with 5+2 and 0+5 programs; n = 4 institutions with 5+2 program only). The characteristics of programs, trainees, faculty, and SIMPL operative assessments were evaluated using descriptive statistics. RESULTS: Operative assessments were completed for 2457 cases by 85 attendings and 86 trainees, totaling 4615 unique operative assessment ratings. Attendings included dictated feedback in 52% of assessments. Senior-level residents received more assessments than junior-level residents (postgraduate year [PGY]1-3, n = 439; PGY4-5, n = 551). Performance ratings demonstrated increases from junior to senior trainees for both resident and fellow cohorts with "performance-ready" or "exceptional performance" ratings increasing by nearly two-fold for PGY1 to PGY5 residents (28.1% vs 40.6%), and from first- to second-year fellows (PGY6, 46.7%; PGY7, 60.3%). Similar gains in autonomy were demonstrated as trainees progressed through training. Senior residents were more frequently granted autonomy with "supervision only" than junior residents (PGY1, 8.7%; PGY5, 21.6%). "Supervision only" autonomy ratings were granted to 21.8% of graduating fellows. Assessment data included a greater proportion of complex cases for senior compared with junior fellows (PGY6, 20.9% vs PGY7, 26.5%). Program Directors felt that faculty and trainee buy-in were the main barriers to implementation of the SIMPL assessment app. CONCLUSIONS: This is the first description of the SIMPL app as an operative assessment tool within vascular surgery that has been successfully implemented in both residency and fellowship programs. The assessment data demonstrates expected progressive gains in trainees' autonomy and performance, as well as increasing case complexity, across PGY years. Given the selection of SIMPL as the assessment platform for required American Board of Surgery and Vascular Surgery Board Entrustable Professional Activities assessments, understanding facilitators and barriers to implementation of workplace-based assessments using this app is imperative, particularly as we move toward competency-based medical education.

Duke Scholars

Published In

J Vasc Surg

DOI

EISSN

1097-6809

Publication Date

September 2023

Volume

78

Issue

3

Start / End Page

806 / 814.e2

Location

United States

Related Subject Headings

  • Workplace
  • Vascular Surgical Procedures
  • Internship and Residency
  • Humans
  • General Surgery
  • Fellowships and Scholarships
  • Education, Medical, Graduate
  • Clinical Competence
  • Cardiovascular System & Hematology
  • 3202 Clinical sciences
 

Citation

APA
Chicago
ICMJE
MLA
NLM
Cox, M. L., Weaver, M. L., Johnson, C., Chen, X., Carter, T., Yee, C. C., … Smith, B. K. (2023). Early findings and strategies for successful implementation of SIMPL workplace-based assessments within vascular surgery residency and fellowship programs. J Vasc Surg, 78(3), 806-814.e2. https://doi.org/10.1016/j.jvs.2023.04.039
Cox, Morgan L., M Libby Weaver, Cali Johnson, Xilin Chen, Taylor Carter, Chia Chye Yee, Dawn M. Coleman, Michael D. Sgroi, Brian C. George, and Brigitte K. Smith. “Early findings and strategies for successful implementation of SIMPL workplace-based assessments within vascular surgery residency and fellowship programs.J Vasc Surg 78, no. 3 (September 2023): 806-814.e2. https://doi.org/10.1016/j.jvs.2023.04.039.
Cox, Morgan L., et al. “Early findings and strategies for successful implementation of SIMPL workplace-based assessments within vascular surgery residency and fellowship programs.J Vasc Surg, vol. 78, no. 3, Sept. 2023, pp. 806-814.e2. Pubmed, doi:10.1016/j.jvs.2023.04.039.
Cox ML, Weaver ML, Johnson C, Chen X, Carter T, Yee CC, Coleman DM, Sgroi MD, George BC, Smith BK. Early findings and strategies for successful implementation of SIMPL workplace-based assessments within vascular surgery residency and fellowship programs. J Vasc Surg. 2023 Sep;78(3):806-814.e2.
Journal cover image

Published In

J Vasc Surg

DOI

EISSN

1097-6809

Publication Date

September 2023

Volume

78

Issue

3

Start / End Page

806 / 814.e2

Location

United States

Related Subject Headings

  • Workplace
  • Vascular Surgical Procedures
  • Internship and Residency
  • Humans
  • General Surgery
  • Fellowships and Scholarships
  • Education, Medical, Graduate
  • Clinical Competence
  • Cardiovascular System & Hematology
  • 3202 Clinical sciences