William P. Robinson2, Odette Taha2, Donald T. Baril2, Andres Schanzer2, Jean Bismuth3, Erica L. Mitchell1
1Vascular Surgery, Oregon Health & Science University, Portland, OR; 2UMass Memorial Medical Center, Worcester, MA; 3Methodist Cardiovascular Surgery Associates, Houston, TX.
OBJECTIVES: Simulation-based training has become an accepted and increasingly utilized method for surgical skill acquisition. However, validated tools to assess technical performance and provide feedback have not been developed for simulated index vascular surgical procedures. We sought to develop and validate a tool for simulated open abdominal aortic aneurysm repair (SOAAAR).
METHODS: Phase 1: Initial tool development based on current practice and expert consensus. Phase 2: Refinement of the tool based on independent, blinded assessor feedback and psychometric testing including feasibility, reliability (internal consistency, inter-assessor reliability) and construct validity.
RESULTS: Phase 1: Three successive iterations of the tool were developed based on expert opinion and psychometric testing. Tool Included procedural checklist, global rating score, final product analysis and overall assessment of operative competence. Phase 2: Internal consistency and inter-assessor reliability improved from the first to third version. A trend was also demonstrated for increasing construct validity (intra-class correlation coefficient=0.72, p=0.0008) with the third iteration of the tool. The final SOAAR tool was feasible, reliable (internal consistency using Cronbach’s α coefficient = 0.826, inter-assessor reliability using spearman’s rho coefficient yielded p<0.05), and face and content valid (content validity index=0.94).
CONCLUSIONS: A psychometrically robust assessment tool for assessing performance of SOAAR was developed and validated. This tool allows for objective assessment of resident performance during open aneurysm repair in the simulated setting. The tool can be used to identify areas for improvement in resident performance and to evaluate and optimize simulation-based training.
AUTHOR DISCLOSURES: D. T. Baril, Nothing to disclose; J. Bismuth, Nothing to disclose; E. L. Mitchell, Nothing to disclose; W. P. Robinson, Nothing to disclose; A. Schanzer, Nothing to disclose; O. Taha, Nothing to disclose.
Posted April 2012