NBME OSCE Creative Community

Launched by NBME in 2022, the Creative Community brought together assessment experts, faculty educators, students and NBME staff from ten U.S. schools to explore how simulation-based formats, specifically standardized patient OSCEs, might better capture the process of diagnostic reasoning rather than only its outcomes. The Community identified a central limitation in prevailing assessment approaches: many instruments emphasize whether learners arrive at correct or plausible conclusions but offer little visibility into how learners generated those conclusions or how they might improve their diagnostic approach.

To address this gap, the group focused on developing assessment formats that reveal and measure learners’ cognitive processes during clinical encounters. Drawing on the combined expertise of NBME staff and participating institutions, the Creative Community developed a novel OSCE case format designed to assess behaviors associated with Hypothesis-Driven Information Gathering. This construct centers on a learner’s ability to generate and iteratively refine diagnostic hypotheses as new information is gathered, and to use those hypotheses to guide subsequent information gathering. The format shifts the emphasis away from end-point diagnostic accuracy toward the observable steps and decisions that constitute clinical reasoning in real time.

The Community’s design work aimed not only to produce an assessment instrument but also to create meaningful, developmentally oriented feedback that learners can use to strengthen their reasoning skills. From 2022 to 2024, pilot studies of the new case format were conducted. Initial pilots demonstrated alignment between learners’ cognitive processes and the assessment design, providing preliminary validity evidence for this approach. These findings suggest the format can capture intended aspects of reasoning and offer actionable insights into how learners think during clinical encounters. While the early results are promising, additional refinement is required both for the assessment itself and for the developmental feedback that accompanies it. The work thus represents a stepping stone toward more nuanced assessment strategies capable of addressing hard-to-measure constructs such as clinical reasoning. The Creative Community included ten schools that participated as thought partners and research collaborators.

 The Community’s approach models how collaborative, multidisciplinary partnerships can generate innovative solutions to complex educational assessment challenges. By prioritizing the measurement of reasoning processes and creating assessment designs that align with those processes, this work has advanced a promising direction for competency-based medical education. Ongoing refinement and further study will be needed to realize the full potential of this work and to translate these innovations into scalable assessment and feedback practices for learners across the continuum.