Home / Medical Education / Councils and Committees / National Academy of Osteopathic Medical Educators / Selection Process

Selection Process

National Academy of Osteopathic Medical Educators logo

Review Panel 

The NAOME Review Panel will be selected largely from a reviewers' pool consisting of Fellows in NAOME. Until the NAOME Fellow numbers reach the necessary number for the review panel, additional members of the Review Panel will be selected from a pool of reviewers designated as educational leaders from each COM and representing the educational impact categories of the applicants.

Reviewer Process

Using protocols of the National Institutes of Health, a primary and secondary reviewer will be designated for each submission. These reviewers will propose initial ratings for all criteria, lead the discussion of the applicant's materials during the review meeting, and help write brief critiques in support of selection decisions.

After the review panel discussion of each candidate, each member of the Review Panel (having reviewed each mini-portfolio in advance) will assign up to 100 points to each portfolio based on how well the mini-portfolio matches the standards for a given awards category, as represented by the standard-setting mini-portfolio samples provided (see Portfolio Preparation).

To receive an award, the average number of points assigned by Review Panel members must meet or exceed the established cutoff of 80 points, and 75 percent of the Review Panel members must give a score of 80 or higher. Panelists will assume that the standard-setting mini-portfolio examples would receive an average of 85 to 95 points.

The process primary and secondary reviewers will use in assigning points will be subdivided into three criteria as illustrated below for the teaching/learning category (see point assignments for other categories with category description): 

Criteria for Determining Quality of Scholarship

(From Scholarship Assessed, Glassik, Huber, and Maeroff, 1997)

These questions illustrate how the criteria apply to multiple areas of scholarship, including scientific and educational research. Based on the award category, members of the peer review panel will look for answers to questions such as these in the information you present in your portfolio.

CriteriaClarifying Questions
Clear, realistic and important goals and/or philosophy
  • Is educational endeavor important to mission of College?
  • Are goals specific and obtainable?
  • Are goals consistent with stated leadership philosophy?
  • Do goals reflect needs of profession, society, learners, other faculty?
  • Is the development of enduring materials guided by a cohesive set of professional goals?
  • Is researchers' line of research important to the field?
Adequate preparation
  • Is teacher prepared to teach effectively?
  • Does leader strive to continuously learn new ways of dealing with challenging issues?
  • Does educator take advantage of educator development opportunities (MTFP, peer review, skill building workshops)?
  • Does faculty have skills matching his/her role in preparing enduring educational materials?
  • Is researcher qualified to conduct research effectively?
Appropriate methods
  • Does teacher use appropriate teaching techniques?
  • Is design of course effective?
  • Does leader get others meaningfully involved?
  • Are actions consistent with current literature?
  • Are systematic instructional design methods used to prepare enduring educational materials?
  • Is design of study appropriate?
  • Does study have sufficient statistical power?
Meaningful results
  • Does educational strategy (e.g., teaching method, course management) serve as model for others?
  • Were stated goals achieved?
  • Do learners use the enduring educational materials as intended? Are desired learning outcomes achieved?
  • Does research study lead to outcomes worthy of publication in the literature?
Effective presentation (Sharing strategies or experiences with peers)
  • Are "lessons" learned about teaching, educational leadership or the preparation of enduring educational materials shared with peers at local, regional or national levels?
  • Do peer reviewers for grant, journal and/or educational award find the presentation of results understandable and credible?
  • Is write-up of research results credible to a local, regional and/or national audience?
Reflective critique
  • Does teacher, leader, or developer solicit and use feedback from learners and peers?
  • Does educator examine multiple perspectives before changing strategies?
  • Does researcher solicit and effectively use advice from colleagues/mentors?

Point Assignments

Criteria

 

Use sample questions after each criterion as a guide for interpretation.

Maximum Points Possible

 

1.  The applicant’s personal standards of osteopathic focus or impact within/upon osteopathic medical education. (The standards of osteopathic focus or impact within/upon osteopathic medical education will be awarded based on the applicant's personal statement and mini-portfolio.

10 points

Application Quality – evidence of quality may also be found in multiple places throughout the portfolio including summary statement, personal statement, and appendices.

2. Clear, realistic, and important goals

(e.g., Are educational research goals specific and appropriate? Do goals reflect the needs of learners and/or educators?)

 

5 points

3. Adequate personal preparation and ongoing self-reflection/improvement

(e.g., Is researcher prepared to investigate effectively? Does researcher participate in educator development opportunities? Does educator solicit and effectively use evaluations from learners and peers?)

 

10 points

4. Adequate methods/Quality of presentation of results (dissemination)

(e.g., Are appropriate educational research techniques used effectively and flexibly? Does research strategy serve as a model for others? How do peers regard process and perceive outcomes? Is educational research portfolio clear? Does educator effectively share lessons learned with peers?)

 

30 points

5. Quantity

(e.g., Number of specific questions examined, number of initiatives (discrete studies) associated with each question, and the number of disseminations. Investment of time and effort in systematically approaching educational questions including dissemination.)

 

35 points

6. Breadth and/or Depth/Scope

(e.g., Different thematic areas, different types of questions focused on different educational issues and/or learner populations, different types of research methods, different venues for disseminating results.)

10 points

Total (80 points minimum needed to receive award)

100 points