Program Director’s Corner – April 2020

Fellow evaluation of faculty members

Christopher McCartney, MD
Program Director, Endocrine Fellowship Program
University of Virginia

[NOTE: ACGME program requirements related to Faculty Evaluation are placed in an Appendix located at the end of this entry.]

Historically, our program’s faculty members uniformly received excellent evaluations from fellows. However, early in my tenure as program director, I sensed an occasional discordance between (a) a given faculty member’s excellent written evaluations (provided by individual fellows) and (b) informal fellow comments about that faculty member. This implied that our system of faculty evaluation might be unreliable in some cases. After some exploration, I learned that fellows had general concerns about our evaluation system: some worried that it was easy for faculty members to determine which fellow wrote a particular evaluation, and some fellows were reluctant to provide negative feedback of faculty in writing (even if completely anonymous). To help address these issues, our program developed a process designed to enhance the anonymity, utility, and reliability of faculty evaluations. Our current process is as follows:

1. A final-year fellow—an “honest-broker” fellow—helps organize the annual faculty evaluation process.
2. The program director provides the honest-broker fellow with faculty evaluation worksheet (the worksheet we use can be found here ).
3. The honest-broker fellow disseminates the evaluation worksheet to all fellows.
4. Using the worksheet, each fellow scores each faculty member in the following domains:

a. Possesses expertise in field of practice (clinic)
b. Helps to develop and implement patient management plans (clinic)
c. Possesses expertise in general endocrinology (consults)
d. Helps to develop and implement patient management plans (consults)
e. Applies basic and clinical sciences relevant to patient care
f. Allows you to independently formulate your own clinical opinion and allows for academic discussions
g. Aids in your critical assessment of patients
h. Establishes and models an environment of professionalism
i. Apparent degree to which faculty member values fellows and their education.

Possible scores are as follows: NA = cannot reliably assess, 1 = poor, 2 = fair, 3 = good, 4 = excellent. Fellows may also enter free text comments as needed.

5. Each fellow submits her/his completed faculty evaluation worksheet to the honest-broker fellow.
6. The honest-broker fellow compiles and anonymizes evaluation worksheets.
7. Anonymized evaluation worksheets are submitted to the program director, who calculates summary scores—means, medians, interquartile ranges, and ranges—for each of the scored domains. (Note: the honest-broker fellow could also calculate these summary scores.)
8. Summary assessments are sent back to the honest-broker fellow, who organizes a fellow-only meeting to discuss results and address the following for each faculty member:

a. Describe areas where the attending is performing well
b. Describe areas where the attending could improve
c. Have you witnessed this faculty member mistreat any learner in any way?

9. The honest-broker fellow takes meeting notes relating to the above three questions and submits the notes to the program director in a Word document.
10. The program director compiles each faculty member’s summary data and group comments; these evaluation summaries are submitted to the faculty member and the Division Chief. (Note: two examples of evaluation summaries can be found here .)

Based on these evaluations, our fellows have indicated that our faculty provide excellent teaching and supervision, although some important suggestions for improvement have been offered in some cases.

Because our program is relatively large (e.g., 7 fellows at any given time), this process makes it difficult to attribute specific evaluations to specific fellows. If a program has fewer fellows (e.g., 2-4 fellows), then fellows may not trust the anonymity of the process, and summary data (e.g., mean, median, interquartile range) may be less reliable when observations are few. In such cases, including all data obtained over the previous 2-4 years may provide more reliable assessment scores and may help limit a faculty member’s ability to attribute specific evaluations to specific fellows.

————————–

Appendix: Excerpt from ACGME Program Requirements for Graduate Medical Education in Endocrinology, Diabetes, and Metabolism (Subspecialty of Internal Medicine) – Editorial revision: effective July 1, 2019

V.B. Faculty Evaluation

V.B.1. The program must have a process to evaluate each faculty member’s performance as it relates to the educational program at least annually. (Core)

Background and Intent: The program director is responsible for the education program and for whom delivers it. While the term faculty may be applied to physicians within a given institution for other reasons, it is applied to fellowship program faculty members only through approval by a program director. The development of the faculty improves the education, clinical, and research aspects of a program. Faculty members have a strong commitment to the fellow and desire to provide optimal education and work opportunities. Faculty members must be provided feedback on their contribution to the mission of the program. All faculty members who interact with fellows desire feedback on their education, clinical care, and research. If a faculty member does not interact with fellows, feedback is not required. With regard to the diverse operating environments and configurations, the fellowship program director may need to work with others to determine the effectiveness of the program’s faculty performance with regard to their role in the educational program. All teaching faculty members should have their educational efforts evaluated by the fellows in a confidential and anonymous manner. Other aspects for the feedback may include research or clinical productivity, review of patient outcomes, or peer review of scholarly activity. The process should reflect the local environment and identify the necessary information. The feedback from the various sources should be summarized and provided to the faculty on an annual basis by a member of the leadership team of the program.

V.B.1.a) This evaluation must include a review of the faculty member’s clinical teaching abilities, engagement with the educational program, participation in faculty development related to their skills as an educator, clinical performance, professionalism, and scholarly activities. (Core)

V.B.1.b) This evaluation must include written, confidential evaluations by the fellows. (Core)

V.B.2. Faculty members must receive feedback on their evaluations at least annually. (Core)

V.B.3. Results of the faculty educational evaluations should be incorporated into program-wide faculty development plans. (Core)

Background and Intent: The quality of the faculty’s teaching and clinical care is a determinant of the quality of the program and the quality of the fellows’ future clinical care. Therefore, the program has the responsibility to evaluate and improve the program faculty members’ teaching, scholarship, professionalism, and quality care. This section mandates annual review of the program’s faculty members for this purpose, and can be used as input into the Annual Program Evaluation.