Program Director’s Corner – May 2020

Faculty expertise and fellow supervision requirements: a potential mismatch

Roberto Salvatori, MD
Program Director, Endocrine Fellowship Program
Johns Hopkins University School of Medical

The ACGME requirements for Endocrinology fellowship programs state that: “Faculty members ensure that patients receive the level of care expected from a specialist in the field. They recognize and respond to the needs of the patients, fellows, community, and institution. Faculty members provide appropriate levels of supervision to promote patient safety.”

Diabetes care has evolved over that past few years, with routine use of high technology devices such as insulin pumps, continuous glucose monitoring apparatuses, and possible connections between the two. Appropriate diabetes care requires a deep knowledge of these devices by the care taker. In centers like ours, where faculty member subspecialize in specific areas of endocrinology, attendings who do not take care of diabetes patients in their own practice (like myself) may be unable to properly supervise fellows during the care of diabetes patients who use these devices in the fellows’ continuity clinic. We have therefore made a substantial change to the fellows’ continuity clinic.

Fellows now alternate the location of their weekly continuity clinic between the general endocrine clinic (where they see non-diabetes patients) and the diabetes clinic. In the latter location they are supervised only by faculty members who are diabetes experts and have expert knowledge of diabetes technology and new drug developments. In diabetes clinic fellows have the additional benefit of having a nutritionist and a diabetes educator to whom they can easily refer patients for additional education and support. The fellows have unanimously and enthusiastically liked this change, finding that it has significantly improved both their education and patients’ safety.

Obviously, a similar approach could be considered for other areas of endocrinology (for instance supervising the care of thyroid patients by faculty members who have a specific thyroid sonogram expertise), but practical issues would prevent the application of this model to each area of endocrinology. Nevertheless, the issue of highly subspecialized faculty members—those who know their area of focus exceptionally well but may not have adequately kept up with general endocrine issues—overseeing fellows on general endocrine consults or in continuity clinic is an important one. A PD must decide if a faculty member is suitable or not for such supervision on a case by case basis, listening to the opinions of the fellows, who ultimately are the best judges of the quality of the teaching they receive.

Program Director’s Corner – April 2020

Fellow evaluation of faculty members

Christopher McCartney, MD
Program Director, Endocrine Fellowship Program
University of Virginia

[NOTE: ACGME program requirements related to Faculty Evaluation are placed in an Appendix located at the end of this entry.]

Historically, our program’s faculty members uniformly received excellent evaluations from fellows. However, early in my tenure as program director, I sensed an occasional discordance between (a) a given faculty member’s excellent written evaluations (provided by individual fellows) and (b) informal fellow comments about that faculty member. This implied that our system of faculty evaluation might be unreliable in some cases. After some exploration, I learned that fellows had general concerns about our evaluation system: some worried that it was easy for faculty members to determine which fellow wrote a particular evaluation, and some fellows were reluctant to provide negative feedback of faculty in writing (even if completely anonymous). To help address these issues, our program developed a process designed to enhance the anonymity, utility, and reliability of faculty evaluations. Our current process is as follows:

1. A final-year fellow—an “honest-broker” fellow—helps organize the annual faculty evaluation process.
2. The program director provides the honest-broker fellow with faculty evaluation worksheet (the worksheet we use can be found here ).
3. The honest-broker fellow disseminates the evaluation worksheet to all fellows.
4. Using the worksheet, each fellow scores each faculty member in the following domains:

a. Possesses expertise in field of practice (clinic)
b. Helps to develop and implement patient management plans (clinic)
c. Possesses expertise in general endocrinology (consults)
d. Helps to develop and implement patient management plans (consults)
e. Applies basic and clinical sciences relevant to patient care
f. Allows you to independently formulate your own clinical opinion and allows for academic discussions
g. Aids in your critical assessment of patients
h. Establishes and models an environment of professionalism
i. Apparent degree to which faculty member values fellows and their education.

Possible scores are as follows: NA = cannot reliably assess, 1 = poor, 2 = fair, 3 = good, 4 = excellent. Fellows may also enter free text comments as needed.

5. Each fellow submits her/his completed faculty evaluation worksheet to the honest-broker fellow.
6. The honest-broker fellow compiles and anonymizes evaluation worksheets.
7. Anonymized evaluation worksheets are submitted to the program director, who calculates summary scores—means, medians, interquartile ranges, and ranges—for each of the scored domains. (Note: the honest-broker fellow could also calculate these summary scores.)
8. Summary assessments are sent back to the honest-broker fellow, who organizes a fellow-only meeting to discuss results and address the following for each faculty member:

a. Describe areas where the attending is performing well
b. Describe areas where the attending could improve
c. Have you witnessed this faculty member mistreat any learner in any way?

9. The honest-broker fellow takes meeting notes relating to the above three questions and submits the notes to the program director in a Word document.
10. The program director compiles each faculty member’s summary data and group comments; these evaluation summaries are submitted to the faculty member and the Division Chief. (Note: two examples of evaluation summaries can be found here .)

Based on these evaluations, our fellows have indicated that our faculty provide excellent teaching and supervision, although some important suggestions for improvement have been offered in some cases.

Because our program is relatively large (e.g., 7 fellows at any given time), this process makes it difficult to attribute specific evaluations to specific fellows. If a program has fewer fellows (e.g., 2-4 fellows), then fellows may not trust the anonymity of the process, and summary data (e.g., mean, median, interquartile range) may be less reliable when observations are few. In such cases, including all data obtained over the previous 2-4 years may provide more reliable assessment scores and may help limit a faculty member’s ability to attribute specific evaluations to specific fellows.

————————–

Appendix: Excerpt from ACGME Program Requirements for Graduate Medical Education in Endocrinology, Diabetes, and Metabolism (Subspecialty of Internal Medicine) – Editorial revision: effective July 1, 2019

V.B. Faculty Evaluation

V.B.1. The program must have a process to evaluate each faculty member’s performance as it relates to the educational program at least annually. (Core)

Background and Intent: The program director is responsible for the education program and for whom delivers it. While the term faculty may be applied to physicians within a given institution for other reasons, it is applied to fellowship program faculty members only through approval by a program director. The development of the faculty improves the education, clinical, and research aspects of a program. Faculty members have a strong commitment to the fellow and desire to provide optimal education and work opportunities. Faculty members must be provided feedback on their contribution to the mission of the program. All faculty members who interact with fellows desire feedback on their education, clinical care, and research. If a faculty member does not interact with fellows, feedback is not required. With regard to the diverse operating environments and configurations, the fellowship program director may need to work with others to determine the effectiveness of the program’s faculty performance with regard to their role in the educational program. All teaching faculty members should have their educational efforts evaluated by the fellows in a confidential and anonymous manner. Other aspects for the feedback may include research or clinical productivity, review of patient outcomes, or peer review of scholarly activity. The process should reflect the local environment and identify the necessary information. The feedback from the various sources should be summarized and provided to the faculty on an annual basis by a member of the leadership team of the program.

V.B.1.a) This evaluation must include a review of the faculty member’s clinical teaching abilities, engagement with the educational program, participation in faculty development related to their skills as an educator, clinical performance, professionalism, and scholarly activities. (Core)

V.B.1.b) This evaluation must include written, confidential evaluations by the fellows. (Core)

V.B.2. Faculty members must receive feedback on their evaluations at least annually. (Core)

V.B.3. Results of the faculty educational evaluations should be incorporated into program-wide faculty development plans. (Core)

Background and Intent: The quality of the faculty’s teaching and clinical care is a determinant of the quality of the program and the quality of the fellows’ future clinical care. Therefore, the program has the responsibility to evaluate and improve the program faculty members’ teaching, scholarship, professionalism, and quality care. This section mandates annual review of the program’s faculty members for this purpose, and can be used as input into the Annual Program Evaluation.

Program Director’s Corner – February 2020

ACGME Endocrine Milestones 2.0

ACGME’s Diabetes, Endocrinology and Metabolism Milestones 2.0 Working Group

As program directors we don’t always celebrate the word “milestone.” We may see the milestones intended to evaluate fellows’ progress as confusing, time-consuming and challenging to get through during our biannual clinical competency committee (CCC) meetings.

After the core competencies were developed and released in 1999 by the Accreditation Council for Graduate Medical Education (ACGME), it became clear that a process was needed to monitor each individual learner’s progress throughout training. We realized that training activities are complex and often require the simultaneous application of multiple competencies. Furthermore, there was a need to help residency and fellowship programs and their trainees understand how each core competency related to the specific training activities of their specialty. The milestone system that was developed provided narrative descriptors for each of the core competencies along a developmental continuum. Each milestone would match to one or more of the competencies and would incorporate content specific to that specialty. The ACGME released the first educational milestones for all accredited residency and fellowship programs in 2013 and 2014. It was determined that all of the internal medicine subspecialties, including endocrinology, would be guided by a single set of 23 milestones.

There were positive measurable outcomes from implementation of the milestones. Some internal medicine programs found that the milestones helped faculty provide better feedback to trainees. Others noted that fellows were less likely to receive only top scores on their evaluations – leading to more discriminative scores, thus helping faculty to determine which trainees might need remediation and in which areas.

However, Since the milestones were common across all internal medicine specialties, they were purposefully authored to be vague and malleable. Specialties proceeded to use different language to describe the same milestones, increasing ambiguity and misunderstanding. The task intended to be assessed by each individual milestone became increasingly less clear and it became apparent that what was an important milestone for an endocrinology fellow was not necessarily the same as for an Internal medicine residency trainee in a different subspecialty. Furthermore, for program directors it remains a challenge to discuss all 23 milestones with a fellow while finding time to focus on both positive and constructive feedback. It can be tempting to spend very little to no time on milestones that seem vague in their intent. Also, the use of negative and positive language in each category can make it more difficult to assign fellows into specific categories.

Realizing that the current milestones needed modification, the ACGME convened subspecialty-specific working groups to consider how milestones could be revised to enhance utility, relevance, and user-friendliness. The main charge of these working groups of content experts are 1) to reduce milestone vagueness and complexity, and 2) to develop milestones that are specialty or subspecialty specific. To help reduce redundancy, each specialty and subspecialty will have 11 “harmonized” milestones. While each specialty-/subspecialty-specific working group can tweak these harmonized milestones, each harmonized milestone must be included in each specialty specific set of milestones.

The working groups are intentionally diverse. Some of the current working group members participated in the previous milestone development process, though most are new. Starting in July 2019, the endocrine specialty specific group was to meet a total of three times over the academic year to develop the updated endocrine-specific milestones. The milestones developed will be available for public comment before implementation.

These new milestones will follow the Dreyfus Developmental Model of Skill Acquisition. This model focuses on how learners acquire new skills through formal instruction and practice. “Milestones 2.0” will still match to the core competencies. However, the categories of “critical deficiencies,” “ready for unsupervised practice,” and “aspirational” will be replaced with levels 1 through 5, matched to the Dreyfus Model stages of the learner – novice, advanced beginner, competent, proficient and expert. In addition, there will be a supplemental guide (similar to the FAQs for program requirements) with examples for each milestone, helping to clarify any questions that CCCs may have about each specific milestone and the intent of the working group.

Once this process is complete, we will have a more user-friendly, fellow-friendly and faculty-friendly set of milestones. The endocrine-specific milestones will include language that makes using a 1 to 5 scale easier. The milestones were authored with the intent to assist with early identification of fellows that may require remediation and will also identify fellows that are progressing rapidly and may need different challenges to become fellow experts. We hope that the new milestones will lead to more productive CCC meetings with a focus on the individualized needs of each fellow.

In a parallel effort the Endocrinology Milestones 2.0 group has developed a series of surveys to evaluate stakeholder (fellow, program leader, and faculty) understanding and preferences in the evaluation process. We plan to use this information to help guide the development of new milestones and to help direct faculty development efforts. Endocrine fellows completed the survey at Endocrine University in November 2019 and February 2020. We plan to administer the program director and associate program director surveys at the upcoming APDEM meeting at the Endocrine Society in San Francisco as well via email. The survey questions examine and characterize the current evaluation process and request your feedback. We hope you will take advantage of this opportunity to provide your comments.

It’s an exciting time. In collaboration with the ACGME, other ongoing projects seek to evaluate the role of the clinical competency committee, review the current status of procedural and scholarship evaluation, and adequacy of time and effort allocations for educators. We will be reaching out to you for help to collect this information. We have a unique opportunity to develop evidence-based recommendations in partnership with the ACGME. We hope to enlist your help as we move forward with these efforts.

References:

ES Holmboe, K Yamazaki, L Edgar, L Conforti, N Yaghmour, RS Miller, SJ Hamstra. Reflections on the First 2 Years of Milestone Implementation. J Graduate Med Educ September 2015.

L Edgar, S Roberts, ES Holmboe. Milestones 2.0: A Step Forward. J Graduate Med Educ. June 2018.

Holmboe ES, Edgar L and Hamstra S. The Milestone Guidebook. ACGME 2016. Available at https://www.acgme.org/Portals/0/MilestonesGuidebook.pdf.

Program Director’s Corner – January 2020

A sunsetting program director’s reflections on program leadership

Christopher McCartney, MD
Program Director, Endocrine Fellowship Program
University of Virginia

I’ve been a program director since 2011, and I’ve valued it a sacred privilege. In early 2019, however, I decided to step down from the program director position effective July 1, 2020.[1] Since this decision, I’ve reflected a fair bit on my tenure as a program director—the goals I did and didn’t achieve, the struggles I encountered, and the lessons I learned. In this Program Director’s Corner, I’ll share my reflections on two particular struggles and what they taught me about program leadership in general.

A goal achieved, but not universally lauded

Our fellowship program was strong in 2011, but I had some changes in mind. My sense was that the distribution of mandatory clinics wasn’t ideal, so one of my first goals was to reorganize these assignments. For example, fellows spent a lot of time in a very prominent, highly-subspecialized clinic.[2] However, some fellows had criticized the amount of time spent in this clinic. My sense was that an excellent training experience did not mandate so much time in this clinic, and I felt that the amount of time fellows spent in this clinic had important opportunity costs. I therefore worked with the Program Evaluation Committee to establish a putatively more ideal mix of required rotations (clinical exposure).[3] However, a small handful of more-senior faculty members—the most vocal of whom worked in the highly-specialized clinic described above—criticized the new rotation structure, at least partly based on a different conception of what’s best for the fellows.

A note about me: I often find such criticism unnerving. I’m a card-carrying introvert, I sometimes doubt myself, and my natural inclination is to avoid confrontation. Moreover, my division was (is) rather hierarchical, I was a relatively early-career faculty member at the time, and my more-senior faculty colleagues had all trained me as a fellow. In this context, I second-guessed my leadership decision. However, several considerations provided me with the courage to stand my ground against these vocal critics. First, the pros and cons of competing rotation-assignment models had been very carefully considered, and I was convinced that the rationale for the new rotation structure was sound. Secondly, the plan had been vetted by, and fully endorsed by, the Program Evaluation Committee. Thirdly, and I think most importantly, the program director is ultimately responsible for furthering the fellows’ bests interests: I remained convinced that the new plan would better serve the fellows, and I knew I had to stay true to that conviction.[4]

A goal unachieved and feelings of failure

For at least 20 years my division has completely depended on fellows to cover our general endocrine consult service.[5] Approximately half-way through my tenure as program director, the lack of reliable external funding for research training led to a divisional incentive to reduce the number of fellows. In this context, it became clear that our total reliance on fellow consult coverage was a potential problem—if any fellow were to require a prolonged absence, we would be at risk for a substantial consult-coverage shortfall. These fears were first realized in the 2017-2018 academic year: one of our fellows elected to stop fellowship after one year, and another fellow would need maternity leave, rendering an 18-week shortfall in fellow consult coverage. If I were to require the remaining five fellows to fully cover the shortfall, I would be asking them to do substantially more consult time than I had judged necessary for their training. And while additional time on the consult service could provide some additional educational benefit, the work involved is somewhat arduous, and I wasn’t convinced that the incremental educational yield justified the extra work.[6] Another potential coverage gap arose in 2019: given a programmatic transition in the Endocrine University program, it was possible that all of our fellows would be allowed to attend Endocrine University in February 2020. Thus, I didn’t schedule any fellow to be on consults that particular week.

In both cases, I asked divisional leadership to engage the faculty in a discussion about how faculty could cover the fellow coverage shortfalls.[7] Such discussions never happened, though, in part related to divisional leadership’s (valid) concern about how to assign faculty coverage in an equitable way.[8] In addition to this reluctance on the part of divisional leadership, the fellows and I were concerned that, because of the long-standing expectations of faculty members, requiring faculty members to cover consults without fellow help could generate fellow-directed resentment on the part of some faculty members. In the end, we addressed both of the above situations by allowing fellows to cover consults as a moonlighting opportunity: this was a fully-voluntary activity,[9] and the division provided generous supplemental pay for the fellows’ efforts. This arrangement was a clear concession on my part, and I can’t quite shake the feeling that I had somehow failed in my duty as a program director. Perhaps I shouldn’t have given any ground? Perhaps I shouldn’t have relented to the prevailing divisional expectation that fellows will always cover consults fully? I console myself with the thought that the moonlighting arrangement may have been the best I could have done at the time. I also believed that the arrangement represented a step in the right direction, and my hope is that it will pave the way for future improvements in this regard.

————-

We program directors are leaders, and being a leader can be challenging. I believe that the above-described challenges highlight some important concepts, which I summarize below.

1. Fellowship programs function within endocrine divisions, and fellowship training requires the educational efforts of individual faculty members. However, the fellows’ best interests may at times conflict with the most-immediate interests of individual faculty members[10] or even divisional units as a whole. Thus, we as program directors can be caught in the middle—between the needs/desires of our primary constituency (fellows) and the needs/desires of our colleagues and/or supervisors.
2. Wisdom includes knowing how hard to push (sometimes, but not always, you have to be a persistent gadfly), and it includes knowing how hard to push back (sometimes, but not always, you have to be stubborn in the face of external pressure). At the same time, we should allow room for, and do our best to learn from, good-faith disagreements about what is in the best interests of fellows.
3. It is important to have an explicit—and ideally a very-compelling—rationale for any program-related decision that could conflict with the most-immediate interests of faculty colleagues and/or the division as a whole. Moreover, the importance of building general consensus (e.g., via the Program Evaluation Committee) cannot be overemphasized. At the end of the day, a program director’s primary responsibility is to advance what she or he believes is in the fellows’ best interests, even when it requires respectfully-adversarial interactions with those higher in the divisional, or perhaps even institutional, hierarchy.

[1] My rationale was two-fold. Most importantly, my program directorship had transitioned from a building mode to a maintenance mode; the program would benefit from “new blood”—someone who would bring new ideas and a renewed energy. Secondly, my division had several more-junior faculty members who could be outstanding fellowship program directors, and at some point it’s important to make way for the next generation of leaders.

[2] This clinic indeed provided an excellent educational experience. The clinic also generated substantial resources and brought substantial prestige to the University; in my admittedly cynical view, the latter considerations had influenced the amount of time fellows spent in this clinic.

[3] We developed a rotation plan we believed would optimally achieve two primary goals: (1) a balanced mix of exposure to the different areas of endocrinology, and (2) a diverse exposure to our different faculty members within those different areas of endocrinology.

[4] I acknowledge here that there will be certain times when your convictions should change on the basis of alternate views that you hadn’t adequately considered.

[5] The fellows have appropriate faculty oversight, of course, but consult fellows do the bulk of the work.

[6] The most common critique I’ve received about my posture goes something like this: the fellows aren’t overworked, and all clinical activity promises an educational benefit. But my thoughts on this issue tend to be dominated by the incentives at play: instead of assigning the work to meet educational requirements, we would have asked the fellows to do the work primarily—if not exclusively—to meet divisional work-capacity needs. I want to note here that our fellows are not reluctant to work. However, they want their work to be tangibly and legitimately justified vis-à-vis their education. I believe their desires in this regard are fully justified.

[8] For example, while the assignments could be determined by lottery, some of our faculty could not adequately function on consults (without fellow help) given ignorance of the mechanics of modern inpatient consultation; as a result, the junior faculty would be unfairly targeted for consult coverage.

[9] I suspect that the fellows still felt some internal pressure to volunteer.

[10] This may even include the program director’s immediate boss—the division chief/director.

Program Director’s Corner – December 2019

Caring for our Coordinators

David Lieb, MD
Program Director, Endocrine Fellowship Program
Eastern Virginia Medical School

There was an interesting study published in the Journal of Graduate Medical Education last issue regarding program coordinators training.

Shah, Naffouje and Ejaz published the results of an important national survey of program coordinators (PC) in graduate medical education (J Grad Med Educ, October 2019). They were interested in surveying PCs to identify any predictors of better performance of their programs. They developed a 58-question survey that was sent out to over 1500 PCs nationwide. 712 PCs responded (47% of those that received survey). Both university-based (59%) and community-based programs were represented. Only 17% of those surveyed received specific coordinator training through the GME office at their institution, with another 15% receiving only peer training. An additional 9% of PCs reported having received both GME and peer training.

I was surprised that only 40% of those PCs survey reported any form of orientation. Only 51% of those receiving GME training found it helpful, compared to almost all of those receiving peer training. All who received training, rather it be from GME directly, or from peers, reported better performance, as defined by lower rates of delayed resident/fellow starts, higher graduation rates, higher rates of resident duty hour compliance, and higher rates of ‘readiness’ for internal GME reviews and GME visits. Having had prior administrative experience, and longer duration as a PC were associated with improved PC performance.

I would have been interested in measures and predictors of PC happiness and burnout. Such data, though limited, does exists for program directors. A survey for family medicine program directors from 2018 (Fam Med 2018 Feb;50(2):106-112) found that 27% of family medicine residency program directors surveyed were ‘highly emotionally exhausted’ – and that this correlated with lack of personal time, an unhealthy work-life balance, and the ‘inability to stop and think about work’. Financial stress also correlated with high rates of emotional exhaustion and feelings of depersonalization. I would imagine that our program coordinators suffer from similar feelings, and it would be important for us to know about the prevalence of such feelings in order to determine methods for improvement.

At my institution, Eastern Virginia Medical School, we have a Graduate Medical Education Program Administrators Committee. This committee meets regularly and is made up of all of our residency and fellowship program coordinators. They discuss all issues important and pertinent to the responsibilities of program coordinators, and help raise awareness of important issues that need to be addressed. This includes significant concerns such as professional titles and salary. Through this committee and our Graduate Medical Education committee, we nominate and award a ‘Program Coordinator of the Year’, and provide a significant amount of peer mentoring and orientation. Our regional institutions also host an annual Regional Program Coordinators Workshop, that includes presentations from DIOs and PCs on topics including professionalism and communication, ERAS and ACGME milestone updates, and CCC/PEC committee pearls. These meetings are well-attended and appreciated by both new and more experienced PCs.

I believe that such committee and meeting involvement provides program coordinators with a much needed source for learning, social interaction and resume building. I certainly feel that way about the PD committees in which I take part. They give me a stronger sense of personal identification as a PD, and I imagine it’s the same for PCs.

There are multiple ACGME resources available for PCs. Some resources for program coordinators include:

Journal of Graduate Medical Education Coordinator Page:
https://www.jgme.org/page/coordinatorjgme?mobileUi=0

ACGME Website Resources
https://www.acgme.org/Program-Directors-and-Coordinators/Welcome

ACGME Coordinator Advisory Group
https://www.acgme.org/Program-Directors-and-Coordinators/Welcome/ACGME-Coordinator-Advisory-Group

I would love to have some of our endocrinology Program Coordinators comment – what do you all think we can do to improve your opportunities and experiences? What do your institutions do for PC orientation? In the past we had a regular PC Corner similar to our PD Corner in the newsletter – if you are a PC that would like to write a post about your experiences, please let me know at liebdc@evms.edu. I’d love to hear from you and I know that others would, too!

Happy Holidays to everyone!

David

Program Director’s Corner – November 2019

Use of Standardized Patients to Teach Transgender Care in Endocrinology Fellowship Programs

Mary Stevenson, MD
Assistant Professor of Medicine
Emory University School of Medicine

One area of growing interest and attention in endocrinology fellowship programs is the necessity of providing training in the unique health care needs of transgender and gender non-conforming individuals. Endocrinology fellows have historically reported insufficient education and guidance in this field. A 2017 survey of second year endocrinology fellows found that only 58.9% had been provided dedicated training in transgender care1. Moreover, of those fellows who had reported instruction in the topic, 40% had received less than two hours of content per year1. To address these deficiencies, endocrinology programs have been implementing teaching strategies that include lectures, small group discussions, online modules/webinars, elective rotations, and direct patient care time2.

The endocrinology fellowship training program at Emory University has been using the Objective Structured Clinical Examination, or OSCE, for the past three years as a teaching tool to improve fellows’ comfort and knowledge in delivering transgender care. Case vignettes of both trans-feminine and trans-masculine standardized patients have been presented to fellows to assess components within the four out of six ACGME core competencies of medical knowledge, patient care, professionalism, and interpersonal and communication skills3. The standardized patient and faculty members provide feedback to the fellows immediately after the simulated encounter. Additionally, fellows have completed pre- and post-encounter surveys about their comfort and medical knowledge in providing transgender care that can provide valuable information to guide further educational opportunities. We believe the OSCE can be a valuable tool to aid in identifying areas of strengths as well as areas for improvement and growth for endocrinology fellows.

A detailed description of the methods and results of one trans-feminine standardized patient case conducted at Emory University during the 2016-2017 academic year will soon be published in the Journal of the Endocrine Society4. We would love to collaborate with other endocrinology fellowship programs interested in implementing OSCE cases to improve endocrinologists’ care for transgender and gender non-conforming patients. Please contact Mary Stevenson, MD at movsteve@emory.edu or Vin Tangpricha, MD, PhD at vtangpr@emory.edu with any questions or for further information.

1. Davidge-Pitts CJ, Nippoldt TB, Natt N. ENDOCRINOLOGY FELLOWS’ PERCEPTION OF THEIR CONFIDENCE AND SKILL LEVEL IN PROVIDING TRANSGENDER HEALTHCARE. Endocr Pract Off J Am Coll Endocrinol Am Assoc Clin Endocrinol. 2018;24(12):1038-1042. doi:10.4158/EP-2018-0307
2. Davidge-Pitts C, Nippoldt TB, Danoff A, Radziejewski L, Natt N. Transgender Health in Endocrinology: Current Status of Endocrinology Fellowship Programs and Practicing Clinicians. J Clin Endocrinol Metab. 2017;102(4):1286-1290. doi:10.1210/jc.2016-3007
3. Swing SR. The ACGME outcome project: Retrospective and prospective. Med Teach. 2007;29(7):648-654. doi:10.1080/01421590701392903
4. Stevenson MO, Sineath RC, Haw JC, Tangpricha V. Use of Standardized Patients in Endocrinology Fellowship Programs to Teach Competent Transgender Care. Journal of the Endocrine Society. Accepted, in press.

Program Director’s Corner – October 2019

Feedback to the Stars

David Lieb, MD
Program Director, Endocrine Fellowship Program
Eastern Virginia Medical School

Giving feedback isn’t always easy. You have to make time to give it, and even when you do, learners may not realize that you are providing it. This issue is so pervasive, that many educators recommend you start any feedback session by saying “I am giving you feedback”! That said, giving feedback to someone who is doing well can be particularly challenging. At least when someone requires specific remediation in their medical knowledge or professionalism, you know what needs to be discussed and can develop a plan for helping the learner change their behavior. But what if someone is doing a great job? Often in our time-crunched world the impulse is to say “I am about to give you feedback – you are doing a great job!” and move on to your next task, knowing that you have given some great feedback.

This is something I’ve found to be difficult. It’s hard to give productive feedback when you aren’t clear what that feedback should be. In my search to do a better job with this I came across a great article by editor Amy Gallo at the Harvard Business Review (which by the way is a fantastic source for helpful managerial-type and leadership-focused articles – our library has it – yours may too). I’ll provide a link to the article at the end of this post.

Gallo starts by mentioning what I’ve already said – that determining the needs for improvement for a top performer can be difficult. She adds that these individuals may not be accustomed to getting feedback, and that this can make feedback sessions more challenging. They may think that they are perfect.  However, she notes that you are doing your learner a disserve by not helping them to grow, and everyone has room to grow. Top performers may have good results (efficient presentations, great medical knowledge) – but it’s how they get those results that is important. Is it at the expensive of good mental health and proper work/life balance? Or at the expense of good relationships with the other fellows? Gallo notes that the behaviors that may help a person succeed may be the same that may hold them back in a different situation.

You should start the feedback session by setting the agenda. Tell the learner you’ll be discussing their current performance, and then future goals and aspirations. You should then express gratitude for the learner’s positive performance. Don’t assume they’ve heard it all before – and certainly don’t assume it isn’t important for them to hear. Express that you value them and their work. Then talk with them about what their goals and aspirations are – and what they value. Maybe it’s a first-year fellow who is interested in a career in research, or a second-year fellow who is interested in becoming a clinical educator. Use the opportunity to determine what obstacles they face in meeting their goals. And ask them how you can help them to succeed and to meet those goals.

Gallo ends by summarizing her ‘Principles to Remember’:

Do:

• Give both positive and constructive feedback to high performers regularly.
• Identify development areas, even if there are only a few.
• Focus on the future and ask about motivations and goals.

Don’t:

• Presume a star has reached the limits of their performance.
• Leave your top performers alone.
• Assume your stars know how appreciated they are.

I hope everyone is having a successful interview season. Now go out there and give some feedback!

David

https://hbr.org/2009/12/giving-a-high-performer-produc

Program Director’s Corner – September 2019

APDEM Council Updates

For this month’s Program Director Corner, Council would like to provide a brief update to membership on some of its current initiatives and projects.

Finance Committee

APDEM Council has created a finance committee responsible for assessing APDEM’s financial wellbeing and identifying potential fiscal opportunities that will support the mission of the organization. The Finance Committee reports directly to Council and will be responsible for:

• Reviewing the annual operating budget and presenting to Council for approval
• Identifying potential sister societies and sponsors
• Exploring investment opportunities and options
• Developing business plans for proposed APDEM projects
• Recommending membership dues rates to Council

We are pleased to announce that the following members have accepted their invitations to serve on this inaugural committee for a two-year term:

• Susan Samson, Baylor College of Medicine (Chair)
• Andrew Gianoukakis, UCLA-Harbor Medical Center (Ex-Officio)
• Christopher McCartney, University of Virginia (Ex-Officio)
• Ole-Petter Hamnvik, Brigham and Women’s Hospital (Member)
• Sara Lubitz, Robert Wood Johnson Medical School (Member)
• Farah Morgan, Cooper University Health Care (Member)
• Vafa Tabatabaie, Montefiore Medical Center (Member)

APDEM Response to ACGME RC-IM Proposed Changes to Program Requirements

Thank you to the 22 program directors who completed a survey regarding the ACGME’s proposed changes to its program requirements for endocrinology.  Using these responses, APDEM Council crafted an advocacy letter to send the ACGME for consideration.  You can read APDEM Council’s comments here.

Updates from the All-In Match Oversight Task Force

As sponsors of the Adult Endocrinology Match, APDEM partners with the NRMP to monitor compliance with the All-In Match policy (https://www.apdem.org/all-in-match-apdem-statement/).  According to our agreement with the NRMP, APDEM must confirm the number of fellows who have entered (and/or will enter) each program as a first-year fellows in 2019.

We have now concluded our reporting and have passed it along to NRMP for monitoring.  Thank you to all the Program Directors and Coordinators who confirmed their fellows.  If you have any questions regarding the match or oversight task force, we encourage you to reach out to the Task Force Chair, Dr. Gianoukakis (agianouk@ucla.edu).

Call for Council Nominations Open

The call for nominations to APDEM Council is open and will remain open until close of business, October 25, 2019.  To self-nominate or to nominate another Council member, please fill out this survey.

As defined by our bylaws, the Council shall consist of seven voting Councilors, including the President and Secretary-Treasurer, notwithstanding the nonvoting Councilors, including the President-Elect and the Immediate Past President. Council members must be voting APDEM members (defined within the bylaws as a Program Director of a current APDEM organization) in good standing. Nominations for membership on the Council may be proposed by any voting member.

Volunteers Needed – Program Director Corner Articles

We encourage APDEM members to get involved with the organization by writing a Program Director Corner article.  Program Director (PD) Corners are an opportunity for program directors and associate program directors to share the resources and tools that have helped their programs and to start a discussion around education practices through the comment feature at the bottom of each article.

We are currently seeking volunteers to write articles for the October, November, December and January articles.  If you are interested, please email APDEM staff at apdem@endocrine.org.  Potential topics include, but are not limited to:

• Professional development opportunities – for junior faculty and fellows
• Interviewing/Recruiting best practices
• Engaging adult learners
• Mentor/mentee relationships

Program Director’s Corner – August 2019

A Holistic Process for Recruiting and Interviewing Endocrinology Fellows: a Mechanism to Increase Diversity

Ricardo Correa MD, Es.D and Karyne Vinales, MD
University of Arizona College of Medicine-Phoenix

The United States health care system is fraught with racial/ethnic and gender disparities.^1,2 The data in endocrinology is weak, but a few studies that compared different medical subspecialties show that our specialty in some geographical regions lacks diversity, when compared to the population of the same area. While the underlying reasons for ongoing inequity are countless, lack of diversity in the physician workforce is a significant contributing aspect.^2,3 Physicians and trainees from underrepresented minority groups are more likely to care for traditionally underserved minority patients,^4,5 yet diversity among the US healthcare workforce has not kept pace with the rapidly changing demographics of the US population.

Even though we as a physician community have known about this problem, there is a lack of research in the graduate medical education arena regarding how to increase the diversity of the trainee workforce. The AAMC has established what is called “holistic review” to tackle diversity in undergraduate medical education, and define it as “a flexible, individualized way of assessing an applicant’s capabilities by which balanced consideration is given to experiences, attributes, and academic metrics.” ^2,6

Several GME programs⁷ have been working on projects intended to increase diversity, mainly when they are located in cities with a very diverse population, including Houston, Phoenix, Los Angeles, Miami, and New York.

During this time of the year, we are receiving applications for our fellowships. We know that many of you have already a plan in place on how to choose applicants for an interview and how to conduct the interview. Also, we know that increasing diversity in small programs like many of our fellowships is not an easy task to accomplish. We want to discuss in this article some strategies that you can use during the application review and on the interview day to address diversity in your program. This is what we call a “holistic process and interview.

The recommendations that we are proposing in this article are early stage and do not have strong evidence, but we can assure all of you that we are working on a stronger process for reviewing and selecting applicants. We expect to have data to present to the entire APDEM community in the upcoming years.

First, we will start with the selection process⁸:

1. USMLE scores: We understand that many programs use the USMLE score as a filter for the selection process. We recommend that besides looking at the USMLE scores as the only entity, you use some modifiers such as: (1) prior leadership experience and core values of the candidate that align with the mission and vision of your institution (this can be found in the applicant’s personal statement); (2) commitment to underserved or underrepresented populations such as volunteer work during residency; and (3) balanced representation of the community where your program is located. A scoring method that has been useful in residencies is to add 2 points for each of the above items.⁶ Using this technique, you can decrease the bias of relying just on an exam but allowing for interviewing applicants that have other abilities that will be good for your fellowship.

2. Letter of recommendations (LORs): Depending on the number of applications that you receive, it can be challenging to read all the letters of recommendation. But, with the new Internal Medicine (IM) Program director standardized letter of recommendation, you will have a better understanding of the applicant’s achievements. We understand that at this point, not every IM Program director uses the standardize LOR and, in those cases, you will need to rely on your ability to obtain the essential data from the LOR. We recommend using the Alliance for Academic Internal Medicine (AAIM) resources on how to appraise a LOR.

Second, the interview process⁸:

3. Training of the interviewers: Faculty members involved in the fellow selection and interview process should go to an unconscious bias training session before the interview season. This training will facilitate discussion about implicit bias, microaggression, and its potential impact on recruitment. Also, this training will create a safe discussion that will also help your program. The Office of diversity and inclusion from your institution can do this training, or there are many resources in the AAMC webpage. If you need something a little bit easier please email us, and we can provide a small presentation on this topic.

4. Ask specific questions to the applicant: We recommend distributing among the interviewers’ certain questions that will address specific competencies of the applicant. Previously, there has been no standardized questionnaire for this (we are working on this during this interview season), but you can create your questions. The main topics that this questionnaire should address are: (1) the ability of the applicant to work in teams, (2) approaches to problem solving, resilience, (3) personality, and (4) capacity to adapt to changes. We recommend that at least two interviewers ask one of the topics above to decrease subjective bias.

5. Use one member of the Diversity and Inclusion (D&I) Office or Committee from your institution to be part of the interview (the officer doesn’t necessarily need to be an endocrinologist): The main focus of this faculty member will be to evaluate the applicant’s response and reaction to a standardized case about discrimination. We have created a scenario that we can provide to you. Besides having an external faculty that understand, involving D&I will make your interview a holistic process.

6. Include your fellows in the interview process: Fellows can provide valuable insight into potential future co-workers. Peer-to-peer communication has been proven to help in the hiring process in other areas (not specifically in medicine). A technique that has been proven in residency programs is that residents (the ones that were involved during that day with the applicants during breakfast, lunch or particular time) answer the following two closed-ended questions for each applicant as a consensus (one evaluation from all the fellows): (1). Do you feel that this applicant will be a perfect fit for your program? And (2) Do you have any concerns or red flags about the applicant? We are currently applying this practice with our fellows and asking for their feedback on potential candidates (as a pilot project).

While we understand that the suggestions provided above to create what we consider a holistic review process and interview have not been evaluated, our team firmly believes that we as an endocrine program director community should lead this change in our country. This is the first step, and we would welcome additional dialogue with our peers that are interested in improving the interview process. If we consistently use the same protocols, and collect results through scientific protocols, our results could support a positive change not only in endocrine field but in the entire GME community.

For more information about this or if you have any question, please contact Ricardo Correa (ricardocorrea@email.arizona.edu) or Karyne Vinales (karynevinales@email.arizona.edu)

References
1. Institute of Medicine, Committee on Understanding and Eliminating Racial and Ethnic Disparities in Health Care. Unequal Treatment: Confronting Racial and Ethnic Disparities in Health Care. Washington, DC: National Academic Press; 2003.
2. Omowunmi Aibana, Jennifer L. Swails, Renee J. Flores, LaTanya Love. Bridging the Gap: Holistic Review to Increase Diversity in Graduate Medical Education. Acad Med. 2019;94:1137–1141.
3. Agency for Healthcare Research and Quality. 2016 National Healthcare Quality and Disparities Report. Rockville, MD: Agency for Healthcare Research and Quality; July 2017 2017.https://www.ahrq.gov/sites/default/files/wysiwyg/research/findings/nhqrdr/nhqdr16/2016qdr.pdf. Accessed August 1, 2019
4. Wayne SJ, Kalishman S, Jerabek RN. Timm C, Cosgrove E. Early predictors of physicians’ practice in medically underserved communities: A 12-year follow-up study of University of New Mexico School of Medicine graduates. Acad Med. 2010;85(10 suppl):S13–S16.
5. Association of American Medical Colleges. Addressing racial disparities in health care: A targeted action plan for academic medical centers. https://members.aamc.org/eweb/upload/addressing%20racial%20disparaties. pdf. Published 2009. Accessed August 1, 2019.
6. Association of American Medical Colleges. Diversity in the physician workforce: Facts and figures 2014. http://www.aamcdiversity factsandfigures.org. Accessed August 1, 2019.
7. Java Tunson, Dowin Boatright, Stephanie Oberfoell, et al. Increasing Resident Diversity in an Emergency Medicine Residency Program: A Pilot Intervention With Three Principal Strategies. Acad Med. 2016;91:958–961.
8. University of Arizona College of Medicine-Phoenix, Diversity and Inclusion Committee. GME program. Endocrinology, diabetes and metabolism project. Personal Communication.

Program Director’s Corner – July 2019

The Faculty Evaluation Meeting

David Lieb, MD
Program Director, Endocrine Fellowship Program
Eastern Virginia Medical School

Sections II.A.4.d and II.A.4.e of the ACGME Program Requirements for Graduate Medical Education in Endocrinology, Diabetes and Metabolism reminds us that as PDs we are to ‘evaluate program faculty’, and to ‘approve’ their ‘continued participation’ in the fellowship program. What are the responsibilities of our faculty? They must ‘maintain an environment of inquiry and scholarship’ and participate in didactics (journal club, case conferences, didactics). Some need to be involved in national committees/educational organizations, have peer-reviewed funding, and/or publish original work or reviews (50% of your Key Clinical Faculty).

And as per the ACGME requirements, section V.B. – the program must evaluate faculty performance (as it relates to the education program) at least once per year. This evaluation must include the faculty member’s ‘clinical teaching abilities, commitment to the educational program, clinical knowledge, professionalism and scholarly activities’. And it must include a written confidential evaluation by the fellows.

Finally, the program must ‘monitor and track’ faculty development (Section V.C.2.b).

So….easy right? Sometimes, but not always. As program director you work with faculty at all levels of professional development – folks that have just finished fellowship – perhaps at your institution, or an outside institution with its own culture; all the way to tenured professors who have been doing it their way for decades (and may not be all that flexible even when it’s required). It helps to have a framework for your annual meeting with each of your faculty. Here are some of my thoughts for what to include in such a framework.

Faculty are busy. You may only have 30 minutes to review their responsibilities, performance for the year (fellow evaluations), scholarly activity, and faculty development needs. Remember – this is not your division chief’s annual evaluation – focus on each faculty member’s evaluations (done by the fellows), their scholarly activity (as it relates to the fellowship), their concerns and how they want to (or need to) grow as a teaching faculty. Don’t lose sight – it’s about their role as teachers.

You should be meeting with each faculty member at least once per year, more often if needed.  Be sure that prior to each meeting you have read through the faculty member’s evaluations from the fellows. They should have an opportunity to review their evaluations prior to your meeting as well.  As with any good feedback session – start by asking ‘How do you feel things are going?’. Do they have any concerns regarding the fellowship and their role in it? What is that role – are they on the CCC? The PEC? Are they responsible for a particular part of the fellows’ procedural education (CGM, U/S, Pumps)? Do they provide didactic lectures? Ensuring that both you and the faculty member understand their role and responsibilities is a good place to start your discussion.

Review their scholarly activity, and see if fellows have been involved. If not, determine if there might be a role for a fellow (or two) in a project. Alternatively, are there projects that the faculty member would like to get started, and would benefit from fellow involvement? What are their goals with respect to medical education? See how you as the program director can provide assistance. Work with your institution’s graduate medical education council, or professional development office, to determine what classes or webinars may be offered locally for faculty development. Live sessions are often recorded and can be accessed later when it might be more convenient for your faculty member to review. It’s helpful to have a list of common topics (teaching in small vs large groups, giving feedback, etc) in professional development that may help you and the faculty member determine where their needs might be.

National organizations such as the Endocrine Society and the American Association of Medical Colleges have in-person workshops that may help your faculty with their development. The Accreditation Council for Graduate Medical Education has online resources that may be useful as well. The AAMC’s MedEdPortal is a wonderful resource for online teaching and assessment resources, as is the ACGME’s Journal of Graduate Medical Education website. Faculty that are particularly interested in GME may want to go to an AAMC or ACGME national meeting, and bring back what they’ve learned to your faculty. There may be funding through your institutions GMEC for such meetings. Below I’ve listed some pertinent/helpful websites.

Occasionally there may be more significant issues regarding a faculty member’s interactions with the fellows and the fellowship program. If you have significant concerns regarding a faculty member’s ability to remain part of the fellowship program, be sure to involve your division chief and/or department chair. Your institutional graduate medical education council may also be helpful depending on the circumstances.

The faculty evaluation meeting is an opportunity to better understand your faculty members, and their needs. It’s also an opportunity to help them reach their goals within medical education. It’s an exciting part of our responsibilities as a program director.

https://www.endocrine.org/training-and-education/career-development-workshops

https://www.aamc.org/members/leadership/

https://www.acgme.org/Meetings-and-Educational-Activities/Other-Educational-Activities/Webinars

https://www.mededportal.org/

https://www.jgme.org/