Program Director’s Corner – November 2018

Milestones are due again…

Odelia Cooper, MD
Associate Professor of Medicine, Division of Endocrinology, Diabetes, and Metabolism
Program Director, Endocrine Fellowship Program
Cedars-Sinai Medical Center
odelia.cooper@cshs.org

The email comes to you alerting you that “Your Endocrinology, diabetes, and metabolism – Internal medicine program has been scheduled to complete the Milestone Evaluations for the residents/fellows within your program.” Yes—the email comes with 2.5 months’ notice, but it often goes on the back burner until December rolls around. Truth be told, with a system in place, the milestone process can go smoothly and efficiently. I would like to share the process we have developed for our program at Cedars-Sinai Medical Center. Our goal is to keep it focused and efficient while providing comprehensive, specific, meaningful feedback to our fellows.

1. Use milestone appropriate evaluations: We changed our evaluation forms to those that APDEM developed to match the milestones, such as the outpatient endocrine evaluation form, insulin pump forms, etc (accessible on APDEM Member Connect* under the “Resources” tab). I include evaluations for each conference that the fellows present at as well as research evaluations (sometimes sent through emails as PDFs to complete). Further, any feedback that faculty, allied health professionals, or patients provide are sent to me directly, and I copy them into a Word document with the dates so that I can track the unsolicited feedback throughout the year.
2. Set up the Clinical Competency Committee meeting: I invite all faculty from our division who have interacted clinically with our fellows.
3. Gather self-assessments: Have fellows complete the online self-assessment forms prior to the CCC meeting.
4. Collect staff and patient evaluations: Ask the clinic staff to collect patient evaluations on each fellow from their continuity panels. I aim to obtain at least 5 patient evaluations per fellow, but more is better. Circulate evaluations to nurses, clinical diabetes educators, and patient service representatives. This all drives into assessing their competency in professionalism.
5. Assess procedural skills: Ensure that the evaluations for thyroid ultrasounds, fine needle aspirations, DXAs, pumps, and CGMs are completed by those who have supervised the fellows the most in these areas.
6. Gather peer feedback: I use the evaluations by our medicine residents on the fellows. I mapped the relevant questions to match the APDEM ones.
7. Distribute research evaluations: I send the APDEM form for Publications, Research, and QI projects to the fellows’ mentors. They are asked to write specific comments on how the fellows are progressing.
8. Calculate preliminary milestone scores: I created a spreadsheet which automatically calculates preliminary milestone scores. I used the matrix that APDEM developed that maps each evaluation question to each of the 24 milestones. I download and calculate the mean scores for each question on each evaluation on the fellows. I input that score into the question on the spreadsheet in each milestone category. For example, on the outpatient endocrine evaluation, I take the mean score from all question 1 scores on this form and input the score for Milestone Patient Care 1. Patient Care 1 milestone is comprised of faculty Q1 and Q2. So, I put in both mean scores for Q1 and Q2 and get a calculated mean score for PC1. For PC2– that is comprised of faculty questions 4, 5,7,8, 11 and AHP question 4, peer question 4. So, I insert all the mean scores for those questions and get a calculated mean for PC2. And so on…

You now have a starting point for milestone scores to review with your CCC.

9. Prepare for the CCC meeting: I draft a written summary of each fellows’ performance. The purpose is two-fold: one is to provide the CCC with background to review prior to the meeting and to discuss at the meeting. The second is that it provides me the written documentation which I will review with each fellow at our semi-annual evaluation meeting and which will be placed into their files. I divide up the document as follows: Patient care (resident, attending, AHP, and patient feedback); Procedures (DXA, thyroid, pumps, CGMs); Research; Publications; Awards; Teaching (i.e. conference presentations); Self-assessments; ITE scores; Career plans; 6 month action plan (added post CCC meeting); fellow feedback on program (added after individual meeting with fellows).  I write in average scores from the evaluations and the comments from both evaluations and emails. A few days before the meeting, I send out to the CCC the preliminary calculated milestone scores, the comments document on each fellow, and my slides on what each milestone means (from ACGME) so that the faculty are reminded of what is expected for each level on the milestone.
10. Run the CCC meeting: I start the meeting by reviewing the comments document for each fellow. In this way, we have the context of what each fellow has accomplished, where he is progressing or struggling. We then go to the actual milestones and decide if the calculated score is accurate or should be moved up or down after the discussion. We then summarize what specific feedback to provide each fellow and what areas to work on.
11. Meet with the fellows: I meet with each fellow individually, review their progress, and how they see their own performance. I explain and present their milestone scores, the CCC recommendations on what to work on, and solicit feedback on our program.
12. Upload milestone scores in Web/Ads.

And you are now done for the next 6 months…but again, this is a continuous process. Having timely, specific, real time evaluations and feedback is truly key to this whole process.

If you wish to contact me for any of my templates, feel free to email me anytime.

Here are is a link for additional background on the role of the CCC and milestones:

https://www.acgme.org/What-We-Do/Accreditation/Milestones/Resources

Best of luck in your milestones.

*If you have trouble accessing the member connect site, please email APDEM staff at apdem@endocrine.org for assistance.