Program Director’s Corner – February 2020
ACGME Endocrine Milestones 2.0
ACGME’s Diabetes, Endocrinology and Metabolism Milestones 2.0 Working Group
As program directors we don’t always celebrate the word “milestone.” We may see the milestones intended to evaluate fellows’ progress as confusing, time-consuming and challenging to get through during our biannual clinical competency committee (CCC) meetings.
After the core competencies were developed and released in 1999 by the Accreditation Council for Graduate Medical Education (ACGME), it became clear that a process was needed to monitor each individual learner’s progress throughout training. We realized that training activities are complex and often require the simultaneous application of multiple competencies. Furthermore, there was a need to help residency and fellowship programs and their trainees understand how each core competency related to the specific training activities of their specialty. The milestone system that was developed provided narrative descriptors for each of the core competencies along a developmental continuum. Each milestone would match to one or more of the competencies and would incorporate content specific to that specialty. The ACGME released the first educational milestones for all accredited residency and fellowship programs in 2013 and 2014. It was determined that all of the internal medicine subspecialties, including endocrinology, would be guided by a single set of 23 milestones.
There were positive measurable outcomes from implementation of the milestones. Some internal medicine programs found that the milestones helped faculty provide better feedback to trainees. Others noted that fellows were less likely to receive only top scores on their evaluations – leading to more discriminative scores, thus helping faculty to determine which trainees might need remediation and in which areas.
However, Since the milestones were common across all internal medicine specialties, they were purposefully authored to be vague and malleable. Specialties proceeded to use different language to describe the same milestones, increasing ambiguity and misunderstanding. The task intended to be assessed by each individual milestone became increasingly less clear and it became apparent that what was an important milestone for an endocrinology fellow was not necessarily the same as for an Internal medicine residency trainee in a different subspecialty. Furthermore, for program directors it remains a challenge to discuss all 23 milestones with a fellow while finding time to focus on both positive and constructive feedback. It can be tempting to spend very little to no time on milestones that seem vague in their intent. Also, the use of negative and positive language in each category can make it more difficult to assign fellows into specific categories.
Realizing that the current milestones needed modification, the ACGME convened subspecialty-specific working groups to consider how milestones could be revised to enhance utility, relevance, and user-friendliness. The main charge of these working groups of content experts are 1) to reduce milestone vagueness and complexity, and 2) to develop milestones that are specialty or subspecialty specific. To help reduce redundancy, each specialty and subspecialty will have 11 “harmonized” milestones. While each specialty-/subspecialty-specific working group can tweak these harmonized milestones, each harmonized milestone must be included in each specialty specific set of milestones.
The working groups are intentionally diverse. Some of the current working group members participated in the previous milestone development process, though most are new. Starting in July 2019, the endocrine specialty specific group was to meet a total of three times over the academic year to develop the updated endocrine-specific milestones. The milestones developed will be available for public comment before implementation.
These new milestones will follow the Dreyfus Developmental Model of Skill Acquisition. This model focuses on how learners acquire new skills through formal instruction and practice. “Milestones 2.0” will still match to the core competencies. However, the categories of “critical deficiencies,” “ready for unsupervised practice,” and “aspirational” will be replaced with levels 1 through 5, matched to the Dreyfus Model stages of the learner – novice, advanced beginner, competent, proficient and expert. In addition, there will be a supplemental guide (similar to the FAQs for program requirements) with examples for each milestone, helping to clarify any questions that CCCs may have about each specific milestone and the intent of the working group.
Once this process is complete, we will have a more user-friendly, fellow-friendly and faculty-friendly set of milestones. The endocrine-specific milestones will include language that makes using a 1 to 5 scale easier. The milestones were authored with the intent to assist with early identification of fellows that may require remediation and will also identify fellows that are progressing rapidly and may need different challenges to become fellow experts. We hope that the new milestones will lead to more productive CCC meetings with a focus on the individualized needs of each fellow.
In a parallel effort the Endocrinology Milestones 2.0 group has developed a series of surveys to evaluate stakeholder (fellow, program leader, and faculty) understanding and preferences in the evaluation process. We plan to use this information to help guide the development of new milestones and to help direct faculty development efforts. Endocrine fellows completed the survey at Endocrine University in November 2019 and February 2020. We plan to administer the program director and associate program director surveys at the upcoming APDEM meeting at the Endocrine Society in San Francisco as well via email. The survey questions examine and characterize the current evaluation process and request your feedback. We hope you will take advantage of this opportunity to provide your comments.
It’s an exciting time. In collaboration with the ACGME, other ongoing projects seek to evaluate the role of the clinical competency committee, review the current status of procedural and scholarship evaluation, and adequacy of time and effort allocations for educators. We will be reaching out to you for help to collect this information. We have a unique opportunity to develop evidence-based recommendations in partnership with the ACGME. We hope to enlist your help as we move forward with these efforts.
References:
ES Holmboe, K Yamazaki, L Edgar, L Conforti, N Yaghmour, RS Miller, SJ Hamstra. Reflections on the First 2 Years of Milestone Implementation. J Graduate Med Educ September 2015.
L Edgar, S Roberts, ES Holmboe. Milestones 2.0: A Step Forward. J Graduate Med Educ. June 2018.
Holmboe ES, Edgar L and Hamstra S. The Milestone Guidebook. ACGME 2016. Available at https://www.acgme.org/Portals/0/MilestonesGuidebook.pdf.