Leadership Corner – March 2023

March 7, 2023

A Follow-Up Commentary on the Endocrinology Subspecialty Exam ABIM Pass Rates – The Sobering News Continues for the 2022 Exam

Greg Hong, MD, PhD
Associate Professor
Program Director, Endocrinology & Metabolism Fellowship
Pituitary Program, Division of Endocrinology, Department of Medicine
University of Virginia Health System

One year later after my initial commentary on the surprising decline in our Endocrinology ABIM pass rates (https://www.apdem.org/program-director-corner/a-commentary-on-the-2021-endocrinology-subspecialty-exam-abim-pass-rates/#:~:text=Of%20note%2C%20the%20pass%20rate,nadir%20of%2074%25%20for%202021) the Endocrinology Program Director (PD) community finds ourselves in a similarly grim situation – the pass rate for 1st time test takers in 2022 remains a discouraging 74% (same as 2021), which represents the lowest pass rate of any internal medicine subspecialty.  In comparison, the average Endocrinology pass rate for the 3 year period from 2018 through 2020 was 88%.  This decline in pass rates over the past 2 years (an almost 15% drop in absolute pass rates) is particularly noteworthy as it is limited to Endocrinology and was not seen to a similar degree in other internal medicine subspecialties.  Certainly it can be noted that other specialties have seen declines in their pass rates over the past 2 years, but none of them have fallen as precipitously as Endocrinology.

Over the past year, the ABIM, with participation from APDEM, has begun analysis of potential reasons for this surprising decline.  Of note, the data obtained so far is specific to the 2021 exam, as we do not have detailed analysis of the most recent 2022 exam available yet.  However I expect that whatever factors contributed to poor 2021 performance are likely still contributing to the poor performance in 2022. A summary of the findings to date, which were presented at the annual APDEM meeting in May 2022, are presented below, with my commentary also included. 

1. Per the ABIM, there have been no changes to the ABIM board exam content or format since 2018 which would explain the marked drop in pass rates.  Of note, candidate perception of the “fairness” of the ABIM board exam has steadily fallen between 2019 to 2021.  While it is certainly tempting to blame the exam itself (especially in light of the candidates’ perception), there does not seem to be any hard data suggesting that the exam itself is the reason for the sudden decline in pass rates.  It is my understanding that analysis of the exam itself continues to be an ongoing endeavor by the ABIM.
2. A greater % than usual of 2020 graduates deferred taking their exam until the 2021 period.  Historically, those who delay taking their board exam perform worse than those who do not.  Hence it could be feasible that lower pass rates were a reflection of an unusual pool of test takers with a higher % of historically poorly performing candidates.  However 2 key findings weaken this possibility: (1) All subspecialties, not just Endocrinology, had a similar make up of test takers (e.g. 2020 graduate deferment was seen across the board and not just limited to Endocrinology) and (2) the 2020 graduates actually outperformed the 2021 graduates – and surprisingly, this better-than-expected performance of the 2020 graduates (e.g. those who had deferred for a year) was limited only to Endocrinology.
3. 2021 graduates performed significantly lower in certain content areas (adrenal, pituitary, female & male reproduction) when compared to 2020 & 2019 graduates.  This suggests that targeted review in these specific areas may be of value to PDs and fellows across the nation.
4. Between 2019 to 2021, in-person review course attendance steadily fell while at home review steadily increased.  While it is unclear whether this is truly playing a role in declining pass rates, it raises the question of whether in-person course attendance is worth funding, particularly for fellows of concern.  Preliminary studies of education in general indicate that in-person learning is often superior to virtual learning (as an example, see Gross et. al, Am J of Distance Education, 37(1):66-77).  This also raises the possibility that the current resources most commonly used for at home review (e.g. ESAP, Endocrine Board Review – both of which come out with new editions every 1-2 years) may not adequately reflect the current ABIM exam content.

At this time we are left without clear answers on the cause of the pass rate decline.  Of significant concern are numerous anecdotal reports from other PDs of excellent fellows failing the ABIM exam (GH: personal communications), suggesting that ABIM exam results do not correlate with our (PD) perception of clinical excellence.  In my opinion, further discussion of the following questions would be of substantial benefit to both the PD community and the Endocrinology community as a whole:

1. How can we identify those in danger of failing the ABIM exam early on?  Previous data indicated that ITE performance correlated with ABIM performance (Horton WB, et al, JES 2021).  Does this relationship still hold in the new environment where the pass rate is now only 74%?  Would there be value in keeping the ITE question pool relatively static (e.g. not using an entirely new question pool every year) such that we could predict future ABIM performance based on past analysis of performance on a static ITE question pool?  What does the current 2022 data say about a correlation between ABIM Internal Medicine Exam performance and Endocrinology Subspecialty Exam performance, and is this similar to years past?
2. Is there a current disconnect between everyday clinical practice/training and what is tested on the ABIM exam?  Should the ABIM content areas (or questions) be adapted to better reflect the current knowledge needed for competency in modern clinical practice situations?  Or do we need to adjust our current educational model (including re-examining the most commonly used board preparation materials) to ensure that we are adequately teaching the ABIM specified content areas (https://www.abim.org/Media/wxbjt5o3/endocrinology-diabetes-metabolism.pdf).

Unfortunately we are left in an unenvious place – faced with a serious problem without knowing the root cause(s), thereby making formulation of specific solutions difficult.  Further investigation into the root cause(s) of the pass rate decline is urgently needed, and APDEM will partner with the ABIM and Endocrine Society to address this critical issue moving forward.  The APDEM council also welcomes any ideas and suggestions on additional steps for APDEM to take to better support those responsible for the future of endocrinology – our PDs and the trainees they educate. 

Please feel free to join the discussion on the APDEM facebook community, or to contact me directly at gkh7z@virginia.edu.

Dr. Greg Hong is a member of APDEM’s Council, and also serves on the Membership Recruitment Committee and is co-chair of the APDEM Fellow Recruitment Subcommittee. 

APDEM Facebook Group – If you are interested in joining the APDEM Facebook group, please search for ‘Association of Program Directors in Endocrinology, Diabetes and Metabolism’ in Facebook. You will be asked a series of questions, and then admitted once your membership in APDEM has been confirmed. If you have any questions about this group, please email David Lieb, Chair, Newsletter Committee, at liebdc@evms.edu.