The suddenly wildly popular application, ChatGPT (AI to answer nearly any question, with answers in paragraph form) will perform well on U.S. medical board exams.
In many situations, the ChatGPT achieved a passing grade of around 60%. ChatGPT also provided justification for its answers or rationale for which answers it knew were incorrect.
The accuracy was higher on Step 3 (clinical care) than Step 1 (basic science), possibly due to undertraining on the Step 1 material.
The article by Kung et al. is online at MedRxiv:
https://www.medrxiv.org/content/10.1101/2022.12.19.22283643v2
Note that the third of twelve authors is the machine, ChatGPT.
 |
| click to enlarge |