cross-sectional·research methods, clinical trial, internal medicine, family medicine, infectious disease·PMC10546234
Accuracy and Reliability of Chatbot Responses to Physician Questions
JAMA Network Open · 35 authors, 22 centres
AI SUMMARY
FIDELITY 100%
This summary was generated by AI from a single paper. It has not been reviewed by a clinician and is not clinical advice. Verify against the source before acting on it.
This cross-sectional study found that a chatbot (ChatGPT) generated predominantly accurate information in response to 284 medical questions from 33 physicians across 17 specialties, with median scores indicating 'nearly all correct' accuracy and 'complete' completeness, though some highly erroneous answers occurred.
Full summary
422 CHARS
This cross-sectional study assessed the reliability of chatbot (ChatGPT) responses to medical queries. Thirty-three physicians from 17 specialties generated 284 questions with clear, guideline-based answers. An investigator entered each question into the chatbot. Limitations include potential rating bias, the use of questions with clear answers not representative of patient queries, and evaluation of only one AI model.
