Staff Writer: Deileta Kamhunga
A new study shows the potential of large language models in radiation oncology education and decision-making.
A recent preprint made available on April 24th has shown that large language models such as chat generative pre-training transformer (ChatGPT)-3.5 and ChatGPT-4 have the potential to be useful tools in radiation oncology education and decision-making. The study evaluated the performance of these models on the 38th American College of Radiology radiation oncology in-training exam (TXIT).
ChatGPT-4 Outperformed Its Predecessor
The study results showed that ChatGPT-3.5 achieved a score of 63.65%, while ChatGPT-4 achieved a score of 74.57%, highlighting the advantages of the latest ChatGPT-4 model. The study also identified ChatGPT-4’s strong and weak areas in radiation oncology to some extent, demonstrating good knowledge of statistics, CNS and eye, pediatrics, biology, and physics but limitations in bone, soft tissue, and gynecology.
Not Yet Ready for Clinical Decision-Making
ChatGPT-4 demonstrated strong capabilities in diagnosing, predicting outcomes, and identifying harmful effects within clinical care paths. However, it exhibited weaknesses in areas related to brachytherapy and dosimetry, as well as in-depth inquiries regarding clinical trials. Although not yet appropriate for clinical decision-making in radiation oncology, ChatGPT-4 could potentially educate the general public and cancer patients.
May Revolutionize Radiation Oncology Treatment
The study suggests that improving ChatGPT-4 will help radiation oncologists make treatment recommendations for complicated cases based on current guidelines and data. As technology advances and more data becomes available, future versions of ChatGPT are expected to deliver even better performance in all fields, including radiation oncology.
Enhances Performance Through Human Feedback Learning
Furthermore, the study suggests that ChatGPT has the potential to continuously learn and improve through feedback via reinforcement learning with human feedback. ChatGPT can incorporate feedback from human experts to refine its responses and enhance its performance over time.
Overall, the study highlights the potential of ChatGPT in medical education, clinical decision-making, summarizing guidelines, and guiding radiation oncologists in gray-zone cases. While there are still limitations and challenges to be addressed, the growing capabilities of ChatGPT and the potential for continuous learning and improvement make it an exciting development in the field of radiation oncology.
Source:
Huang, Y. (2023). Benchmarking ChatGPT-4 on ACR Radiation Oncology In-Training Exam (TXIT): Potentials and Challenges for AI-Assisted Medical Education and Decision Making in Radiation Oncology. https://www.semanticscholar.org/paper/Benchmarking-ChatGPT-4-on-ACR-Radiation-Oncology-in-Huang-Gomaa/76927634164d7c51514c63075779611d91709402
