A Case Study of Neurosurgical Applications of Artificial Intelligence and Chat-GPT in Patient Assessment and Treatment Recommendation

Introduction: Our purpose was to assess the predictive and analytic capabilities of ChatGPT-3.5 and 4 to determine artificial intelligence’s potential utility as an assistive clinical tool, with a particular focus on application to the field of neuro-oncology.

Methods: ChatGPT-3.5 and 4 were provided with the same twenty example neuro-oncology cases and asked to provide a diagnosis, treatment plan, and prognosis for each case. These responses were first compared to the correct diagnoses, treatment plans, and prognoses for these cases, and judged as correct or incorrect based on whether they contained the majority of the essential information from the true diagnosis, treatment plan, and prognosis. Twenty neurosurgeons then independently rated the responses on a scale of one to ten, with ten being the highest score, for nineteen of the example cases.

Results: ChatGPT-4 outperformed ChatGPT-3.5 in both forms of assessment. In the initial assessment of correct responses, ChatGPT-4 had an accuracy of 85%, 75%, and 61.5% for diagnosis, treatment plan, and prognosis, while ChatGPT-3.5 only had an accuracy of 65%, 10%, and 38.5%. The average score given by the twenty independent neurosurgeons to ChatGPT-4 for its accuracy of diagnosis, treatment plan, prognosis prediction, and overall performance was also much higher than for ChatGPT-3.5: 8.3, 8.4, 8.5, and 8.4 vs. 5.9, 5.7, 5.7, and 5.8, respectively, for each category.

Conclusion : ChatGPT-4 demonstrates great promise as a predictive application of artificial intelligence in neuro-oncology for the diagnoses, treatment plans, and prognoses for patients, as attested to by the program’s performance in this study and its assessment by surveyed neurosurgeon reviewers.