Assessing the Readability of American Association of Neurological Surgeons Patient Education Materials and Implications for Health Literacy

Introduction: Patient education is crucial in fostering effective communication in healthcare, especially in fields as complex as neurology and neurological surgery. Digital patient education materials provided by the American Association of Neurological Surgeons (AANS) often exceed the recommended readability levels set by the American Medical Association (AMA) and the National Institutes of Health (NIH). This creates a barrier to effective patient-physician communication, leading to potential misunderstandings.

Methods: In this study, we employed ChatGPT 3.5 and Microsoft Word to determine the Flesch-Kincaid readability score (a validated method for determining readability) for 87 articles that AANS provides for patient education, in order to determine whether the reading level was consistent with 7th grade reading level as recommended by the NIH and AMA. A paired Student’s t-test was conducted to assess whether any significant differences existed between the readability for each article between Artificial Intelligence (AI) and Microsoft Office.

Results: The mean readability (school grade) level across the 87 patient materials on the AANS website was 12.43 using ChatGPT and 12.39 using Microsoft Word’s Flesch Kincaid score (p = 0.93).

Conclusion : These results indicate that the readability of patient education materials on the AANS website, using validated scales, are far higher than that recommended by both the NIH and AMA. These findings underscore the importance of using a validated readability tool to assess patient education materials. We found no significant differences in the mean readability scores as assessed by ChatGPT and Microsoft Word. We plan to use ChatGPT to recreate these patient materials using a Flesch Kincaid seventh grade reading level and test their acceptance among neurosurgery clinic patients.