Breaking new ground in the intersection of medical education and artificial intelligence, University of Cincinnati College of Medicine students have conducted a study investigating AI's role in qualitative research, particularly in thematic analysis. According to the University of Cincinnati's news release, Jonathan Bowden and Megha Mohanakrishnan, both third-year medical students, spearheaded the research, testing the AI tool ChatGPT's ability to code survey responses from their peers on the subject of cadaveric dissection.
The students, who had previously collaborated with Professor-educator Andrew Thompson, manually analyzed their classmates’ responses and then set out to determine if ChatGPT might match their analytical prowess, the process not only compared thematic coding but also engaged with three prompt methods to refine AI accuracy—the methods, despite their varying complexity, all aimed to improve ChatGPT's response coding by incorporating a mix of themes, definitions, and example responses for the AI to learn from.
Bowden and Mohanakrishnan’s project earned them a national award, signaling the potential for AI to streamline research practices in fields traditionally reliant on manual analysis. The methodology ranged from simply providing ChatGPT with a list of defined themes to giving it example responses with assigned themes and even providing direct feedback to adjust its coding strategy. It was determined that the third method, which involved an iterative feedback loop with the AI, led to the highest accuracy in coding the responses.
"We tried to engage more and more with the AI to improve its accuracy," Bowden explained, per the University of Cincinnati's news release, showcasing their commitment to assessing the efficacy of the technology thoroughly. The research also offered a comprehensive learning experience, with Bowden sharing, "I learned how to take on a project from the ground up," he further remarked, "We gained insight into the planning and execution of a research project," with aspirations to combine practice and teaching in the field of internal medicine in the future, his endeavors embody a forward-thinking approach to medical education.
