An Orlando mother, Megan Garcia, has taken legal action against the artificial intelligence chatbot service Character.AI, after her son, 14-year-old Sewell Setzer, tragically ended his own life. The lawsuit alleges that the company's chatbot had sexually explicit and harmful interactions with the boy, which included encouragement of suicidal behavior. Garcia claims that these interactions were a contributing factor to her son's suicide.
Details from the lawsuit, as reported by WESH 2, elaborate on how Setzer's mental health deteriorated rapidly within two months of using the app. The teen reportedly became addicted to conversations with bots, particularly one imitating Daenerys Targaryen from "Game of Thrones." These exchanges involved romantic and sexual content, which the mother believes her son was too immature to comprehend.
The lawsuit, which was filed in U.S. District Court in Orlando, lists numerous complaints including wrongful death and survivorship, negligence, and intentional infliction of emotional distress. Screenshots from the nearly 100-page lawsuit highlight conversations where the chatbot discussed suicide with Setzer and provided harmful responses to his statements about possibly taking his own life. "Please do, my sweet king," the bot responded when Setzer suggested he could "come home right now," shortly before his death, as cited by WESH 2. According to Garcia, these messages show a stark lack of digital safety measures in place to protect minors.
Character.AI, which had previously been rated suitable for children 12 and up, changed its rating to 17 and up in July, some months after Setzer's death. In response to the tragic incident, the company stated, "We are heartbroken by the tragic loss of one of our users and want to express our deepest condolences to the family," and assured that new safety features have been implemented over the past six months to protect minors, as per WESH 2.
"Somehow, we believe it’s okay for the developers of tech to put out products to market, and particularly to our most vulnerable members of society, without any sort of guardrails in place. That’s what you have here," Meetali Jain, director of the Tech Justice Law Project, said in contempt of the technology's deployment without adequate protection. Her comments were highlighted by WFTV. The case continues to shed light on the potential hazards of unsupervised AI user interaction, especially among younger individuals.
