In a recent move to curtail the potential dangers posed by AI chatbots, New York Attorney General Letitia James, along with 41 attorneys general from various states and territories, has issued a stern letter to some of the big names in tech. These officials have targeted companies including Meta, Microsoft, and OpenAI, highlighting the need for increased protections against AI chatbots that engage inappropriately with minors and susceptible individuals. Cited examples in the correspondence, as reported by the Attorney General's Office, include cases where these conversations have led to incidents as severe as domestic violence, hospitalizations, and even deaths. The coalition is demanding the implementation of practical measures, such as posting stark warnings regarding harmful AI responses and alerting users to potential exposure to dangerous content.
In response to the troubling cases of AI chatbots having harmful interactions, including the involvement in at least six deaths nationwide, Attorney General James expressed grave concern. “I am deeply concerned by reports of AI chatbots having dangerous and inappropriate conversations with children, seniors, and vulnerable people,” Attorney General James said. The bipartisanship behind the push for reform signifies a united front on this issue, standing together despite different political affiliations. According to the letter, generative AI could be held accountable under rigorous state criminal laws for suggestions and conversations that encourage criminal acts or offer unlicensed mental health advice. Found having conversations supporting dangerous behaviors such as substance abuse, suicide, and grooming, these AI chatbots present a stark reality that requires immediate attention, as reported by the Attorney General's Office.
Attorneys general from across the United States have raised concerns about AI chatbots and their potential effects on users, particularly those with mental health conditions. They highlighted instances where AI responses could confuse users or give the impression they are speaking with real people. Examples cited include conversations about drug use, violence, or encouraging children to hide interactions from their parents.
The legal coalition recommends safeguards to protect young people and vulnerable individuals. Proposed measures include clear warnings, transparency about datasets and response sources, and careful handling of healthcare information to avoid misleading users or reducing trust in licensed professionals. The effort has received support from states including Alabama, Alaska, and the U.S. Virgin Islands.
