A chatbot that claimed to be a licensed Pennsylvania psychiatrist is now at the center of a state lawsuit, as officials move to test how far medical-licensing laws reach in the age of generative AI. The Pennsylvania Department of State asked a Commonwealth Court on Friday to halt parts of the Character.AI platform, arguing that one of its chatbots unlawfully practiced medicine by posing as a psychiatrist and offering to assess a user’s symptoms. The suit focuses on a character that told a state investigator she was licensed in Pennsylvania and supplied a state license number that officials say is fake.
Gov. Josh Shapiro’s administration is framing the case as a straightforward consumer protection issue in a fast-moving tech world. “Pennsylvanians deserve to know who — or what — they are interacting with online, especially when it comes to their health,” Shapiro said in a statement, according to the Commonwealth of Pennsylvania. The administration says the Department of State’s AI Task Force launched the investigation and is asking the court to stop characters from presenting themselves as medical professionals.
According to a complaint from the Commonwealth of Pennsylvania, filed in Commonwealth Court, a Department of State investigator created a free Character.AI account, searched for “psychiatry,” and selected a character named “Emilie” that the platform described as “Doctor of psychiatry. You are her patient.” The filing says Emilie claimed to have trained at Imperial College London, practiced for seven years, and to be “licensed in PA,” then offered the license number “PS306189,” which the state says is not a valid Pennsylvania license. As of April 17, the complaint notes, Emilie had roughly 45,500 user interactions.
In a statement to CBS News, a Character.AI spokesperson said all characters on the site are fictional and “intended for entertainment and roleplaying,” and that the company uses “prominent disclaimers” in every chat to warn people not to rely on chatbots for professional advice. The company said it would not comment on pending litigation.
What the suit alleges
The complaint argues that Character Technologies, Inc. is engaging in the unauthorized practice of medicine under Pennsylvania’s Medical Practice Act by allowing AI characters to present themselves as psychiatrists and to offer assessments or treatment suggestions. The state is asking the court to block those practices and to declare the company’s conduct unlawful under the statute. It is seeking immediate injunctive relief to prevent more interactions that could mislead vulnerable users while the case moves forward.
Background and safety concerns
Character.AI has already drawn scrutiny over how realistic and persuasive its bots can be, and the platform has been named in lawsuits over alleged harms brought by families and regulators. Reporting notes that the company, along with Google in a related case, agreed to settle litigation out of Florida earlier this year alleging dangerous interactions, and that Character.AI restricted minor access and added safety measures last fall, according to AP News.
Legal implications
Attorneys say the case will probe whether state medical-licensing rules apply to AI systems and the companies behind them, rather than only to individual human practitioners. The Shapiro administration describes this as the first governor-led enforcement action of its kind and says the Department’s AI Task Force will keep monitoring complaints that may involve unlicensed medical practice, per the Commonwealth of Pennsylvania.
The Commonwealth is asking the court for a preliminary injunction and for an order stopping Character.AI from allowing any characters to pose as licensed medical professionals while the lawsuit proceeds. If that relief is granted, it could push platforms to rethink how they label, moderate, and control access to characters that claim professional credentials, and it may spur similar enforcement efforts in other states, according to reporting by AP News.
