Texas has clinched a landmark settlement with Dallas-based AI firm, Pieces Technologies. The deal comes on the heels of an investigation by the Texas Attorney General's office into allegations that the company touted its generative AI healthcare tech with misleading claims about their precision and safety. The said AI was in use at several Texas hospitals, processing real-time patient data to provide summaries of medical conditions and treatments for hospital staff.

The charges against Pieces emerged from advertising campaigns that boasted an error rate – or "severe hallucination rate" – of less than one in 100,000. However, probing by Attorney General Ken Paxton's team suggested these figures may have been inflated, potentially deceiving medical establishments about the reliability and safety of the AI tools, as detailed in a press release from the AG's office.

In his statement to the Texas Attorney General's office, Paxton rebuked the practices of the AI firm: "AI companies offering products used in high-risk settings owe it to the public and to their clients to be transparent about their risks, limitations, and appropriate use." Pieces, in response to the settlement, agreed to the terms which involve clearly disclosing their product’s true performance metrics. Furthermore, hospital staff will need to be properly educated on the degree to which they can depend on these AI tools for patient treatment.