In artificial intelligence, understanding the data used to train models is essential. A recent MIT study found that there is often a lack of transparency about these datasets. The researchers examined over 1,800 text datasets and discovered some serious issues with them.
With a degree of misattribution that anyone could describe as problematic, over 70 percent of the datasets had licensing information missing, and about half the datasets analyzed contained erroneous information. Obtained information from a new study published by MIT reveals the extent to which this opacity can damage not just a model's performance, but also its fair application in real-world scenarios.
Addressing this widespread issue, the MIT team, including Alex "Sandy" Pentland, a professor at the MIT Media Lab, and Robert Mahari, a graduate student, introduced the Data Provenance Explorer—a tool designed to generate concise summaries of a dataset’s sources, creators, licenses, and permissible uses. "One of the best ways to understand the capabilities and limitations of an AI model is understanding what data it was trained on," Mahari told MIT News. This diagnostic tool could serve as a beacon for AI developers seeking clarity on the datasets they employ for model training.
Clear information about datasets is crucial, especially when data originally licensed for specific uses gets mixed into larger collections and loses its original license details. Shayne Longpre, a co-author of the study, warns that people might "People can end up training models where they don’t even understand the capabilities, concerns, or risk of those models, which ultimately stem from the data," the statement obtained by MIT News. The study also noted that most dataset creators are from the global north, raising questions about the cultural relevance and usefulness of these models in other regions.
The study, which delved into the licensing and ethical dimensions of AI datasets, has broad implications for the future of AI development. The researchers performed a structured audit that eventually reduced the datasets with unspecified licenses to around 30 percent after manually backtracking to find the missing information. They also reported a spike in restrictions on datasets created in recent years—2023 and 2024, suggesting a growing concern over the commercial misuse of academic work.
Stella Biderman, the executive director of EleutherAI who wasn't part of the study, stated that "Many people who want to build AI systems for public good are currently quietly struggling to figure out how to handle data licensing, because the internet is not designed in a way that makes data provenance easy to figure out," according to MIT's article. The team plans to broaden its research to include multimodal data and has started discussions with regulators to promote data transparency from the beginning.
