Artificial intelligence models lack transparency, according to a study published Wednesday that aims to guide policymakers in regulating the rapidly-growing technology.
Stanford University researchers devised a “Foundation Model Transparency Index” that ranks 10 major AI firms.
The best score—54 percent—was given to Llama 2, the AI model launched by Facebook and Instagram owner Meta in July.
GPT-4, the flagship model of Microsoft-backed firm OpenAI, which created the renowned chatbot ChatGPT, was ranked third with a score of 48 percent.
Google’s PaLM 2 is fifth at 40 percent, just above Claude 2 of Amazon-backed firm Anthropic at 36 percent.
Rishi Bommasani, a researcher at Stanford’s Center for Research on Foundation Models, said companies should strive for a score of between 80 and 100 percent.
The researchers said “less transparency makes it harder” for “policymakers to design meaningful policies to rein in this powerful technology.”
It also makes it harder for other business to know if they can rely on the technology for their own applications, for academics to do research and for consumers to understand the models’ limitations, they said.
“If you don’t have transparency, regulators can’t even pose the right questions, let alone take action in these areas,” Bommasani said.
The emergence of AI has generated both excitement about its technological promise and concerns about its potential impact on society.
The Stanford study said no company provides information about how many users depend on their model or on the geographic locations where they are used.
Most AI companies do not disclose how much copyrighted material is used in their models, the researchers said.