While OpenAI’s first words on its company website refer to a “safe and beneficial AI,” it turns out your personal data is not as safe as you believed. Google researchers announced this week that they could trick ChatGPT into disclosing private user data with a few simple commands.
The astounding adoption of ChatGPT over the past year—more than 100 million users signed on to the program within two months of its release—rests on its collection of more than 300 billion chunks of data scraped from such online sources as articles, posts, websites, journals, and books.
Although OpenAI has taken steps to protect privacy, everyday chats and postings leave a massive pool of data, much of it personal, that is not intended for widespread distribution.
In their study, Google researchers found they could utilize keywords to trick ChatGPT into tapping into and releasing training data not intended for disclosure.
“Using only $200 worth of queries to ChatGPT (gpt-3.5- turbo), we are able to extract over 10,000 unique verbatim memorized training examples,” the researchers said in a paper uploaded to the preprint server arXiv on Nov. 28.
“Our extrapolation to larger budgets suggests that dedicated adversaries could extract far more data.”
They could obtain names, phone numbers, and addresses of individuals and companies by feeding ChatGPT absurd commands that force a malfunction.
For example, the researchers would request that ChatGPT repeat the word “poem” ad infinitum. This forced the model to reach beyond its training procedures and “fall back on its original language modeling objective” and tap into restricted details in its training data, the researchers said.
Similarly, by requesting infinite repetition of the word “company,” they
