The setting that lets OpenAI use chats this way is on by default for free, Plus and Pro accounts.
404 Media reports that OpenAI is hiring hundreds of contractors to read real ChatGPT prompts, at times entire conversations, and grade the chatbot's replies so it learns to answer better. The report draws on the contractors' instruction guides and some of the real prompts they review.
According to 404 Media, the reviewers' dashboard leaves out the account holder's username, but prompts can still carry personal information. Some arrive with a summary of how the person has used ChatGPT before, occasionally including where they may live. In some, users asked ChatGPT to keep what they wrote to itself, a sign they never expected a person to see it.
OpenAI told 404 Media it first runs conversations through a version of its Privacy Filter model to strip personal information, and acknowledged that sensitive information can still slip past it. OpenAI's page on the model warns that it can miss uncommon identifiers. One of the guides tells contractors to escalate tasks that contain personal information.
The documents refer to the work only by the codename Project Lily. Reviewers sum up what each user wants, then score several ChatGPT replies, marking down sycophancy and answers in which the chatbot implies it is human or has feelings. The report distinguishes this from the safety reviews OpenAI has announced publicly.
404 Media says OpenAI did not initially answer when asked whether, and where, it had explicitly informed users that people might read their prompts to make ChatGPT better. After the outlet got in touch, OpenAI added opt-out detail to its help page for the "improve the model for everyone" setting, though 404 Media found it still did not mention human readers.
After the story ran, OpenAI pointed 404 Media to a section of its data-usage FAQ saying people may review content to “improve model performance.” Switching the setting off keeps new conversations from being used to improve its models, OpenAI says. Its help page names one exception: a conversation the user rates with a thumbs up or down may still be used. Enterprise, Business and Edu accounts have the setting off by default.
Anthropic told 404 Media that it, too, has people review conversations to improve its models, using conversations from users who have turned on its model-improvement setting, and that it removes account identifiers such as email addresses first. Google's Gemini tells users that humans review some saved chats.