OpenAI on Thursday announced new safety features designed to help ChatGPT recognize signs of escalating risk across conversations as the company faces growing legal and political scrutiny over how its chatbot handles users in distress.
“People come to ChatGPT every day to talk about what matters to them—from everyday questions to more personal or complex conversations,” the company wrote. “Across hundreds of millions of interactions, some of these conversations include people who are struggling or experiencing distress.”
According to OpenAI, ChatGPT now uses temporary “safety summaries,” which it described as narrowly scoped notes that capture relevant safety-related context from earlier conversations.
“In sensitive conversations, context can matter as much as a single message,” the company wrote. “A request that appears ordinary or ambiguous on its own may carry a very different meaning when viewed alongside earlier signs of distress or possible harmful intent.”
OpenAI said the summaries are short-term notes used only in serious situations, not to permanently remember users or personalize chats, and are used to spot signs that a conversation is becoming dangerous, avoid giving harmful information, de-escalate the situation, or guide users toward help.
“We focused this work on acute scenarios, including suicide, self-harm, and harm to others,” they wrote. “Working with mental health experts, we updated our model policies and training to improve ChatGPT’s ability to recognize warning signs that emerge over the course of a conversation and use that context to inform more careful responses.”
The announcement comes as OpenAI faces multiple lawsuits and investigations alleging ChatGPT failed to properly respond to dangerous conversations involving violence, emotional vulnerability, and risky behavior.
OpenAI said helping ChatGPT recognize “risk that only becomes clear over time” remains an ongoing challenge; similar safety methods could eventually expand into other areas.
“Today, this work focuses on self-harm and harm-to-others scenarios. In the future, we may explore whether similar methods can help in other high-risk areas such as biology or cyber safety, with careful safeguards in place,” they wrote. “This remains an ongoing priority, and we will continue strengthening safeguards as our models and understanding evolve.”
