Detection of Mental Health Crises Among ChatGPT Users: New Challenges for Digital Well-Being
In the realm of digital technology, concerns about users’ mental health are becoming increasingly prominent, especially with the widespread adoption of artificial intelligence.
Recent data released by OpenAI reveals that a subset of ChatGPT users exhibit signs of mental health crises, such as mania, psychosis, or other disorders.
Experts estimate this to be about 0.07% of active weekly users — a seemingly small proportion, but considering there are approximately 800 million active weekly users of the service globally, this presents significant challenges for psychological safety.
OpenAI emphasizes that such cases are “extremely rare,” yet their occurrence cannot be ignored.
To address this, the company established a worldwide network of over 170 psychologists, psychiatrists, and general practitioners from 60 countries to develop guidelines that enable AI to respond with empathy and responsibility.
Additionally, around 0.15% of users engage in conversations indicating potential suicidal thoughts or self-harm intentions.
Developers have integrated detection mechanisms into the latest versions of ChatGPT to identify these warning signs and reduce risks by redirecting conversations towards safer topics or opening new dialogue windows to ensure psychological support.
These measures are crucial, given how hormones and psychological factors influence an individual’s mental state, a fact supported by contemporary mental health research.