Facebook Expands Self-Harm Prevention Programme That Monitors Users’ ‘Thoughts of Suicide’

By Jennings Brown on at

Facebook is expanding its artificial intelligence-based suicide prevention efforts. The company said today that it has plans to eventually monitor and respond to suicidal intent on Facebook “worldwide,” excluding the European Union.

Today, Facebook’s vice president of product management, Guy Rosen, published a blog about the company’s efforts to detect and report users who express “thoughts of suicide.” The post states that Facebook is “[u]sing pattern recognition to detect posts or live videos where someone might be expressing thoughts of suicide, and to help respond to reports faster.” But Rosen said the company is also relying on more human reviewers who investigate suicide- and self-harm-related posts. This “Community Operations team” is comprised of thousands of workers, including a “dedicated group of specialists who have specific training in suicide and self-harm.” Reviewers use software that points them to portions of videos that have a spike in comments and reactions.

These workers also uses pattern-recognition software to determine what order they should review flagged posts and videos, and to determine which local authorities they should alert about potential suicide attempts. The flagging software looks for certain phrases, like “Can I help?” and “Are you ok?”

Facebook did not respond to a Gizmodo request for more information on how the machine-learning software flags posts that might include suicidal thoughts, which could lead to people getting reported to their local authorities.

Facebook first announced in May that it was hiring 3,000 people to help monitor the social networking site for comments, posts, and videos about murder and suicide. Ryan Calo, a professor at University of Washington who specialises in cyber law and privacy, expressed concern to Reuters over the notion of Facebook scanning comments and posts for harmful activity. “Once you open the door, you might wonder what other kinds of things we would be looking for,” Calo told Reuters.

Rosen wrote that, in the last month, Facebook connected with first responders on more than 100 cases stemming from suicide-related Facebook posts. That does not include the reports that Facebook has received from concerned users. Facebook also produced a video of police officers in Chautauqua County, New York, discussing a case in which Facebook reported a post of a person threatening to commit suicide. According to the sheriff and deputy in the video, local authorities were able to track the person down through a “cellphone ping” and took her to a hospital.

Rosen wrote that Facebook is expanding its use of suicide-prevention AI outside of the United States and plans to eventually apply the system across the world, excluding the European Union. Facebook did not respond to a Gizmodo request for information on what other countries will be included in this effort. [Facebook Newsroom]


If you struggle with suicidal thoughts, please call the Samaritans from any phone on: 116 123.


More Facebook Posts: