The American Medical Association is calling on Congress to establish safeguards for AI chatbots used in mental healthcare, citing concerns about patient safety, misinformation and privacy risks.
The association outlined four focus areas for lawmakers in letters sent to leaders of the Senate Artificial Intelligence Caucus, Congressional Digital Health Caucus and Congressional Artificial Intelligence Caucus, according to a May 14 new release.
Here are three things to know:
- The organization called for stronger transparency requirements for AI chatbots. Recommendations include requiring chatbots to clearly disclose that users are interacting with a machine rather than a human clinician and whether any human oversight is involved. The AMA also said AI chatbots would be prohibited from presenting themselves as licensed clinicians or claiming to provide the same services as healthcare professionals.
- The AMA urged Congress to establish targeted recommendations for AI mental health tools. Proposed measures include prohibiting chatbots from diagnosing or treating mental health conditions, such as recommending medications or offering anxiety and depression diagnoses. The organization also recommended requiring chatbots to identify suicidal ideation and self-harm risks, provide referrals to suicide prevention resources and undergo stronger scrutiny when technologies are designed for children and adolescents.
- The organization also seeks restrictions on advertising, data sharing and information retention within AI mental health chatbots. Recommendations include discouraging advertising inside mental health support tools, prohibiting advertising targeted at minors and limiting the collection and retention of sensitive user information. The AMA also urged Congress to require safeguards against unauthorized sharing of conversation and connections to third-party services without user approval.
At the Becker's Fall Behavioral Health Summit, taking place November 4–5 in Chicago, behavioral health leaders and executives will explore strategies for expanding access to care, integrating services, addressing workforce challenges and leveraging innovation to improve outcomes across the behavioral health continuum. Apply for complimentary registration now.
