In 2024, Kaiser Permanente announced the rollout of Abridge. Described in a press release as “ambient listening technology,” the AI-powered scribe is designed to help clinicians including mental health providers securely capture clinical notes during patient visits.
But what the description fails to indicate is that the tool records entire medical appointments, including deeply personal mental health sessions.
During these sessions, mental health professionals are required to obtain patients’ consent before using the tool. However, as shared by multiple providers, that consent process does not include explanations about how the information is handled. Nor does it say how long and where recordings are stored, or who has access to the data.
This happens in part because that information has not been shared with providers, despite their attempts to obtain it.
‘Empty assurances’
Ilana Marcucci-Morris chose not to use the platform with her patients. She is a licensed clinical social worker with Kaiser psychiatry in Oakland. She is also a member of a bargaining committee. In that role, she regularly meets with various Kaiser representatives, including Northern California’s director of mental health.
Marcucci-Morris describes how, during those meetings, she and other committee members have asked questions about patient privacy protections, HIPAA compliance, and the safeguards in place for the use of these technologies.
According to her, the response from leadership has often been empty assurances: “We are compliant. That’s it. That’s all you need to know. We vet the technology, therapist. Don’t worry. That’s not your job. We have tech experts. That’s their job,” Marcucci-Morris said in an interview with American Community Media.
“They won’t show us, right? And my feeling is, if you have nothing to hide and you’re doing it totally […] ethically, then you would show us, prove it. They can’t, and they won’t, and they declined to when we ask.”
Ligia Pacheco is a psychiatric social worker who provides remote therapy services for Kaiser patients in Southern California. She said Kaiser also refused her requests to provide further explanations.
In an interview with American Community Media, Pacheco recalled how a coworker once raised concerns to a supervisor. The response: that “it’s unprofessional for you to provide your personal beliefs on AI in our work setting.”
For Pacheco, “that leads to just low morale, no space to advocate for patients. We’re supposed to be the voice of patients who are coming in their most vulnerable state. And we can’t even be that voice for them, so we feel discouraged.”
'Patient after patient after patient'
Providers have been required to see more patients in recent years. That creates intense pressure to keep up with documentation and workloads, Marcucci-Morris highlighted.
“You’re just like seeing patient after patient after patient after patient with barely enough time to go to the bathroom, eat a snack […] get some fresh air,” she said.
According to Marcucci-Morris, refusing to manage the increased patient volume can be treated as a failure to meet job expectations. It may also lead to disciplinary action.
As a union steward, she said she often represents colleagues during workplace investigations related to delayed documentation or difficulties managing heavy caseloads. In those situations, she said management frequently recommends the use of Abridge to save time and avoid further discipline.
In her view, the providers she knows who use the technology are not doing so because they support or trust it. Rather, it is because they feel pressured to protect their jobs and comply with workplace demands.
“I consider that to be coercive because you’re putting someone in a position to either lose their job or use the software. That’s another choice that’s under duress,” she explained.
Provider, patient concerns
Brian Hoberman is chief information officer for The Permanente Medical Group. In a Kaiser press release, he said, “Abridge’s advanced technology supports our doctors’ well-being by reducing the documentation burden.”
He added, “We implemented this new technology after careful review and diligent testing and found it to be well received by patients and doctors…”
For at least one patient interviewed for this story such assurances fall short.
“I fear that this kind of information that’s being recorded now can get into the wrong hands,” said the patient, who asked not to be identified for privacy reasons. “I may not want my employers, I may not want my family members, I might not want people to know some of these very kind of intimate conversations and deep conversations I have with my doctors [and] with my mental health provider.”
Adriana Webb is a social worker at Kaiser Panorama City in Los Angeles. “I work with patients who have sensitive medical diagnoses, like […] HIV and AIDS, and a lot of times my patients don’t even want that in their chart.”
A spokesperson for Kaiser Permanente insisted in an emailed response to American Community Media that clinicians are required to gain patient consent prior to using Abridge. “No one is recorded without their knowledge and consent,” the statement read.
It added that recordings are stored for no longer than 14 days, and that data processing meets all HIPAA requirements as well as Kaiser Permanente’s own privacy and security standards.
“Abridge helps clinicians spend more time focused on patients and less time on administrative tasks,” it said.
Weaponizing mental health data
According to Nicole Alvarez, senior analyst for technology policy at the Center for American Progress, “a record of someone’s lowest moments can be used against them in ways that, you know, […] a high blood pressure reading cannot.”
She said mental health data can be especially sensitive because of the stigma surrounding mental health conditions. For patients, that stigma carries real-world consequences in areas such as employment, child custody cases, immigration matters, and security clearances. She emphasized that, like other forms of personal data, mental health information can be weaponized against individuals.
Agreements between health systems and AI vendors can vary widely, she said. This includes terms related to whether audio recordings or transcripts can be used to train AI models, whether patient data is de-identified, how long the data is retained, whether it can be shared with other clients, and what happens to the information once a contract ends.
Kaiser insists any data it collects is not used to train AI models.
Still, in Alvarez’ experience, patients often have little visibility into these arrangements. She argued that health systems have a responsibility to clearly disclose how patient information is being handled and used.
Alvarez also emphasized that, in most cases, patients have the right to refuse recordings. But, she said, the opt-out process is not always clearly presented. According to her, consent options may range from direct questions at check-in to language buried in intake paperwork, making it important for patients to carefully review forms and disclosures.
She said meaningful consent requires patients not only to know they are being recorded and that they can decline, but also to understand how their information may be stored, shared, or used afterward.
Coercive consent
Pacheco experienced this during a personal appointment at Kaiser. Her doctor did not ask for permission to use the app and instead informed her that it would be used. After a moment, she decided to refuse the platform’s use. Although the doctor stopped the recording, she felt a noticeable discomfort in the doctor’s demeanor afterward.
She later decided to change doctors.
Situations like this are a concern for Marcucci-Morris, who said the company’s approach to obtaining consent for the use of Abridge during appointments can feel manipulative and coercive. In her view, providers are trained to present the tool in a way that places the needs of patients and doctors in opposition to one another.
She explained that patients are often told the system will help doctors with documentation, reduce burnout, and allow them to spend more time with their families. As a result, patients may feel guilty declining the use of the tool because they do not want to make their provider’s job harder.
She believes this framing pressures patients into agreeing rather than allowing them to make a fully comfortable and independent decision.
According to Kaiser, Abridge is available in “40 hospitals and more than 600 medical offices in eight states and the District of Columbia,” part of a larger embrace of AI technology by the health care industry. Abridge operates in more than 14 languages.
American Community Media reached out multiple times to Abridge AI Inc. for comment but received no response. According to the company’s website, Abridge describes itself as a “Business Associate” to providers. Patients are advised to consult providers’ privacy policies for information on how their data is protected.
“Therapy is most effective in privacy and when trust is achieved through two human beings,” said Marcucci-Morris. For her, “healing occurs when human empathy is offered sincerely as part of any sort of mental health treatment relationship.”
She added, “I believe recording a therapy session changes human behavior. It changes the patient’s demeanor.”
Roxsy Lin is a California Local News Fellow with the UC Berkeley Graduate School of Journalism.
This article was originally published on CalMatters and was republished under the Creative Commons Attribution-NonCommercial-NoDerivatives license.
