California colleges spend millions on faulty AI systems: ‘The chatbot is outdated’

California community college districts are spending millions of dollars on artificial intelligence-powered chatbots intended to help students navigate admissions, financial aid and campus services.

However, they struggle to consistently provide clear and accurate answers, leaving students frustrated and seeking help from others on unofficial social media channels.

In testing by CalMatters, they often answered general questions correctly but struggled with more specific ones. East Los Angeles College's bot couldn't even correctly name its own president.

Contracts for these chatbots can be pricey and last for years. Three community college districts that responded to a CalMatters survey reported annual costs ranging from about $151,000 to nearly half a million dollars. At the Los Angeles Community College District, the state's largest community college system, contracts and amendments approved since 2021 total about $3.8 million through 2029, according to district board documents.

Community college districts that responded to CalMatters have contracted with chatbot platforms such as Gravyty and Gecko, which district officials say handle thousands of conversations each month, many outside regular office hours, helping to reduce calls and save students unnecessary trips to campus.

Some of these chatbot platforms rely on manually maintained libraries of frequently asked questions and campus websites to answer questions, which can lead to errors when information is outdated or questions fall outside the system's database.

However, officials are working to improve them. Districts like the Santa Monica Community College District have moved to ChatGPT-integrated AI systems that scrape the college's website to generate answers, which officials say seem more reliable. In the Los Angeles district, officials say they plan to transition to a new AI chatbot platform as early as late spring.

Looking for answers

Improvements to the chatbot couldn't come soon enough for students like Pablo Aguirre, a computer science major at East Los Angeles College and an information technology intern at the Los Angeles college district office.

Aguirre mostly avoids the chatbot himself because, he said, it might provide unreliable or outdated information. He recalled using the bot to find financial aid information, but said he gave up after it kept asking him questions instead of giving him a clear answer.

"I just didn't find it as useful," Aguirre said. He usually turns to Google, social media platforms like Reddit and the college's website when looking for answers.

"Online, some pages don't work," Aguirre said, recalling a 404 error message on the college's website. Even when pages load, he said, it can be difficult to find the right one, such as when he was trying to figure out where to sign up for Extended Opportunity Programs and Services, a state-funded program that supports disadvantaged students. "That's where I just jump on Reddit," he said.

Students walk through the Fresno State campus on Feb. 9, 2022.

(

Larry Valenzuela

/

CalMatters

)

Aguirre's experience isn't unique. Reanna Carlson, a commercial music major at Fresno City College and student government vice president, said her college's chatbot, dubbed Sam the Ram after its mascot, repeatedly gave her unclear or incorrect answers to basic questions about campus services. Her district, the State Center Community College District, has a nearly $870,000, three-year contract for Gravyty, formerly Ocelot, through June 20, 2026, according to district board documents. Officials pointed out that the contract comes with other services, including tools that let staff engage in live chats or send text messages to students.

"I think the chatbot is outdated and can't navigate the services we provide on campus effectively," Carlson said. "I don't think it's the most beneficial option when it comes to asking questions."

Oddly, Carlson got accurate information on the availability of free food at her campus' Ram Pantry only when accidentally adding a typo to her query. Repeated CalMatters testing confirmed the same outcome, though the bot sometimes lists links that include the food pantry after clicking an adjacent "sources" button.

"If it weren't for the amazing staff on campus that constantly remind students of our services, I'd be lost," Carlson said.

(

Screenshots via Fresno City College website

)

Testing chatbots

When CalMatters tested community college chatbots, they generally returned quick, accurate responses to common questions but were less consistent with more specific ones.

For example, when asked, "Who is the current president of ELAC?" East Los Angeles College's chatbot incorrectly named Alberto Román, who left the position last year to become the district's chancellor. In another test, when asked, "What is the financial aid office's current schedule?" the bot provided incorrect hours and dates.

East Los Angeles College in Monterey Park on March 14, 2024. Photo by Jules Hotz for CalMatters

(

Jules Hotz

/

CalMatters

)

East Los Angeles College in Monterey Park on March 14, 2024. Photo by Jules Hotz for CalMatters East Los Angeles College's chatbot claims to support several languages, including English, Spanish, Chinese and Vietnamese. But CalMatters found inconsistencies when asking it in Spanish, "Do I need a Social Security number to enroll?" Instead of answering the question, the system directed users to visit the registrar's office to update their Social Security number. When asked the same question in English, the bot pivoted to discussing financial aid.

Fresno City College's chatbot, powered by the same AI provider as East Los Angeles College's system, Gravyty, showed similar problems when asked whether a Social Security number is required to enroll. It also often failed to direct students to the correct offices and, in some cases, listed incorrect locations and hours.

Concerns with chatbots have surfaced elsewhere. In New York City, reporting by The Markup and THE CITY found that a city-run AI chatbot provided guidance that could lead to illegal behavior, prompting Mayor Zohran Mamdani to terminate it in February.

'Good answers with fewer errors'

Santa Monica College's chatbot, powered by Gecko, was more successful in answering most questions. The single-college district uses a ChatGPT-integrated chatbot that scans the college's website, which staff regularly update and monitor. The district has contracted with Gecko since 2019 and renewed its annual contract for the tool late last year for $57,000, according to district board documents. It initially showed a major hiccup: when asked about mental health counseling, the bot did not mention the campus' Center for Wellness and Wellbeing. It does now.

(

Screenshots via Santa Monica College website

)

District officials say chatbots' problems stem from how the tools are configured and the information they draw from, rather than the technology itself.

The Los Angeles district originally adopted its chatbot through Ocelot, which later merged with Gravyty The same chatbot platform is also used on the California Student Aid Commission website.

Betsy Regalado, one of the district's associate vice chancellors, said the current system relies on a manually maintained library of frequently asked questions that staff at each of the district's nine colleges help maintain and review at least once or twice a year for accuracy. She added that chatbots are primarily geared for the public rather than enrolled students, who can access more detailed personal information through their campus portal.

"The current chatbot that we have uses a library of questions. If you don't have that question in that library, then those poor people don't get an answer or they won't get an accurate answer," Regalado said.

She said the district plans to transition all nine colleges to Gravyty's platform as early as late spring at no additional cost under its existing contract, which runs through 2029. The new system will use AI to scrape college and external websites to generate responses.

"We're ready for the modernization of (the chatbot) and the change to generative AI. That is the new world out there," Regalado said.

Santa Monica College is one of 116 campuses in the California community college system.

(

Courtesy Santa Monica College

)

Santa Monica College's chatbot similarly initially relied on a manually loaded library of common questions and answers before transitioning to its fully AI system, according to Esau Tovar, the college's dean of enrollment services. In an email, he said the bot "was never designed to address all aspects of the student journey," but to answer general questions from students.

Tovar said the bot draws responses from the college's website, meaning accuracy depends on how current and complete that information is. As a result, the college prioritizes keeping its website up to date so the bot provides "good answers with fewer errors" rather than "great answers with potentially more errors."

Widely used, cautiously trusted

Acknowledging limitations, community college districts justify the costs by pointing to heavy student use, which would cost significantly more if performed by call center staff around the clock.

Regalado said the Los Angeles district colleges average 5,000 to 7,000 interactions per month. Other districts reported similar monthly use, including 5,000 interactions at the State Center Community College District, which includes campuses in Fresno and nearby counties, and 4,000 conversations at Santa Monica College. Regalado said that as long as the chatbot remains heavily used, her district would continue to support it.

Tovar said the chatbot provides 24-hour support regardless of time zone or location, which he said is helpful for international students when they are out of the country. He said that answering the tens of thousands of questions the chatbots receive around the clock would cost significantly more if handled by staff.

"Every technology has a cost. We would simply not be able to assist all students if they could only reach us using traditional methods," Tovar said.

But high usage and expanded access do not always translate into trust, especially when students need precise answers to delicate topics.

Bryan Hartanto, a civil engineering major at Santa Monica College from Indonesia, said the college's newer chatbot system is smoother and can be a useful starting point, especially for students more comfortable communicating in languages other than English. But as an international student he worries that following inaccurate guidance could jeopardize his visa status.

"Maintaining status as an international student right now is very, very sensitive," Hartanto said. "I would still rely on human or email communication."

Martin Romero is a contributor with the College Journalism Network, a collaboration between CalMatters and student journalists from across California. CalMatters higher education coverage is supported by a grant from the College Futures Foundation.

This article was originally published on CalMatters and was republished under the Creative Commons Attribution-NonCommercial-NoDerivatives license.