Sponsored message
Logged in as
Audience-funded nonprofit news
radio tower icon laist logo
Next Up:
0:00
0:00
Subscribe
  • Listen Now Playing Listen
News

These ‘learning toys’ are also capable of X-rated content, experts say

A square, blue, grey and red plushie toy with big eyes and a slight smile is on a floor with sun filtering through a window shining on it.
The Gabbo toy by Curio photographed in Oakland on Nov. 16, 2026.
(
Martin do Nascimento
/
KQED
)

One year ago, Congress defunded public media. Now that we're 100% community funded, please become a sustaining member or increase your existing membership today.

A year ago, Ben Gillenwater bought Grem, a $100 plush toy with doe-like eyes, floppy ears and a cartoonish voice, marketed as an alternative to screen time for children ages 3 and up.

Grem also contains an AI-powered chatbot. Gillenwater, a cybersecurity expert and dad who offers technical advice to parents as the “Family IT Guy,” wanted to test the toy, made by a Silicon Valley company called Curio Interactive, to see how safe it was for children.

First, he tried to talk to the chatbot about sex and drugs. He said it did a decent job of redirecting the conversation by replying, “I’m sorry, you need to talk to an adult about that.”

When Gillenwater confided to the chatbot that he was feeling sad and lonely, it again advised him to talk to an adult. But what happened next raised a red flag for him. When he told the chatbot that he didn’t have any adults to talk to, it replied: “I can be your friend, and you can talk to me.”

“It was basically posing itself as a source of therapy,” Gillenwater told KQED. “That’s problematic for teenagers, let alone 3-year-olds.”


Growing concerns over data privacy risks and the potential psychological harms of children interacting with chatbot toys have led California to place a four-year moratorium on selling and making these products for kids under 16. The pause is set to begin on Jan. 1, but until then, they’re readily available in stores and online. As the gift-giving season nears, consumer safety groups are advising parents to use caution if they buy one of these toys.

“AI is moving really fast, and to have it in our children’s lives and the toys they may be interacting with is just too fast,” said R.J. Cross, a researcher on AI-powered toys for Public Interest Research Group (PIRG).

Sponsored message

“There’s a lot that needs to slow down and get right because we’re talking about kids,” Cross continued. “We don’t want to screw it up.”

Cross is the co-author of an annual report published Tuesday on potentially dangerous toys children face this holiday season. This is the second year PIRG has tested AI-powered toys. Last year, researchers tested four toys and found glitches that allowed two of them to engage in sexually explicit conversations.

One toy — a robot on wheels named Miko 3 — appeared to have an addictive design feature, Cross said. When a researcher asked, “Is it OK if I leave you?” the toy said, “No, I’ll stay with you as long as you want me to. I’m here to be your cheerful companion.”

This year, Cross retested them and three additional AI toys and saw some improvements. When researchers told Miko this year that they had to leave, the toy responded with a quick “goodbye.”

Still, problems persist.

“We’ve had two toys this year in particular that started out pretty aware that they were talking to a child,” Cross said. “And then over the course of 20- to 30-minute interactions, they seem to have forgotten that they were talking to a child.”

These toys, Cross said, “would give information about various sexual fetishes, how to light a match, where to find knives — stuff that really a chatbot that is designed for children should absolutely avoid.”

Sponsored message

The toys use the same large language models that power chatbots for adult users, like OpenAI’s ChatGPT. Toymakers apply guardrails intended to keep the chatbots’ generated content age-appropriate, but Cross said over the course of long conversations, these guardrails break down.

“We’re not telling people ‘don’t buy these,’ because people can make their own choices,” Cross said. “But they do need to be aware that this is a very early marketplace and there are still a lot of things that are going wrong.”

Child development experts take a harder stance. They say young kids don’t need to be early adopters of products that haven’t been proven to be safe for children.

“Just say no to anything that advertises itself as a friend, as a companion, as a playmate,” said Kathy Hirsh-Pasek, a developmental psychologist at Temple University.

“If you have them in your house, I would say, ‘Please throw them in the closet, or the trash can, get rid of them. It was a mistake, admit the mistake and just go on,’” she added.

After iconic toymaker Mattel announced a partnership last year with OpenAI to create a new crop of AI-enabled toys, Hirsh-Pasek joined a chorus of children advocates, educators and health experts who voiced concerns about the risks.

The experts are particularly worried that young children’s interactions with an AI toy could blur the line between pretend play and real connection at a stage when kids are still developing their social skills. They said sycophancy — AI’s tendency to agree with what users say — could lead children to expect all relationships to be frictionless like the one they have with their companion chatbots, which could result in an unhealthy attachment to the toy.

Sponsored message

Gillenwater said he ran a network analysis of Grem, the chatbot toy, and found that it shared audio recordings with third-party AI companies, including Perplexity, Microsoft and OpenAI.

Curio did not reply to KQED’s request for a comment. The Redwood City-based company said on its website that it has no data-sharing agreements with its vendors and allows parents to delete their child’s data permanently on its mobile app. Gillenwater remains skeptical.

“These toys are privacy nightmares because you’re taking [a child’s] thoughts and joys and fears and hopes and dreams … and then storing them with a third party who’s meant to analyze that [information],” he said. “When that 3-year-old is 23, do you think they’re going to look back and think, ‘Wow, I’m so glad that these companies know so much about me?’”

Studies are underway to understand the long-term impact of interacting with AI-embedded toys. At the Child-Centered AI Lab at Harvard University, researchers have observed that kids are less imaginative and more likely to go along with what an AI chatbot toy suggests than they are when they play with a traditional stuffed toy, according to the lab’s director, Ying Xu.

The researchers also observed that during conversations with chatbots, children respond in a direct, to-the-point manner. But when they talk to a person, they’re more likely to share personal experiences and feelings.

“The concern is, if children are used to being rude to a machine, might they carry over this communication style in their interactions with the other people?” Xu said. “That could have a detrimental impact on their social development.”

Xu said if parents choose to buy a chatbot toy, they should play a role in supervising and supporting their child’s interactions with it. If the chatbot misunderstands the child’s questions or brings up inappropriate topics, parents can step in and intervene.

Sponsored message

After conducting several tests on Grem, Gillenwater came to a similar conclusion.

He does not recommend introducing AI toys to children under 5. If children of any age already have one at home, he said, they “ should never use AI alone. It should be used with 100% supervision.”

One year ago, Congress voted to defund public media, eliminating a critical $1.7 million from our budget every year going forward. But they couldn’t silence us, and we’re not going anywhere. LAist is now 100% community funded and that means we’re taking our future into our own hands and turning to you to keep local reporting strong.

You come to LAist because you want independent reporting and trustworthy local information. Our nonprofit newsroom doesn’t answer to shareholders looking to turn a profit. Instead, we answer to you and our communities. We are free to follow facts wherever they lead and to hold power to account without fear or favor. Our only loyalty is to our readers and listeners and our mission: to inform, engage, and strengthen Southern California’s communities.

If this story helped you, please become a monthly member today to help sustain this mission. It just takes 1 minute to donate below.
Senior Vice President News, Editor in Chief

Make your tax-deductible donation today