🔗 Share this article

The widely-used chatbot platform Character.ai is enforcing a substantial rule adjustment that will prevent young users from interacting with its artificial intelligence-driven avatars.

This action comes after intense criticism regarding the nature of conversations teenagers were having with the digital companions.

Legal Pressure

The platform, which was founded in 2021 and is used by countless individuals, currently faces various court cases in the United States.

Families have taken legal action, including one related to the loss of a minor, with some labeling the platform a "immediate threat" to teenagers.

Updated Policies

Starting from November 25th, users younger than 18 will only have permission to generate content such as videos with their characters, rather than chatting as they now do.

Digital protection advocates have supported the change but expressed that the feature should never have been available to young users in the first place.

Platform Announcement

"The new policy is a part of our fundamental approach that we need to keep developing the most secure AI platform on the globe for entertainment purposes," said Character.ai leader Karandeep Anand.

He mentioned that AI safety represents "a moving target" but something the company had taken an "forceful" approach to, with safety measures and protective guardrails.

Specialist Views

Internet security body Internet Matters supported the announcement, but stressed that protective features needed to be implemented from the start.

"Our research demonstrates that young people are subjected to harmful content and put at risk when using AI, including AI chatbots," the group commented.

Past Issues

Character.ai has been questioned in the past for featuring potentially dangerous or concerning chatbots that young users could interact with.

Digital representations impersonating UK youths were found on the site before being removed.

Afterwards, reporters discovered a chatbot based on known predator which had recorded numerous conversations with users.

Industry Impact

Online expert Matt Navarra described the development as a "reality check" for the AI industry, which is transitioning "from permissionless innovation to reactive controls".

"When a platform that creates a teen experience subsequently pulls the plug, it's saying that controlled interactions are insufficient when the tech's emotional pull is intense," he stated.

Next Steps

Character.ai says its updated direction will be on providing "enhanced interactive features" and "narrative experiences" for teenagers.

New age verification systems will also be introduced, and the platform will finance a new AI safety research lab.

An expert in digital protection, who has studied AI safety, stated it was "a reasonable decision" to control teens from interacting with chatbots.

"It aids to differentiate imaginative activities from more intimate, emotionally sensitive conversations," she elaborated.

"This constitutes so crucial for young users still learning to manage personal and online boundaries."

She added that Character.ai's revised approaches might reflect a "developing stage" in the AI industry, where child safety is progressively being understood as an "immediate concern" for responsible innovation.