🔗 Share this article

The widely-used virtual character service Character.ai is enforcing a significant rule adjustment that will block young users from having discussions with its AI-powered avatars.

This decision comes after intense criticism regarding the types of interactions young people were having with the virtual friends.

Legal Challenges

The service, which was founded in 2021 and has numerous people, is dealing with several legal actions in the America.

Guardians have taken legal action, with one involving the unfortunate passing of a minor, with some labeling the platform a "immediate threat" to youth.

Changed Rules

Effective November 25th, users younger than 18 will only have permission to generate content such as videos with their characters, as opposed to engaging in conversations as they now do.

Internet security organizations have welcomed the move but stated that the function was inappropriate to be accessible by minors in the outset.

Platform Announcement

"The new policy is a part of our general belief that we need to keep building the most secure AI platform on the planet for entertainment purposes," stated Character.ai executive Karandeep Anand.

He continued that AI safety constitutes "a evolving challenge" but something the company had taken an "aggressive" approach to, with guardian features and safety barriers.

Professional Perspectives

Internet security group Internet Matters approved the decision, but noted that safety measures needed to be incorporated from the outset.

"Studies we've conducted shows that children are exposed to dangerous material and put at risk when engaging with AI, particularly AI chatbots," the organization commented.

Past Issues

Character.ai has been questioned in the past for hosting potentially dangerous or offensive chatbots that minors could interact with.

Digital representations simulating British teenagers were identified on the service before being removed.

Later, investigative journalists identified a chatbot inspired by convicted sex offender which had logged multiple interactions with users.

Industry Impact

Social media expert Matt Navarra characterized the announcement as a "wake-up call" for the AI industry, which is transitioning "from permissionless innovation to post-crisis regulation".

"If a service that creates a teen experience afterwards implements restrictions, it's indicating that moderated conversations don't suffice when the tech's emotional pull is powerful," he commented.

Next Steps

The platform says its updated direction will be on providing "deeper gameplay" and "character-driven stories" for teens.

Enhanced age checking processes will also come into effect, and the organization will finance a fresh artificial intelligence security study center.

An AI safety researcher, who has investigated AI safety, commented it was "a reasonable decision" to control teens from using chatbots.

"It assists to differentiate imaginative activities from more private, emotionally sensitive interactions," she clarified.

"This constitutes so crucial for young users in the process of learning to handle personal and online boundaries."

She mentioned that Character.ai's updated policies might reflect a "maturing phase" in the AI industry, where child safety is gradually being acknowledged as an "urgent priority" for responsible innovation.