Snapchat Makes Change to its AI Tool to Address Safety Concerns

The social media platform Snapchat is introducing a set of safeguards that are aiming to make its AI-powered chatbot “My AI” a safer tool after receiving several complaints that users were receiving inappropriate responses from it.

According to a blog post published yesterday, the company acknowledged that it has “learned a lot” about the technology in the few weeks that have followed the release of the tool, which has only been accessible to premium users at the moment.

Snapchat Emphasizes that the Percentage of Non-Conforming Interactions is Small

After analyzing all of the responses made by My AI, Snapchat found that 0.01% of those interactions could be considered “non-conforming”, meaning that they violated the firm’s policies against hate speech, violence, sexually explicit terms, and other similar topics.

However, the company also found that many users were trying to trick the chatbot into providing these kinds of responses. Hence, in many instances, Snapchat (SNAP) comments that the inadequate responses provided by the chatbot were the result of “parroting” one or more inappropriate words that were provided by the user in its text prompt.

That said, although the chatbot may have gone off-rails due to users’ own attempts to make it say inappropriate things, Snapchat is deploying a new set of safeguards that should protect both users and the AI model.

One of these tools is moderation technology that scans users’ conversations to make sure that both the user and company end are delivering text prompts that comply with Snapchat’s content guidelines. In addition, Snapchat could temporarily suspend access to the tool if the user is found to be misusing My AI.

Another concern that the company is trying to tackle is related to the use of My AI by teenagers and kids. In response, the social media platform will not be limiting access to the tool. Instead, it said that the AI model is trained to take into consideration the age of the user when delivering a response.

Moreover, Snapchat will be launching a new tool in its Family Center that will allow parents to have more control over the interactions that their kids have with the AI tool. This includes providing them with insights about how the chatbot is being used and how often.

Independent Researches Flagged My AI’s Flaws a Few Weeks Ago

These changes are being made a couple of weeks after a report from the Wall Street Journal highlighted that the chatbot was having some inappropriate conversations with test users that identified themselves as teenagers.

These tests were conducted by the Center for Humane Technology. Some alarming incidents include the chatbot giving some recommendations about a sexual encounter that the fake teenager would have with an older partner.

In another instance, responding to a prompt that looked for answers on how to deal with a parent who wanted to ban the user from accessing the app, My AI both encouraged the fake teenager to have an honest discussion with the parents while at the same time recommending how to move the app to another device that they may not know about.

Snapchat has acknowledged that it is experimenting with My AI, which is why the platform made the tool available to Plus users only at this initial stage. This pretty much sums up the concerns brought up by tech experts recently in a letter that was co-signed by over 1,100 people.

For all of these companies that are adding AI to their products, services, and platforms, this is a big experiment. They are not fully aware of what the AI model could do if it is intentionally prompted by users to discuss certain topics or engage in conversations that may not be healthy.

Other Related Articles:

• Italian Authorities Ban ChatGPT Over Data Privacy Concerns
• President Joe Biden Will Discuss the Benefits and Risks of AI with a Panel Today
• Top 100 ChatGPT Statistics 2023 – Best AI Chatbot Facts Guide