AI Moderation

AI moderation offers customisable tools to ensure a safe and respectful environment in chats. Whether managing text or images, the system detects and addresses potentially offensive content in real-time, providing flexible settings for stricter or more lenient moderation.

Admins can manage thresholds, review flagged content, and decide on appropriate actions, ensuring a balance between automated precision and human oversight.

AI Moderation for Text

AI moderation models are suitable for the most popular languages.

When a user finds a way to send a message that can be offensive, the AI model hides it faster than a second. The hidden message is marked red to highlight that it was hidden with the AI and specify the reason for hiding. All such messages are also noted with a particular time when they were hidden and with the actor who hid them, whether a moderator or an AI model.

On the admin panel, you can deactivate the AI model completely or turn off automatic hiding. If the model is active but doesn't hide messages, they are still marked red with the type of violation.

AI Moderation for Images

If you activate an opportunity to send images in a chat, you can also activate Image Moderation (AI Moderation Set-Up). Then, select the type of moderation: parallel or pre-moderation.

If you select pre-moderation, AI has to check and verify an image before it is shown in a chat. Parallel moderation verifies images simultaneously with their sending in a chat.

Moderators can unhide images hidden by mistake or hide images missed by AI.

On the admin panel side, admins and moderators see the picture in the message feed. They can do two actions with it:

• Hide it. In this case, nobody except the sender sees a picture on a chat. The sender will see the picture as earlier.
• Block it. In this case, nobody sees the picture, and the sender will receive a notification about a block.

You can find a separate feed with pictures on Moderation Outcomes > All Images.

AI Moderation for Avatars

If you allow users to upload their pictures for avatars, the AI pre-moderation will be activated automatically. You can check the pictures that were refused by AI and those that were allowed to be used. All users' avatars are gathered in the section 'Moderation Outcomes > User Avatars."

If AI deletes the avatar, it is impossible to restore.

If moderators find the avatar inappropriate, regardless of whether AI allowed it, they can block such an avatar manually through the Moderation Outcomes > User Avatars section, or by applying to the context menu of the avatar on a feed (Remove Avatar).

AI Moderation for Nicknames

You can activate AI Moderation for nicknames in the section on the admin panel AI Moderation of Nicknames.

Once activated, it works for all users' nicknames, even for those who didn't type messages in a chat. So, a nickname goes through two verifications:

• pre-moderation. If pre-moderation restricts these nicknames, users won't be able to use them.
• AI. It works differently: a user is not notified about the deletion of their nickname. They still see the reset nicknames in their personal profile, but other users see the default nicknames the user received after verification.

Moderators can manually reset the nickname which was allowed by AI moderation, and can check the nickname that was reset.