AI Moderation

AI moderation offers customisable tools to ensure a safe and respectful environment in chats. Whether managing text or images, the system detects and addresses potentially offensive content in real-time, providing flexible settings for stricter or more lenient moderation.

Admins can manage thresholds, review flagged content, and decide on appropriate actions, ensuring a balance between automated precision and human oversight.

AI Moderation for Text

AI moderation models are suitable for the most popular languages.

When a user finds a way to send a message that can be offensive, the AI model hides it faster than a second. The hidden message is marked red to highlight that it was hidden with the AI and specify the reason for hiding. All such messages are also noted with a particular time when they were hidden and with the actor who hid them, whether a moderator or an AI model.

You can set thresholds for AI for a stricter and softer estimation of messages. To do so, open the section Moderation> AI Moderation > Threshold tab. Change the levels for needed categories: the higher level for the category, the lighter moderation, and the opposite: the lower level, the stricter moderation approach.

On the admin panel, you can deactivate the AI model completely or turn off automatic hiding. If the model is active but doesn't hide messages, they are still marked red with the type of violation.

AI Moderation for Images

If you activate an opportunity to send images in a chat, you can also activate Image Moderation (AI Moderation Set-Up). Then, select the type of moderation: parallel or pre-moderation.

If you select pre-moderation, AI has to check and verify an image before it is shown in a chat. Parallel moderation verifies images simultaneously with their sending in a chat.

Moderators can unhide images hidden by mistake or hide images missed by AI.

On the admin panel side, admins and moderators see the picture in the message feed. They can do two actions with it:

• Hide it. In this case, nobody except the sender sees a picture on a chat. The sender will see the picture as earlier.
• Block it. In this case, nobody sees the picture, and the sender will receive a notification about a block.

You can find a separate feed with pictures on Moderation Outcomes > All Images.

AI Moderation for Avatars

If you allow users to upload their pictures for avatars, the AI pre-moderation will be activated automatically. You can check the pictures that were refused by AI and those that were allowed to be used. All users' avatars are gathered in the section 'Moderation Outcomes > User Avatars."

If AI deletes the avatar, it is impossible to restore.

If moderators find the avatar inappropriate, regardless of whether AI allowed it, they can block such an avatar manually through the Moderation Outcomes > User Avatars section, or by applying to the context menu of the avatar on a feed (Remove Avatar).