AI Moderation

AI moderation offers customizable tools to ensure a safe and respectful environment in chats. Whether managing text or images, the system detects and addresses potentially offensive content in real-time, providing flexible settings for stricter or more lenient moderation.

Admins can manage thresholds, review flagged content, and decide on appropriate actions, ensuring a balance between automated precision and human oversight.

AI Moderation for Text

AI moderation models are suitable for the most popular languages.

When a user finds a way to send a message that can be offensive, the AI model hides it faster than a second. The hidden message is marked red to highlight that it was hidden with the AI and specify the reason for hiding. All such messages are also noted with a particular time when they were hidden and with the actor who hid them, whether a moderator or AI model.

You can set thresholds for AI—for a stricter and softer estimation of messages. To do so, open the section Moderation> AI Moderation > Threshold tab. Change the levels for needed categories: the higher level for the category, the lighter moderation, and the opposite: the lower level, the stricter moderation approach.

You can deactivate the AI model completely or turn automatic hiding off on the admin panel. If the model is active but doesn't hide messages, they are still marked red with the type of violation.

AI Moderation for Images

If you activate an opportunity to send images in a chat, you can also activate Image Moderation (Moderation > AI Moderation). Then, select the type of moderation: parallel or pre-moderation.

If you select pre-moderation, AI has to check and verify an image before it is shown in a chat. Parallel moderation verifies images simultaneously with their sending in a chat.

Moderators can unhide images hidden by mistake, or hide images that were missed by AI.

On the admin panel side, admins and moderators see the picture on the message feed. They can do two actions with it:

• Hide it. In this case, nobody except the sender sees a picture on a chat. The sender will see the picture as earlier.
• Block it. In this case, nobody sees the picture, and the sender will receive a notification about a block.

You can find a separate feed with pictures on Moderation > Images.

AI Moderation for Avatars

If you allow users to upload their own pictures for avatars, the AI pre-moderation will be activated automatically. You can check its settings in the 'Moderation > AI moderation' section. Also, you can check the pictures refused by AI and the pictures allowed to be used. All users' avatars are gathered in the section 'Moderation > Avatars."