San Mateo. Roblox is an online gaming platform on which users can create their own computer games and play games created by others. The platform was developed by David Baszucki and Erik Cassel and released in February 2006. A central element of the platform is its chat functions, through which users and players communicate with one another.
Those chats are already monitored with the help of artificial intelligence to prevent the exchange of offensive or prohibited content. According to the company, the protection of younger users is a high priority. In future, however, users’ messages will be rewritten by AI in real time. The benchmark for such changes will be what the platform itself considers more appropriate according to its own criteria.
Inevitably, questions arise about where the limits lie and whether the boundary towards far-reaching censorship through linguistic control has already been crossed.
In practice, the chat partner will no longer necessarily receive what the author actually wrote, but rather what the AI considers correct and compliant with the rules. At present, supposedly or genuinely prohibited words are masked with ‘#’ symbols. That system is now being replaced by a new mechanism that silently reformulates so-called forbidden expressions into alternative wording that Roblox describes as ‘more respectful language that remains closer to the user’s original intent’.
The platform itself provides an example. If someone types ‘Hurry TF up!’, the recipient will see the message ‘Hurry up!’. The phrase ‘Hurry TF up!’ combines ‘Hurry up’ with the vulgar abbreviation ‘TF’, short for ‘the fuck’. In everyday usage it expresses strong impatience when someone is moving too slowly.
Erasing vulgar language
Teenagers often use such language, which may sound vulgar to some, as a form of sociolect or simply to appear cool. The coarse expression is rarely the true content of the message. Roblox thus assumes the role of a kind of moral authority that alters language and thereby restricts the individual’s ability to express themselves authentically.
The company states that all participants in a chat will be notified whenever such a censorship measure occurs. Even so, the person who typed the original message has no possibility of preventing the replacement of their own words before the message is sent. As a result, the recipient may receive something different from what was actually written, while the system’s wording is attributed to the sender.
The definition of ‘forbidden expressions’ extends beyond simple obscenities. It also includes ‘misspellings, special characters or other methods used to evade the detection of obscenities’. In other words, the AI is also tasked with recognising deliberate attempts to circumvent the filter and rewriting those as well. Abbreviations, altered spellings or other communicative tricks designed to bypass censorship will therefore also be reformulated.
The system grows ever more refined
As part of the same initiative, Roblox has expanded its text-filtering system in order to ‘detect more variations of language that violate its community standards’. The net designed to capture prohibited language will therefore become wider and increasingly tight. Synonyms or slightly altered wording may eventually still be recognised and altered in accordance with Roblox’s rules.
Language risks becoming poorer when the forceful vocabulary of human speakers is reduced to a simplified form imposed by AI and according to rules defined unilaterally by platform operators.
Roblox has thus developed a system that goes beyond simply blocking language. It replaces it according to its own criteria. With an AI system that learns and adapts, this also means that the language model may continue to modify its rules depending on how it is trained.
The message that leaves the keyboard is no longer necessarily the message that arrives. The recipient reads words the sender never chose. Those words are nonetheless attributed to the sender, accompanied by a notice that the original wording was deemed unacceptable. In other words, the platform decides what the sender has said.
Currently justified by youth protection
For now, the feature is activated in chats between verified users of similar age groups. Roblox has also introduced a group called ‘Trusted Connections’, a function available to users aged 13 and above who have completed an age verification and are connected with people they know. All chat control and manipulation introduced here is justified as part of youth protection.
Last month, Roblox began requiring age verification for certain chat functions. Once verification is completed, users may communicate with players in neighbouring age groups. For example, the group aged 9–12 can interact with users aged 13–15.
Minecraft, another gaming platform, also filters swear words. There, however, the approach is more transparent in how the filter operates. Marked words are replaced by symbols or the message is blocked entirely. The words typed by the sender do not appear as different words under the sender’s name. That, too, is a form of manipulation, but at least it openly presents itself as such.
The system now introduced by Roblox works differently. It effectively puts words into the sender’s mouth. Users’ statements are rewritten in real time without their consent. That process does not occur on the basis of legal regulation or broadly accepted ethical reflection, but according to definitions of ‘acceptable language’ formulated by a private company.
Artificial intelligence will further expand and refine this process in order to capture ever more linguistic variations the platform considers undesirable. For now, the focus lies on swear words. What follows next? Self-defined racism? Post-colonial terminology? Gender language? In principle, almost anything could be included.
Risky tools in the name of child protection
It can hardly be repeated often enough that many tools ultimately designed for surveillance and control begin with a seemingly laudable objective: protecting children. An electronic governess monitors the language of young users and responds by correcting it directly. As a parent, the first question is where exactly that correction is meant to lead.
No swear words? Half of youth literature would have to be censored. No sexism? Of course anything that prevents sexual abuse deserves support. But what about whispered conversations among teenagers who are trying to uncover life’s great mysteries? Perhaps such conversations do not belong online at all and should remain exchanges face to face. Do we really need artificial intelligence to tell us that? Young people do not build resilience and self-confidence by being wrapped in cotton wool.
In the adult world the implications would be even more serious. One must assume that everything currently being tested in the name of child protection could be extended to adult communication at the press of a button. The justifications would merely change – protection against ‘hate speech’, safeguarding democracy or one of the other buzzwords frequently invoked to justify new forms of intervention. Once a tool exists, it will be used.
Would we want a WhatsApp chat moderated by AI in real time? If text can be altered, spoken language can be altered as well. When speaking on the telephone, did one really hear what the other person said? Or did an AI system smooth or adjust the statement before it reached the listener?
The path chosen by Roblox is more than merely questionable. Even in the context of youth protection it is far from harmless. Communication in which the message changes between sender and recipient is no longer truly communication. At best it is moderation – at worst manipulation.
