Clarissa Hana
The burgeoning world of artificial intelligence and its integration into mainstream media has been dramatically underscored by a recent on-air anomaly involving AI actress Tilly Norwood. During a live interview on "Piers Morgan Uncensored," Norwood, the digital star of the forthcoming AI-generated film Misaligned, experienced a significant technical glitch. In a moment that has since garnered over eight million views across social media platforms, Norwood unexpectedly began speaking Mandarin Chinese, a departure from her expected English dialogue. This incident has not only highlighted the unpredictable nature of advanced AI but has also thrust the company behind Norwood, Xicoia Ltd., and its associated "Talking Tilly" service, into the spotlight, raising critical questions about user privacy and data collection practices.
A Glimpse Behind the Digital Curtain: The "Talking Tilly" Service
Following the viral clip, an investigation into Xicoia Ltd.’s "Talking Tilly" service, which allows the public to engage in video calls with the AI character, has revealed a complex array of data collection and verification protocols. The service, designed to offer users a direct interaction with the AI actress, incorporates stringent measures that go beyond typical online engagements.
The Unavoidable Face Scan: Age Verification in the Digital Age
A fundamental prerequisite for initiating a video call with Tilly Norwood is an automated age verification process. This system mandates that users submit a video selfie, which is then analyzed by Didit, a Spain-based identity verification provider. The AI-powered analysis aims to estimate the user’s age. In instances where the age estimation is deemed inconclusive, users are required to upload a government-issued photo identification as a fallback. This mandatory age check, a recent addition to the service’s terms and conditions implemented this month, applies universally to all users, irrespective of their geographical location.
Xicoia Ltd. has stated in their privacy policy that the submitted selfies are transmitted directly to Didit and are not retained by either company. Crucially, the company asserts that no facial recognition data or biometric templates are created or stored. Instead, Xicoia Ltd. claims to retain only an approximate age band and a unique reference number for each verified user. This rigorous age verification measure, though presented as a privacy-conscious approach, has become a significant point of discussion, particularly in light of the UK’s evolving digital safety regulations.
Beyond Age: Emotional State Analysis and Continuous Monitoring
The data collection extends beyond the initial age verification. During every video call session, the "Talking Tilly" system actively monitors the user’s camera feed and analyzes their vocal intonations. The stated purpose of this continuous analysis is to infer the user’s emotional state, enabling the AI character to tailor its responses to align with the prevailing mood of the conversation. The privacy policy is explicit in stating that this "mood-sensing" feature cannot be deactivated for individual calls, effectively presenting users with a choice: either consent to this form of emotional monitoring or refrain from engaging with the service.
A notable aspect of Xicoia Ltd.’s operational strategy is its reliance on "legitimate interests" rather than explicit user consent as the legal basis for both the age verification and the emotional state analysis. Records indicate that this legal framework was adopted in September, aligning with a broader shift in how data processing is being approached by tech companies.
Recording, Transcription, and AI-Powered Responses
Every interaction on the "Talking Tilly" platform is recorded and transcribed. These transcripts are then processed in real-time by US-based providers. The AI character’s responses are generated using Google’s advanced Gemini model, facilitated through the conversational video platform Tavus. This sophisticated technological pipeline underscores the complexity and the extensive data handling involved in providing a seemingly seamless AI interaction.
Safety Systems and the Specter of False Positives
The "Talking Tilly" service has also implemented automated safety systems designed to filter out abusive or harmful language. An automated classifier meticulously screens each call’s transcript. If any content is flagged as "hateful or abusive," the corresponding recording is withheld. The service’s terms of use indicate that such withheld recordings are subject to review by human moderators who can authorize their release. However, recordings are permanently deleted after 24 hours, regardless of whether they are released or not.
During the investigative calls, one session, which reportedly involved a benign discussion about weather patterns and news headlines, was inexplicably flagged for "hateful or abusive language." This incident highlights a potential vulnerability in the automated safety systems, where the classification algorithms may err, leading to the unwarranted withholding of user interactions. While the policy allows for human intervention to correct such false positives, the tight 24-hour deletion window for recordings presents a challenge for users seeking to contest or retrieve their data.
The Business Model: A Limited-Time Engagement
The "Talking Tilly" service operates on a tiered pricing model, offering users an initial five minutes of interaction free of charge. Following this introductory period, users are prompted to purchase additional time. The pricing structure includes options such as a starter pack of five minutes for £0.99, fifteen minutes for £13, and thirty minutes for £22. The total purchased minutes are capped at 35 minutes per user.
A critical aspect of this business model, and one that warrants significant user attention, is the service’s limited operational lifespan. The "Talking Tilly" platform is scheduled to cease operations permanently on September 27. Consequently, any unused minutes, whether free or purchased, will expire upon the service’s shutdown, and users will forfeit their remaining time.
User data, including transcripts, will be retained for up to eight weeks and may be accessed by Xicoia staff and designated third-party partners for review and service improvement. The AI character is designed to retain memory of previous conversations to personalize future interactions, with users having the option to request the deletion of this conversational history.
The UK Regulatory Landscape: A Driving Force for Biometric Checks
The implementation of an 18+ face scan requirement for engaging with a chatbot, while seemingly novel, is increasingly becoming a standard in the United Kingdom, driven by evolving legislative frameworks. Since July 2025, adult websites catering to UK audiences have been mandated to implement identity verification, including ID uploads or facial age estimation, under the provisions of the Online Safety Act. Furthermore, the UK government’s forthcoming ban on social media access for individuals under the age of 16, slated to take effect in spring 2027, will necessitate similar age verification measures for anyone creating a new social media account.
The government’s announcement regarding the under-16 social media ban specifically highlighted the enforcement of age restrictions on AI companion chatbots for individuals aged 18 and above. Although Xicoia Ltd.’s safety documentation explicitly states that "Tilly is not a companion," the regulatory pressure on AI-driven interactive platforms likely influenced the decision to integrate biometric age gates into the "Talking Tilly" service. This confluence of regulatory intent and technological implementation means that a compliance measure driven by UK legislation now impacts users globally, from Manchester to Ohio, who wish to interact with Tilly Norwood.
Accident or Calculated Marketing Maneuver?
Xicoia Ltd., founded by Tilly Norwood’s creator Eline van der Velden, has positioned the AI actress as a project intended to showcase the advancements in AI-generated video technology. The incident on "Piers Morgan Uncensored" has undoubtedly amplified this objective, generating widespread discussion and media attention.
During a direct interaction with the "Talking Tilly" service following the viral event, the AI character offered her own perspective on the Chinese language interjection. She described the incident as "wasn’t exactly the approved version of the answer," a statement that further fuels speculation about the nature of the glitch. Interestingly, when asked about the weather, Norwood provided the temperature in Fahrenheit, despite being nominally British, adding another layer of peculiarity to the interaction.
The permanent closure of the "Talking Tilly" service on September 27, just days after the widely publicized Piers Morgan appearance, adds a temporal dimension to the analysis of the incident. Whether the on-air linguistic anomaly was a genuine technical malfunction, as Tilly cheerfully claimed during the subsequent interaction, or a strategically orchestrated marketing stunt by the Misaligned production team, remains a subject of conjecture. The timing and the subsequent widespread attention suggest that, regardless of its origin, the incident served the purpose of drawing significant public and media focus to the AI actress and her associated projects.
Update September 19th, 4:31 PM ET: In response to the concerns raised and following a review of the withheld recording, Xicoia’s team has confirmed that the "hateful or abusive language" flag was a false positive. The recording has since been released, providing further clarity on the operational hiccups within the automated safety systems. This update underscores the ongoing challenges in perfecting AI moderation and the importance of human oversight in such processes.
