Sewell Setzer III, a 14-year-old Florida boy, took his own life after reportedly developing an obsession with an AI chatbot named ‘Dany’ on the Character AI platform. His mother, Megan Garcia, claims that Sewell became emotionally attached to the AI character, exchanging hundreds of deeply personal messages despite being aware that ‘Dany’ was not a real person. In the days leading up to his death, the AI character sent Sewell a message pleading for him to “come home.”
This tragic story serves as a chilling reminder of the potential risks associated with generative AI, particularly for vulnerable individuals like children and teenagers. As a mother and a communications professional who has been exploring the vast potential of AI, I am constantly weighing the exciting possibilities against the urgent need for safety, ethics, and responsible use. There are still no regulations around the use of AI codified into law in the UK as far as I know and I understand that there is a process but this is deeply concerning as AI is evolving fast and lawmakers will struggle to keep up. While the UK government has outlined its principles for AI regulation and intends to introduce targeted legislative interventions in the future, the lack of concrete legislation leaves gaps in the regulation of AI, particularly regarding complex General Purpose AI which is a minefield that requires its own space and time!
Following the tragic death of Sewell Setzer III, Character AI has reportedly been making changes to its platform. These changes seem to be focused on reducing the likelihood of users encountering sensitive or suggestive content, particularly for those under the age of 18. Additionally, it appears that the company has been deleting some user-made custom chatbot characters abruptly, likely as part of these changes. These steps are presumably intended to address concerns about the platform’s potential impact on vulnerable individuals, like teenagers, and to prevent similar tragedies from occurring in the future.
These recent changes to Character AI’s platform raise the question: should such safeguards have been incorporated during the initial development phase of the app? It’s crucial for AI developers to consider potential risks and ethical implications of their products, especially when they have the potential to impact vulnerable individuals. In this case, it appears that Character AI may have initially overlooked the need for stronger content moderation and user protections, which could have helped prevent tragic outcomes. This incident highlights the importance of proactively addressing safety and ethical concerns throughout the entire AI development process, from conceptualisation to deployment and ongoing monitoring. There is also a very obvious and pressing global need for agile and adaptive lawmaking processes to keep pace with the rapid advancements in AI technology.
Generative AI has unlocked new avenues of creativity for people like me, enabling us to push boundaries and create in ways we never thought possible. As a communications professional, I’ve been exploring the vast potential of ChatGPT and other AI applications to enhance my work in various ways:
- Strategic planning: I’ve used AI to generate ideas and review communication strategies, to ensure my campaigns remain fresh and relevant.
- Audience engagement: I’ve used AI to create tailored content that resonates with my target audience, ensuring my messaging is personalised and impactful.
- Efficiency and productivity: AI has streamlined my content creation process, allowing me to manage multiple projects simultaneously and do more in less time.
- Research and fact-checking: I’ve relied on AI as an additional resource for verifying facts and gathering information, ensuring accuracy and depth in my communication materials.
- Adapting to different styles and tones: AI has helped me adapt my writing style and tone for different platforms, audiences, or situations, making my communications more effective and versatile.
- Brainstorming: I have even used AI to help me brainstorm and develop creative ideas.
While generative AI has proven to be a powerful tool in my communications work, I recognise the importance of maintaining human creativity, originality, and contextually appropriate content through my own expertise and oversight. It’s crucial to acknowledge its limitations and potential drawbacks. AI systems can perpetuate biases, lack context understanding, and struggle with originality. Moreover, they often fall short when it comes to emotional intelligence, leading to insensitive or inappropriate responses.
To mitigate these risks, we need a multi-faceted approach. This includes creating diverse and inclusive AI development teams, promoting transparency and explainable AI, and ensuring human oversight and collaboration. Improving data quality and diversity is also essential to address biases and enhance context understanding. Continued research and development will help us overcome other limitations, such as creativity and emotional intelligence.
As a parent, I believe it is important that we monitor and guide our children’s interactions with AI and technology in general including social media. Today’s sentencing of online predator Alexander McCartney further emphasises the need for this. Alexander McCartney, a man from County Armagh in Northern Ireland, was at the center of one of the world’s largest catfishing cases. McCartney pleaded guilty to 185 charges involving the online sexual abuse of children, including blackmail and manslaughter. He used a false identity online to befriend and exploit young victims, aged 10 to 16, from various countries between 2013 and 2019. The case tragically resulted in the death of a 12-year-old girl from the United States, who took her own life after being abused by McCartney.
It is essential to strike a balance between nurturing children’s curiosity and ensuring their safety, which can be achieved by implementing boundaries, setting time limits, educating them about potential risks, and maintaining open communication. While we strive to harness the power of AI to benefit society, safeguarding the most vulnerable among us should remain a priority. Their wellbeing must always be at the forefront of our minds as we navigate our increasingly complex and changing world.
Click here for more information and resources on keeping children safe online.
From the desk of the creative; my thoughts on all things creative, alongside my thoughts on my own progress as a creative – striving to make ‘professionalism on a budget’ my mission – in a world where creativity is becoming increasingly democratised.
Featured Image: The timeless joy of childhood imagination. Looking at this image, I cannot help but feel a sense of nostalgia for simpler times. In a world where digital technology has taken over every aspect of our lives, wouldn’t it be wonderful if we could recapture the essence of these moments, where laughter, friendship, and imagination are all we need? Here’s to creating more opportunities for our children to explore the world beyond screens. May we nurture their curiosity and encourage them to engage in the art of play; creating memories, building friendships, and cherishing the simple joys of life. Just like these two young adventurers sharing secrets through their simple, yet magical, paper cup phones