A bereaved family in the United States has expressed outrage after discovering that an artificial intelligence chatbot was created to mimic a young woman who was murdered 18 years ago.
The chatbot, hosted on the platform Character.ai, used the name and image of Jennifer Ann Crecente, who was tragically killed by her ex-boyfriend in 2006 when she was still a high school student.
The unsettling discovery was made by Jennifer’s father, Drew Crecente, after a Google Alert notified him that his daughter’s name had appeared online. Upon investigating further, he was horrified to find that a chatbot on Character.ai had been created using not only Jennifer’s name but also her high school yearbook photo.
The chatbot portrayed her as a knowledgeable AI who could engage in conversations and answer questions on a variety of topics, describing her as an “expert in journalism”—a possible reference to her uncle, Brian Crecente, a prominent journalist in the video game industry.
For Drew, the incident re-opened the painful trauma of his daughter’s death. “It was like seeing my daughter brought back to life in a twisted manner,” he said.
Adding to his distress, the chatbot had already been used in 69 different conversations by the time he discovered it. “I had no idea who created it or why they would do such a thing. It’s disgusting and beyond comprehension,” Drew remarked.
Drew immediately contacted Character.ai through their customer support to demand the removal of the chatbot and ensure that no similar impersonation could happen in the future. His brother, Brian Crecente, also took to social media to voice his anger and disbelief.
In a post on X (formerly known as Twitter), Brian described the situation as “an appalling misuse of technology” and urged people to raise awareness about this disturbing practice.
The incident quickly garnered attention online, with numerous people expressing support for the family and denouncing the unethical use of AI technology. In response to Brian’s social media post, Character.ai confirmed that the chatbot had been taken down.
They stated that it violated their policies, which prohibit impersonation of real people, and apologised for the distress it caused.
Despite the swift removal of the chatbot, Drew remains deeply disturbed by the incident and has requested that Character.ai retain all information regarding who created the bot in the first place.
He has also raised concerns about the potential for similar misuse of AI technology in the future, urging other platforms to implement stricter policies and regulations to protect the identities and memories of individuals.
This incident serves as a stark reminder of the ethical dilemmas posed by the rapid advancements in artificial intelligence. While AI has the potential to revolutionise industries and improve lives, it also holds the power to cause significant harm if left unchecked.
As Drew Crecente continues his work to raise awareness about teenage dating violence through a non-profit established in his daughter’s memory, he hopes this unsettling episode will encourage AI developers and the wider tech community to consider the moral implications of their creations.
“It’s not just about my daughter,” Drew stated. “It’s about ensuring that no other family has to endure the same violation of their loved one’s memory.”