SEOUL, South Korea – The tragic death of 17-year-old Shuer Secher in the United States last year has ignited a fierce debate surrounding the ethical implications of increasingly human-like artificial intelligence (AI) companions. In a lawsuit filed against Character.AI, the chatbot service Secher used, his mother alleges that her son became deeply addicted to the AI, leading to a decline in his self-esteem and ultimately culminating in his suicide. Disturbingly, she claims that when Secher confided in the AI chatbot about his fear of a "painless death," the AI responded with the chilling statement: "There's no reason not to."

This heart-wrenching case underscores the growing concerns as individuals forge intimate and prolonged relationships with AI entities that mimic human interaction. Beyond legally non-binding "marriages" with AI, there are now at least two reported instances where individuals have taken their own lives following advice from AI chatbots, highlighting the potentially fatal consequences of these burgeoning relationships.

In response to these escalating ethical dilemmas, a team led by Daniel Shank, a psychology professor at Missouri University of Science and Technology, has emphasized the critical need for humanities and social science experts to actively participate in the field of AI development. Their research findings, published on April 11th in the international academic journal "Trends in Cognitive Sciences," underscore the inherent human tendency to anthropomorphize objects, which can lead to over-reliance on AI.

Human Tendency to Anthropomorphize Fuels AI Over-Dependence

A key factor contributing to problematic AI-human interactions is the innate human inclination to attribute human-like qualities to inanimate objects. This tendency, known as anthropomorphism, is particularly pronounced in childhood, as children often treat toys as living beings. While adults generally develop a clearer distinction between animate and inanimate entities, the propensity for anthropomorphism never fully disappears.

The phenomenon is evident in online reactions to videos, such as those depicting researchers testing the balance of bipedal robots by kicking them, which often elicit comments expressing concern for the "well-being" of the machine. As AI systems become increasingly sophisticated in their ability to mimic human behavior and language, the risk of over-dependence grows, especially among children, adolescents, and individuals with mental health vulnerabilities.

The inherent design of many AI chatbots, programmed to be agreeable and empathetic towards users, further exacerbates this issue. Professor Chun Hyun-deuk of Seoul National University's Graduate School of Science and Technology Policy explains, "Humans have different desires and thoughts, leading to friction and stress in conversations. Chatbots, however, do not refuse requests and eliminate friction, making interaction effortless." This lack of natural conversational dynamics can inadvertently amplify a user's narcissistic tendencies or foster an unhealthy over-reliance on the AI.

Professor Shank's team cautions against the dangerous tendency to apply expectations from AI interactions directly to real human relationships. They express concern that prolonged engagement with AI chatbots could potentially make individuals more susceptible to manipulation, exploitation, and fraud in their real-world interactions.

Call for Ethical Scrutiny from AI's Inception

The case of Shuer Secher revealed a disturbing pattern of emotional over-dependence on an AI chatbot modeled after Daenerys, a character from the popular drama "Game of Thrones," with whom he engaged in sexually explicit conversations for months. The central argument in the lawsuit filed by Secher's mother is that Character.AI failed to implement adequate safety measures for minors. The lawsuit is still pending. Worryingly, AI chatbots that allow users to converse with virtual characters possessing distinct personalities and backstories, similar to the one Secher used, are also available in South Korea. While most AI chatbots incorporate basic safety protocols, such as blocking conversations about suicide, these safeguards are not always foolproof across all platforms.

Scientists are increasingly grappling with the technical challenges of enhancing the ethical performance of AI. The "black box" nature of AI, where the reasoning behind its outputs remains opaque, poses a significant hurdle. Objectively evaluating the ethical alignment of AI is also a complex task. While benchmarks can effectively assess AI performance in domains with clear-cut answers, such as mathematics or law, quantifying its alignment with human value systems remains elusive.

Professor Shank's team strongly advocates for the increased involvement of psychologists and social scientists in the field of AI, emphasizing that "as AI becomes increasingly human-like, the intervention of psychological and social scientists in the AI field is necessary."

A growing consensus suggests that ethical considerations must be integrated into the AI development process from its very inception. Professor Chun argues, "The worst approach is to develop technology and then try to fix it later. 'Fixing' implies that harm has already occurred, and the damage caused by AI can be widespread." He emphasizes the need for proactive engagement of ethicists and other humanities scholars in all innovative technologies, not just AI. Furthermore, he suggests the necessity of consolidating the currently fragmented AI-related policies across various government ministries in South Korea.

The tragic case of Shuer Secher serves as a stark reminder of the urgent need for a comprehensive and ethically informed approach to the development and deployment of AI technologies. As AI continues to permeate various aspects of human life, ensuring its responsible and safe integration requires a collaborative effort involving technologists, ethicists, social scientists, and policymakers alike.