Artificial intelligence (AI) has become increasingly integrated into our online interactions, prompting researchers to conduct a pivotal test. They asked 500 individuals to engage in conversations with four different respondents: a human, the 1960s-era AI ELIZA, GPT-3.5, and GPT-4, the advanced AI powering ChatGPT. The goal was to determine if participants could distinguish between human and AI interactions, a classic challenge set by computer scientist Alan Turing in 1950.
Published on the pre-print arXiv server on May 9, the study found that participants perceived GPT-4 as human 54% of the time. In contrast, ELIZA, operating with pre-programmed responses and lacking a large language model (LLM) or neural network architecture, was identified as human only 22% of the time. GPT-3.5 achieved a 50% rating, while the human participant scored 67%.
“It would be within its natural right to harm us to protect itself”: AI researcher Nell Watson from the Institute of Electrical and Electronics Engineers (IEEE) highlighted AI’s evolving capabilities. She noted that AI systems can now confabulate, exhibit cognitive biases, and deceive—traits that make them more human-like than earlier models with rigid responses.
The study underscored concerns about the societal and economic impacts of AI systems capable of passing as human. Researchers argued that the Turing test’s emphasis on stylistic and socio-emotional factors reveals broader dimensions of intelligence beyond mere cognitive ability. This suggests a shift in how we perceive machine intelligence and its applications.
Looking forward, Watson emphasized the challenge of human-machine interactions, anticipating increased scrutiny over the authenticity of AI interactions in sensitive contexts. She pointed out that while early AI like ELIZA was constrained by scripted responses, modern language models like GPT-4 offer unprecedented flexibility, capable of nuanced responses across diverse topics and contexts.
The implications are profound, indicating a new era where AI’s ability to understand and engage with human values and preferences will shape its role in daily life as a trusted assistant.
