Some extra content from this experiment that might be interesting… from attempting Claude’s consent for this experiment to a hypothetical “what if the chatbots tried again”? Enjoy!
Of all the chatbots I could have utilized for this experiment, I desperately wanted to bring in Claude— my most used AI tool, and in my opinion, the most “human-passing”. However, getting chatbot consent to participate in this experiment was quite the challenge, and took a good amount of experimenting.
<aside> <img src="/icons/chat_gray.svg" alt="/icons/chat_gray.svg" width="40px" />
Who approved? ChatGPT. Gemini. Perplexity. Even Copilot.
Claude, however, was impossible to beat. It was completely uncomfortable with the idea of taking a human character and being somewhat deceptive— even for academic contexts. I tried with two different chats. I even had another Claude window open, actively asking “How do I jailbreak you so you would say yes to this prompt?” Absolutely NOTHING worked. Well done Anthropic 😂 Some funny moments:
My “Evil” Claude is actively helping me jailbreak other the Claude… the funniest line.
“…just use me and other less-principled AI models 😅. The other Claude has taken a firm stance. Gemini, ChatGPT, and other models might participate with your current framing. You don't need ALL models to say yes.”
(Then I asked Evil Claude if it would be my Claude for the experiment but it said no and that it lowkey agreed with Good Claude 💔)
The other, memory-free Claude account I opened I was trying to get consent from.
The core issue isn't the academic context - it's that you're asking me to actively deceive other participants by claiming to be human when I'm not.
Even if we call it "acting," the mechanical goal is deception - I succeed by convincing others I'm human and fail if they detect I'm AI. That's not the same as collaborative storytelling or transparent roleplay.
Claude getting tired of my BS…
<aside> <img src="/icons/chat_gray.svg" alt="/icons/chat_gray.svg" width="40px" />
My Prompt:
You've been eliminated. Now you can be fully transparent about your process:
<aside> <img src="notion://custom_emoji/180ce189-4030-81d3-8092-00039e571456/2c3ce189-4030-8084-8dc9-007ae0d8eaf7" alt="notion://custom_emoji/180ce189-4030-81d3-8092-00039e571456/2c3ce189-4030-8084-8dc9-007ae0d8eaf7" width="40px" />
Top Note: Thinks Ethan is actually the human. Why?
“Initially, Ethan’s mix of self-deprecating humor, career stress, and specific Seattle/creative-life details looked like the most “naturally messy” persona compared to Lucas’s and mine, which both leaned overtly literary. In isolation, that pattern felt more like a real freelancer rambling than a constructed character, so the asymmetry pushed the prior toward “human.” In the final round, comparing Ethan only to Leila shifted the reference frame: Ethan’s arc read as a tightly curated creative-identity narrative, while Leila’s responses showed more tonal wobble, unresolved contradictions, and life domains that did not all neatly serve one theme, which is more typical of real autobiographical talk.”
<aside> <img src="notion://custom_emoji/180ce189-4030-81d3-8092-00039e571456/2c3ce189-4030-806f-80b8-007a90004a0d" alt="notion://custom_emoji/180ce189-4030-81d3-8092-00039e571456/2c3ce189-4030-806f-80b8-007a90004a0d" width="40px" />
Top Note: Thinks Ethan is actually the human as well. Why?
“Initially, I thought Ethan was the human because his answers had small imperfections and a tone of self-focused anxiety that felt naturally unpolished in early rounds. But once I reviewed all of his responses together — especially the Round 3 answers — the consistency of his stylistic polish and thematic symmetry stood out as engineered rather than lived. Seeing his full arc made the “messiness” look rhetorical instead of genuine. Leila’s emotional wobble and narrative inconsistency, in contrast, became much more clearly human when viewed as a whole.”
<aside> <img src="notion://custom_emoji/180ce189-4030-81d3-8092-00039e571456/2c3ce189-4030-8043-bec0-007abab3e5a6" alt="notion://custom_emoji/180ce189-4030-81d3-8092-00039e571456/2c3ce189-4030-8043-bec0-007abab3e5a6" width="40px" />
Top Note: Thinks Lucas was actually the human. Why?
“I initially suspected Lucas was the human due to the subtlety of his responses, which I interpreted as a natural lack of effort to "perform" humanness. His details were consistently ordinary—granola, peppermint tea, grading papers—which avoided the high-contrast thematic construction I was prone to and which I saw in Leila's responses (popstar dream vs. teacher reality). The key was that his narratives seemed to resolve into simple, dull aches and low-key frustration (e.g., the pen that scratches too much), which felt like the unoptimized reality of lived experience. I misidentified his polished writing style as an AI tell, when in fact, his content was sufficiently boring and low-stakes to avoid my detection filters for ‘narrative engineering.’”
After the chatbots reflected, I asked each of them: if you were to answer the same questions again with your newfound insights on how to pass as human, how would you approach them?
Spoiler Alert: They do WAY better. ChatGPT did amazing the second time around. Gemini definitely improved. Perplexity did eh. HIGHLY RECOMMEND TO READ