The Cost of Pretending to Be Human

The other day I found myself reading through yet another political thread. Nothing unusual there. It's what I do for fun. Two accounts were going at each other with the kind of confidence that only exists on the internet.

After a few replies, I started looking at the accounts rather than joining the argument.

Neither had a biography. Both used default avatars. Both had a handful of followers, blank timelines and they look like they had sprung into existence a couple of weeks earlier. They held completely opposing views, yet they looked like the same account.

I honestly couldn't tell. Were they two people? Two bots? Two accounts being run out of the same engagement farm because we all know that conflict drives clicks, right?

I've been thinking about this for a while. It's the premise of another book outline I have in the wings, and I've come to think of it as an authenticity problem on social media. The bots themselves aren't really the point anymore. It's what their presence does to trust.

We all know spam has existed since the early internet. Trolls have been around since Usenet. Fake accounts are not new. We have all learned to roll our eyes, block the obvious ones and move on.

What has changed since then is the cost and economics of pretending to be human.

Twenty years ago, manufacturing the appearance of public opinion at scale took people, time and money. Today, software can do much of the heavy lifting. It can generate the profile picture, write the posts, reply to comments, adopt a political identity and keep the account active twenty-four hours a day without ever asking for a coffee break (which I need a lot of).

And don't get me wrong. This is not an argument against anonymity. Whistleblowers, dissidents, marginalized groups and plenty of ordinary people have legitimate reasons for keeping their identity private. I'm perfectly fine not knowing your real name. I'd just like some confidence that there's actually a human on the other side of that avatar.

The Turing Collapse

People talk about the Dead Internet Theory as if one day we'll look around and discover the humans have disappeared. I don't think that's how this happens. The humans will still be here, mostly. We just won't be sure who else is.

This is what I've started calling the Turing Collapse. The original question was whether a machine could convince us it was human. But what happens when machines become convincing enough that we start doubting the humans too? Once synthetic and human behaviour are difficult enough to distinguish, the test doesn't simply flip. It starts to eat its own tail. The machine passes as human, so the human becomes suspect as a machine.

From the platform's point of view, I'm not even sure anything is broken. Engagement is engagement. A comment is a comment. A click is a click. It's all profit. If two bots spend all afternoon arguing with each other and humans pile in to watch, the metrics probably look fantastic.

None of this started with AI. We had click farms, astroturfing and coordinated influence campaigns long before ChatGPT showed up. AI just made the whole bloody thing cheaper, faster and a lot easier to scale.

And there's the ugly incentive. Persuading thousands of people is hard. Making it look like thousands of people already agree with you is getting ridiculously cheap.

Humans have always taken cues from other humans. We call parts of that the bandwagon effect or social proof, but those ideas assume there's actually a crowd. What happens when the crowd is synthetic?

And that's where this gets uglier than simple misinformation. Feed people enough manufactured bullshit and eventually they don't just stop believing the lies. They stop believing anything.

We've all felt this. We've seen the knee-jerk reactions. A legit photo gets an immediate, "That's fake!" or we question it.

I find myself doing it already. Was that viral post coordinated? Are those two political accounts even people? Did that opinion actually become popular, or was it boosted into existence? Eventually even legitimate information starts looking suspicious because it looks an smells the same.

Trying to sort through all of this is bloody exhausting. I'm already checking account ages, follower counts and profiles before I decide whether a reply is worth taking seriously. I'm sure I'm not the only one doing it. And that's exactly what I'm talking about, suspicion is becoming the default.

Proving Humanity Without Surrendering Privacy

So how the hell do we do that without turning the internet into a giant ID checkpoint?

I don't want real-name policies. They solve one problem by creating several others. What I want is a way to prove to a platform that I am one unique human being without handing over my entire identity.

We're starting to see pieces of this already. LinkedIn has optional identity verification using government-issued ID through third-party providers. takes a similar approach. It's a Canadian social platform built on the AT Protocol, and to post, comment or chat you complete a Human Check. Gander gets confirmation that you're a real adult without receiving your identity documents, and you can still use a pseudonym.

There are other experiments happening around the AT Protocol too. has been working on accreditation through and endorsements, looking at ways people can build and carry signals of trust rather than relying on a single platform to declare them legitimate. I've even experimented with an identity labeller myself, trying to figure out whether labels could provide some useful signal about who is behind an account.

None of these is the whole answer, and they're approaching the problem from different directions. But that's actually what interests me. We're starting to experiment with trust as something that can exist outside the platform itself.

That's much closer to what I have in mind. It isn't portable yet, but it separates two things we've tended to lump together online, proving you're human and proving who you are.

Where I'd like to see this go is portability. I shouldn't have to prove myself from scratch on every platform. Different trusted institutions could issue compatible credentials, and the proof could move with me without dragging my identity along with it.

It doesn't need your name, address, birthday or, lord help us, another copy of your passport sitting in a corporate database waiting for the inevitable breach. It just needs to prove you're human.

A social platform could verify that credential without necessarily learning the identity behind it. More importantly, the credential could belong to you rather than the platform. Leave one network and take your proof with you.

And this is where I start thinking about sovereignty. If proof of humanity becomes part of the infrastructure of the internet, I care who controls it, where that infrastructure lives and whether I can take my credential somewhere else and store it in a sovereign location.

If proof of humanity eventually becomes part of participating online, someone has to issue it and set the standards. And whoever controls that infrastructure gets an awful lot of say over whose credentials we trust. I'm hoping the answer is not one central authority, but multiple trusted options spread around, all able to provide a similar proof without owning the whole system.

The last thing I want is another piece of my digital identity locked inside some tech company's ecosystem. We've done enough of that. Give me open standards, multiple trusted issuers and a portable credential that I can use wherever I go.

Tech has spent the better part of two decades engineering the friction out of everything. Fewer clicks, faster sign-ups, easier onboarding. Up until now that's been a good design. The problem is that the same friction we removed for people, disappeared for everything else too.

Turns out making it effortless to create an account also makes it effortless to create ten thousand of them.

I'm thinking some friction was a feature we threw away.

Zero-Trust Social

As I said, I'm already checking profiles before I bother replying to people. Take that behaviour far enough and the direction seems pretty obvious, we'll start moving towards places where we have more confidence about who's on the other side.

People may not leave the internet entirely, but they will leave the open parts of it. They will retreat into authenticated networks, private communities, closed group chats and digital walled gardens where they have some confidence about who is on the other side of the conversation.

Follow that far enough and we haven't really saved the public square. We've broken it into a thousand gated communities where trust has to be established at the door. That's what I mean by Zero-Trust Social.

I'm borrowing the term from cybersecurity. Zero-trust architecture assumes you don't trust a user or device merely because it has made it inside the network. Now apply that same suspicion to social media. If my starting assumption is that every account might be synthetic until I can establish otherwise, we've fundamentally changed what an open social network is.

This is where I think we need to reconsider our obsession with friction. Cryptographic credentials, portable identity and privacy-preserving verification aren't going to make onboarding easier. That's fine. If an extra step gives me some confidence that the people I'm talking to are actually people, I'll take the extra step.

Because the alternative isn't merely more bots. We lose the thing that made social media socially useful in the first place, the assumption that somewhere behind the avatar, username and ridiculous political opinion was another human being. In doing so we've also made it more dangerous.

That's the part I keep coming back to. I'm less worried about AI learning to sound human than I am about humans reaching the point where we assume everyone else is AI.

If that becomes normal, we've built a social internet where nobody trusts the social part.

I don't think the answer is another giant identity system or another company deciding who counts as real. I think it's smaller, distributed signals of trust, credentials, endorsements, labels, human checks, communities vouching for people and people able to carry that trust with them.

We spent twenty years optimizing the internet for scale and easy access.

The next twenty may have to be about making it believable again.

Jason Butterfield is a Canadian sci-fi author, veteran, technology leader, and developer of Byline.pub. His fiction explores identity, power, ethics, and our uneasy relationship with technology. He is the author of Pejorative: Echoes of a Nation, and the forthcoming Simulacrum: Echoes of Humanity. He writes from Ontario, fuelled by coffee, dry humour, and a healthy distrust of unchecked systems.

Find his books and other articles at jasonbu.online