How much of your organization knows where it’s using AI as a legitimate, disclosed tool or piece of content, versus where it’s letting AI create the misconception that a customer, employee or member of the public is dealing with a human being? That’s the disclosure question every company should be asking before an AI-generated asset becomes public facing.

Listen to “The Rise of the AI Corporate Stand-in” (Audio Deep Dive)

Most of you are probably familiar with focus groups: a small sampling of a company’s target demographic used to gauge opinions and perceptions on everything from new packaging and brand positioning to the company’s next strategic direction. Now, imagine tapping into a much wider audience of hundreds or even thousands of people, or conducting a one-on-one interview with your target audience anytime you want, from the comfort of your laptop.

The Use of Synthetic Audiences Versus Human Insights

Synthetic Audience platforms have been around for a few years, starting as an experiment and evolving into an accepted industry resource. Think of synthetic audiences as digital twins that simulate how a human would respond to a question. These AI audiences are built much like a flight simulator—i.e., real-world data, such as surveys, purchase histories and behavioral trends, trains the model to respond the way your actual target audience would. The key difference is that the pilot knows it’s a simulator. A synthetic stand-in works because everyone assumes it’s real—until they discover it’s not. Then what?

“How do you strike a balance, deploying AI-generated assets that support a company or service, without coming across as misleading?”

The use of synthetic AI as a cost-effective stand-in for a corporate brand is something we’ve covered before, whether it was AI influencers reshaping the $250 billion creator economy or synthetic music on Spotify. Then there’s the proliferation of AI slop, where both individuals and companies push out low-quality AI-generated content, images, text, audio, video and websites that pretend to be authentic while offering little to no real value to the audience. Wired recently covered AI slop’s impact, and how websites and apps are incorporating ways to flag, label and ban AI-generated content.

Numerous studies have examined how relying on AI can reduce our cognitive skills and critical thinking, but there’s a broader impact on corporate business as well. Harvard Business Review recently named it “knowledge decay,” stating that “AI-generated low-quality output degrades the information companies rely on to make decisions.” Left unchecked, this erodes trust and can quietly cheapen or discredit the organization producing it.

Which raises the real question of how do you strike a balance, deploying AI-generated assets that support a company or service, without coming across as misleading?

AI Search and Contaminated Content Pools

Since AI’s introduction, more marketing and content teams have discovered a fast and easy way to push out content designed to game SEO algorithms, and more recently, AI search platforms. The assumption was simple: the more content published, the more likely it would be cited by AI platforms, driving traffic back to the site. In reality, this tactic has backfired.

This study found that 86% of articles ranking in Google Search are still written by humans, with only 14% AI-generated. The same goes for AI platforms; 82% of articles cited by ChatGPT and Perplexity are human-written, and when AI-generated articles do rank, they tend to rank lower than the ones created by humans.

Yes, the flood of AI slop is contaminating the content pools LLMs source from, making some of it indistinguishable from human-generated content. But as we’ve found in the AI search audits we run for clients, LLMs still tend to favor third-party platforms like Reddit, reviews and YouTube—sources they perceive as truthful and credible—over a corporate AI stand-in.

Leading with Transparency

I use AI a lot, mostly for research on topics like this one, or as a digital data analyst to help me identify patterns and opportunities for the AI search audits I run for clients. I use it to double-check my grammar and keep these posts on topic. I also use it to generate both audio conversations of these posts and infographics that highlight their key takeaways. I’m fully transparent about all of it. I’ve also used it to prototype web concepts and build digital tools from scratch, helping me organize the avalanche of topics that populate my inbox every day.

With all the talk of AI cloning, misrepresentation and being misleading, this article in today’s New York Times offers a genuinely interesting contrast. Harvard Business School is now offering an eight-week, $699 program called HBS Foundry, an entrepreneurship bootcamp that guides participants through refining a startup idea, practicing investor pitches and rehearsing board meetings. Alongside live weekly sessions with real instructors, students can practice on demand with AI avatars trained on actual Harvard faculty, avatars they know are AI, available anytime, not limited by a professor’s calendar.

Any professor, like any professional mentoring or teaching, only has so many hours in a day and so much reach. This creates a real opportunity: the ability to rehearse a pitch or work through an idea on your own schedule, instead of waiting for someone else’s availability. It also raises a longer-term question. Will this eventually extend to doctors, lawyers, and other professionals we want more access to, where getting their time first means going through their AI clone? It’s worth thinking about. Are we being misled here, or are we tapping into something genuinely useful, just delivered differently? Maybe it’s not so different from learning from a recorded lecture instead of a live one. Is there really a difference?

But what about the growing deployment of secondary AI platforms that create intricately designed PowerPoint presentations, clone voices and iterate marketing campaigns by the hundreds in minutes? Or talking video avatars that now stand in for people on Zoom calls, making it harder to distinguish them from a real recording? Individuals and companies are building their own personal or corporate AI stand-ins.

Adobe’s research found that 46% of consumers would be more likely to unsubscribe from emails if they knew AI had written them, with consumers already estimating that they receive around six AI-generated emails every week. A separate study backs this up: Meltwater and YouGov found that 86% of consumers believe AI-generated content should be disclosed, while 32% said they’d trust a brand less knowing content was AI-generated, versus only 15% who’d trust it more.

There’s a bigger issue with AI deepfake video and audio: it’s becoming harder and harder to tell who, or what, is real.

The $25 Million Deepfake

Instead of misleading a consumer, what happens when AI deepfakes convince a staff member to transfer $25 million to Hong Kong bank accounts?

Arup, a British engineering group, can attest to this. A staff member received a message he believed was from Arup’s CFO, regarding a confidential transaction. The message directed him to a video conference with an AI-generated CFO and AI-generated colleagues. He was instructed to make a series of transfers totaling $25 million to five different Hong Kong bank accounts. Only afterward did he, and the company, realize it was fraud.

For CFOs and CIOs, as well as their boards, this represents an entirely new and genuinely alarming dimension of cybersecurity. AI deepfakes and voice cloning can now deceive a company, or even the public, when a message purporting to come from the company is sent through social channels, and the audience has no way of knowing what they’re being told is entirely false.

Balancing Disclosure and Trust with Being Misleading

On January 20, 1944, the U.S. Army activated the 23rd Headquarters Special Troops, known as the “Ghost Army.” This unit of artists, engineers and soldiers deployed inflatable tanks, fake radio transmissions and visual deception to convince German forces that entire additional divisions were stationed across Europe, shifting their attention away from key ground and helping the U.S. win key battles.

We’re all running some version of that operation now, just aimed at a different audience. Individuals and companies are building their own ghost armies, misleading companies and customers alike.

Let’s state the obvious: everyone, including you reading this, is using AI to some degree. It could be checking grammar in an email, reviewing and assessing data, or drafting a LinkedIn post. But if you’re using it to write client reports, are you disclosing that? If you’re publishing three or more AI-generated blog posts a week, is your audience aware of this?

If not, you’re misleading them.

This is yet another important AI oversight question for company leadership and their boards to be asking when it comes to content pushed out via LLMs to drive awareness or visibility: Is it ours, written in our own voice and shaped by our customers’ actual questions? If we’re using AI, are we clearly labeling it as AI-generated? Or was a topic handed to an LLM, executed and posted within minutes, resulting in our customers calling us out and handing us a PR nightmare?

Understanding how AI already talks about your company is the first step in knowing whether you’re controlling that story, or whether an AI stand-in is. If you’re interested in seeing how AI currently cites your company, and which competitors are getting cited instead, I’d welcome the chance to show you. Let’s talk.Email me at mike@scheinerinc.com.

As always, thank you for reading and/or listening.

Image Source: US Army