This quote from Sam Altman got the industry going:
“AI will handle “95%” of marketing work done by agencies + creatives.”
It’s provocative. And the industry didn’t help.
Industry publications published pieces to support the idea that AI could do things like keyword research:
Over 150k people read that piece…
I’m confident that a someone saw read it and told their team that they need to lower their investment in keyword research tools because AI can do it.
Huge mistake.
I’ll share why in this essay.
When ChatGPT first emerged (as well as other AI tools), marketers were worried that it was coming for their jobs. Since then, ChatGPT has been a powerful tool for improving productivity and achieving certain tasks. One task that ChatGPT has been claimed to do well is SEO keyword research and social media research. Today, I’m going to share with you research that outlines how prominent hallucinations are when ChatGPT is assigned these tasks and what marketers should know about these use cases.
What Are ChatGPT Hallucinations?
ChatGPT hallucinations refer to the inaccurate or irrelevant responses generated by the AI tool. This can happen when ChatGPT is tasked with complex tasks such as SEO keyword research and social media research.
While ChatGPT may be able to generate a large quantity of keywords or topics, it lacks the ability to understand context and intent. As a result, the responses produced can be nonsensical or misleading.
The Dangers of Relying on ChatGPT for SEO
SEO is a crucial aspect of digital marketing, as it helps websites rank higher in search engine results pages (SERPs). However, using ChatGPT for SEO keyword research can lead to disastrous consequences. Firstly, relying on ChatGPT’s hallucinated responses may result in using irrelevant or low-volume keywords, which can negatively impact a website’s search ranking.
Additionally, ChatGPT does not have the ability to analyze keyword competition and search volume accurately. This means that it may suggest highly competitive keywords with low search volume, leading to wasted time and effort on targeting the wrong keywords.
Moreover, ChatGPT’s lack of understanding of context and user intent can lead to misleading keyword suggestions. For example, if a user types in “best hiking shoes,” ChatGPT may generate responses for “best running shoes” instead. This can mislead marketers into targeting the wrong audience and ultimately harm their SEO efforts.
The Risks of Using ChatGPT for Social Media Research
Another potential issue with relying on ChatGPT for social media research is the lack of diversity in its responses. As an AI model, ChatGPT learns from existing data and produces outputs based on that information. However, this means that it may not offer a wide range of perspectives or insights. This can be problematic when conducting market research as it may limit the scope of understanding about a target audience.
Moreover, ChatGPT’s responses are only as good as the data it has been trained on. If the data is biased or incomplete, then the AI-generated content may also contain these biases and inaccuracies. This can be particularly concerning when researching sensitive topics or demographics.
The Research: How Does ChatGPT Hold Up?
Despite the social media buzz suggesting that ChatGPT can perform the same tasks as your SEO team or content writers, this couldn’t be further from the truth. Believing this myth will lead you down a dead end, causing harm to your SEO strategy. ChatGPT is a powerful tool, don‘t get me wrong. But it falls short in several critical areas of SEO. Relying on it for SEO will result in misleading insights, flawed ideas, and inaccurate data.
Unlike an SEO Tool built for the industry, ChatGPT does not collect and refine this type data. Unlike a human analyst, ChatGPT cannot engage in market research, or interact with audiences 1-1 to understand their needs. The results of our recent study which will be found below confirms that ChatGPT can’t be trusted to perform accurate keyword research or identify relevant subreddits for your product. This brings me to my first example, debunking the use of ChatGPT to find active subreddits.
Using ChatGPT to find Subreddits
By now, you might be firmly aligned with my perspective, cautiously skeptical, or somewhere in between regarding ChatGPT’s capabilities. However, I am confident that I can debunk this myth one example at a time. This brings us to the first: using ChatGPT to find subreddits.
Imagine you’re deeply invested in the world of Pickleball, either as a business enthusiast or a fan seeking community engagement on Reddit. You turn to ChatGPT to discover the most popular pickleball-related subreddits, eager to connect with like-minded individuals.
Here’s what ChatGPT suggests as the most popular subreddits on pickleball:
- r/pickleball
- r/pickleballtips
- r/pickleballstrategy
- r/pickleballvideos
- r/pickleballsales
It even goes on to share what some of the most popular topics are within each subreddit. Right away, this seems like a game-changer! But is it too good to be true?
At first glance, this list looks great. A potential goldmine for engaging with niche communities. However, as we all know, appearances can deceive. So to be sure, let’s verify the accuracy of this data before drawing conclusions.
When we take a closer into each of these subreddits, we can see that they actually have minimal or no activity. R/pickleballtips has only one member, r/pickleballstrategy has 2 members, and R/pickleballvideos doesn’t even exist.
After researching and actually looking at each of these subreddits, the reality is radically different. The subreddit r/pickleballtips has one single member, and r/pickleballstrategy only has two. Shockingly, r/pickleballvideos doesn’t even exist.
These discrepancies clearly demonstrate the inaccuracies in ChatGPT’s data, misleading marketers into believing they have found valuable communities when, in fact, they haven’t.
Marketing professionals who are seduced by the allure of AI should take note of this cautionary tale. Despite AI’s transformative impact across various industries (including marketing), identifying online audiences and communities requires human judgment and context. Which ChatGPT just can’t replicate.
If you’re still on the fence and think maybe this was a fluke, let’s get into my next example – one of my favorites to debunk.
Using Chat GPT to Conduct Keyword Research
Another widespread misconception is ChatGPT’s capability to conduct keyword research, which is quite concerning.
For example, this post claims that to find keywords that rank, you can use Chat GPT. Could Chat GPT seriously replace SEO tools like Moz, Ahrefs or SEMrush?
This seems like another too-good-to-be-true moment but I’ll give it the benefit of the doubt.
In this post, Chat GPT indicates that the keyword “copywriting resources” has 790 monthly searches and a difficulty score of 39. However, when we log into Ahrefs and enter the same keyword into the Keyword Explorer, we find it has only a search volume of 40 and a difficulty score of 8. Chat GPT’s estimates were significantly off.
When I checked the next keyword, “copywriting tips and resources,” there was no data available. I even reviewed every other keyword on the list, and none of them matched up to what Chat GPT was sharing. It was way off.
This post reached a whopping 60,000 users. Even if we assume half of those were bots, that’s still 30,000 real people who might have been misled.
Let’s narrow it down further—say only two-thirds of those users actually care about keyword research. That’s still a big number, around 20,000 people thinking they can ditch their trusty SEO tools.
To make matters worse, even well-respected publications like Search Engine Journal seemed to buy into this idea. They shared a screenshot in an article with a detailed list of keywords including categories, target personas, search intent, keyword volumes, and difficulty scores.
It looks legit at first glance, ready to plug right into your content plan. But is it accurate? According to ChatGPT, the keyword “search engine optimization” supposedly has a search volume of 88,000, “seo marketing” at 9,000, and just “seo” at 6,600. For the sake of research, let’s compare that with data from a solid SEO tool like Ahrefs.
The differences are massive! Just look at the keyword “seo” for example. Ahrefs says it has a search volume of 191,000, while ChatGPT says only 6,600. That’s not just a small difference. Chat GPT is messing up by tens of thousands.
That could seriously mess up your SEO strategy and have you miss out on big opportunities… or have you think something is a larger opportunity then it actually is. Imagine you show these messed-up keyword lists to your boss, only to find out later they’re way off.
Imagine presenting these flawed keyword lists to your CMO, only to find that the keywords have vastly inaccurate search volumes and difficulties. This discrepancy can significantly impact your SEO strategy, leading to poor performance and missed opportunities
In today’s online world, where info spreads fast and people trust what they see, my advice is simple: Question everything, especially when it seems too good to be true.
To see a detailed explanation and demonstration of these points, check out the video below:
Should You Use ChatGPT For SEO?
We analyzed over 500 different keyword volume estimates from some of the top SEO tools in the world to compare with the data provided from ChatGPT. The goal of this was to understand how ChatGPT’s estimates would compare to a tool that has spent years to gather keyword volume data that could be trusted by SEO marketers. Our conclusion based on an in-depth analysis of the keywords and their estimates is that no SEO should trust or rely on ChatGPT for gathering keyword data.
Our analysis found a gap between every keyword so significant (sometimes 1,000x) that it could be a costly mistake for any SEO to rely on ChatGPT for this information. We have included charts below outlining three use cases in which ChatGPT could have been used for keyword research but through a comparative analysis it became clear that it was not to be trusted.
Let’s check out another example, comparing keywords like “CRM analytics,” “CRM software,” and “CRM platform.”
You’ll see some big differences between what ChatGPT says and what the actual SEO tools show.
And it’s the same story with topics like VOIP—there’s a big gap in the numbers.
Still not convinced?
Let’s try one more query. Let’s run an analysis across sunglasses:
As you can see, the discrepancy between what ChatGPT thinks the average monthly search volume is vs what an actual keyword research tool does is significant.
Go ahead and try it yourself. Get specific with ChatGPT about doing keyword research on a topic you know well. Set the difficulty, volume, and all that. Then compare the results with your favorite SEO tool. You’ll probably be shocked at how different ChatGPT’s data is compared to the real thing.
This just shows how important it is to trust real SEO tools and human judgment when you’re planning out your digital marketing strategy.
The Reality of AI in Marketing
Now that you (hopefully) understand that ChatGPT isn’t anywhere close to replacing your SEO team, you might be wondering, “Okay… so what now?”
Firstly, stop relying on ChatGPT for keyword research—or any research, for that matter. AI does not eliminate the need for conducting thorough research, understanding the intricacies of subreddit dynamics, or leveraging proprietary tools that provide data-backed insights for informed decision-making.
Instead, broaden your perspective on how AI can be utilized at work. While AI has revolutionized aspects of marketing by automating tasks and enhancing productivity, it cannot replicate the depth of specialized data and insights that dedicated SEO tools like Ahrefs or SEMrush, as well as human expertise, can provide.
That is why I created the AI Marketing Console. It’s designed to integrate AI into your workflow, improving the overall quality of your work and saving you time. The mission of this tool is to give your productivity superpowers, leaving your colleagues wondering how you operate so effectively!
Understanding its capabilities and limitations will allow you to strategically integrate ChatGPT into your workflow to enhance efficiency and creativity. Let’s explore practical ways to effectively use ChatGPT at work.
How to Use ChatGPT at Work
ChatGPT offers practical benefits across various tasks, making it a versatile tool to add to your kit. Here are several impactful ways to effectively integrate ChatGPT into your daily workflow, for optimizing efficiency and fostering innovation:
Increase Efficiency
Enhance your efficiency and productivity using ChatGPT to streamline tasks such as:
-
- Data analysis: Analyze complex datasets and reports by inputting raw data to ChatGPT. It will summarize key insights, trends, and anomalies, saving you time on manual data review.
- Meeting summaries: After meetings, input notes into ChatGPT for concise summaries of key decisions, action items, and discussion points. This helps participants stay informed without lengthy recap sessions.
- Competitor analysis: Input competitor data and industry reports into ChatGPT to swiftly analyze market trends, competitive positioning, and emerging threats. This allows you to stay updated and make strategic adjustments promptly.
Content Ideation
Use Chat GPT to enhance creativity and generate content ideas. Here are a few examples:
- Audience insights: Provide ChatGPT with customer feedback, social media interactions, and survey responses to analyze common themes, sentiments, and interests among your audience. Use these insights to brainstorm content ideas tailored to your target demographics.
- Content gap: Input your existing content library into ChatGPT to analyze topics, formats, and engagement metrics. ChatGPT can suggest new content ideas that fill gaps in your strategy, ensuring comprehensive coverage and meeting audience needs.
- Social media posts: Generate new and engaging social media posts. Input previous successful content examples into ChatGPT to analyze patterns and themes that resonated with your audience. By leveraging these insights, ChatGPT can suggest fresh ideas and variations that align with your brand voice and appeal to your followers.
For even more inspiration, check out this free guide with over 100 ChatGPT prompts for marketers. It’s filled with ideas you can use to drive better results.
Content Scaling
Efficiently scale your content creation efforts by utilizing Chat GPT for things like:
- Multilingual content creation: Translate existing content into multiple languages accurately and quickly. Input content in one language, and ChatGPT can generate translated versions suitable for different international markets, speeding up the localization process.
- Video script generation: Use ChatGPT to generate detailed video scripts based on concepts or outlines. This streamlines scriptwriting for video content production, ensuring consistency and clarity in messaging across various projects.
- Email marketing campaigns: Draft email copy for marketing campaigns with ChatGPT. Input campaign objectives, target audience information, and desired tone. ChatGPT can generate engaging email content including subject lines, body text, and calls to action, accelerating the email creation process and facilitating personalized communication with subscribers.
Conclusion
If you want SEO or social media success…
Firing your marketing team for ChatGPT is a mistake.
While ChatGPT is a powerful tool, it is not a replacement for your SEO team or specialized keyword research tools. Use it to enhance your productivity and efficiency, but don’t rely on it for strategic tasks that require detailed research and human insight. Celebrate the ways AI can support your work without falling into the trap of overestimating its capabilities.
Understanding its strengths and limitations will allow you to strategically integrate ChatGPT into your workflow to optimize efficiency and foster innovation. Whether it’s streamlining data analysis, generating content ideas, or scaling content creation across languages and mediums, ChatGPT can be a valuable ally in your marketing toolkit.
Embrace AI as a complement to human expertise rather than a substitute. To help master this, check out the AI Marketing Console—a tool crafted to enhance productivity and streamline workflows using AI.
Remember, success in marketing requires a blend of technological tools and human creativity. ChatGPT is just one piece of the puzzle—a tool that, when used wisely, can enhance your capabilities and empower you to achieve your marketing goals more effectively.