16 Jun How to Mine Competitor Websites for Untapped Keyword Opportunities via @jeremyknauff
Do you feel like you could generate more revenue if you could just crack the code on driving more organic search traffic to your website?
I have some good news and some bad news.
The bad news first – there isn’t a silver bullet. There’s no “one sneaky trick” that lets you dominate organic search, no matter what some “guru” is trying to tell you on Facebook.
But since you’re reading this article on Search Engine Journal, you probably already know that.
The good news is that you can “crack” the proverbial “code” and earn significant organic search traffic by mining your competitors’ websites to find untapped content keyword opportunities.
This is the foundation that enables you to create original, useful content that your visitors will seek out and engage with.
Most people think that identifying worthwhile topics to write about is relatively straightforward, right?
Just plug some keywords into your favorite keyword research tool, sort through the data that it returns, and then begin planning your content around that.
Well, yes and no.
That’s a good start, but it’s really just scratching the surface.
With this approach, you may come up with a lot of topics, but if this is the only approach you use, you’re competing over the same small pool of topics and leaving a lot of opportunities on the table.
But by leveraging your competitors’ websites, you can uncover a tremendous number of additional topics with sufficient search volume, that you may never have even considered.
The first step is to identify your competitors. I don’t just mean direct competitors – I mean any website that is outranking yours for core topics. If they are getting in front of visitors before you have a chance, they are your competitor from a search perspective.
From here, we’ll need to find out what these websites rank for using SEMrush. Simply enter a competitor’s URL, and then click to “View full report” button in the “TOP ORGANIC KEYWORDS” section, and you’ll be provided with a comprehensive list.
Some of the topics will be predictable. Others might be surprising.
As you begin to analyze the data, you’ll find topics that you’ve never thought of. In some niches, keyword research tools may not have accurate data. I’ve run into this from time to time for obscure topics.
For example, I recently stumbled upon a specific keyword phrase that was widely known and used within a particular industry, but none of the tools showed any search volume for it.
However, I had accurate first-hand knowledge that it alone was responsible for over 6,000 monthly visits. As you might imagine, we immediately targeted this phrase.
I bring this up to highlight the fact that you can’t rely entirely on the data provided by any keyword research tool. You can use that data as a starting point, but then you’ll need to dig deeper to identify hidden opportunities.
Expanding Our Pool
From here, we will expand out into tangentially related websites to identify even more opportunities.
There are a lot of relevant topics that most of your competitors are not writing about, but that your potential customers are interested in.
Often, these topics won’t be tied directly to buying intent. While that might make it seem counterintuitive to target these types of topics, it offers a powerful opportunity because it helps you reach people earlier in the buying process. That gives you a chance to put your brand in front of potential buyers long before your competitors can.
It can also help you to demonstrate greater expertise, and inspire trust.
This is because while most of your competitors are only creating content about their products or services, you’ll be creating more comprehensive content that answers visitors questions at all stages of the buying process.
This shows them that you’re more knowledgeable than your competitors, and also that you care just as much about serving them as you do about selling your products or services.
What we’ll do at this stage is identify:
- Websites our competitors are linking to.
- Websites that are linking to them.
This could end up being just a few hundred websites or it could be millions, depending on the industry. Either way, we’re not going to just take the data at face value.
Compiling the Data
First, let’s talk about how we’re going to compile the data, then we’ll talk about how to sort it.
Who Are Our Competitors Linking Too?
This is important information because it tells us what they find valuable.
It’s also important because if they’re linking to a particular website, it’s likely that it isn’t a direct competitor to them, which means it also probably isn’t a direct competitor to you.
Hold on to this data because it will be useful outside the scope of researching content topic ideas. It can be a treasure trove of link building opportunities as well.
The easiest and most effective way to compile your competitors outbound links is to run Screaming Frog to crawl their website, and then export that data to a CSV file.
Who Is Linking to Our Competitors?
Next, we need to find the websites that link to them. My preferred tool here is SEMrush. I recommend exporting this data as a CSV file as well.
This is important information because if they are linking to your competitors, they are likely relevant to your website and the topics they’ve written about will probably be of interest to your audience.
Filter the Data
From here, we will filter this data by relevance and quality. Skip the garbage link and article directories, PBNs, and irrelevant or low-quality websites.
Now take the list that remains, and begin dumping that data into SEMrush to find out what topics those websites rank for. Again, this can all be exported as CSV files.
You can simply export all of the data and then remove the irrelevant or inappropriate topics in Excel or Google Sheets, but I prefer to sort the data in SEMrush before exporting. This way, I only export exactly what I want.
In many cases, keyword research tools will have relatively accurate data on search volume, so that’s where I like to start my sorting.
Next, I’ll sort by keyword difficulty, and then by average position.
We’re looking for topics that offer reasonable search volume and face minimal keyword difficulty. We want low-hanging fruit.
If these other sites are ranking for a topic, but not ranking well, that’s a sign that it may be a hidden gem.
I recommend that you merge the data from each CSV file into one. This enables you to use the aggregate data to spot trends you might miss by looking at data from just one website at a time.
If you do this, be sure to annotate which rows are websites your competitors are linking to, and which are websites linking to them.
Plan Your Content
Once you’ve filtered the data, it’s time to start planning your content around it.
While you may be tempted to prioritize topics based simply on search volume, it’s smart to think a little more strategically.
Search volume is certainly a factor, but you’ll also want to look at how difficult it will be to rank for a topic, and it’s value to your business.
Low-Hanging Fruit
You’ll obviously want to make it a top priority to create content around the topics you and your competitors have overlooked. This is generally spread thinly across a broader range of topics.
In other words, you probably aren’t going to find many topics that will drive large volume on their own.
One tactic I like to use is to find phrases that are used frequently on a website, but that don’t have their own page.
On smaller websites, this is something you can do manually. On larger websites, you’ll need to use a more automated method. One approach to this using Screaming Frog is outlined in this Moz article.
You’re going to be creating new content here anyway, so you might as well do it right from the start.
Make sure it’s comprehensive, properly structured, and contains images and video. If you fail to do that, someone is likely to put in just a little more effort and outrank you.
On the other hand, if you can make it seem like too much effort, the majority of people will give up long before they outrank you and a lot of people won’t even try.
Set the Bar Higher
Equally important is to identify topics your competitors rank for with weak content. This might mean:
- Thin content.
- Less than ideal content ranking for a topic.
- Decent content with few or no worthwhile links.
In cases like these, creating a comprehensive piece of content and building relevant, authoritative links to it can be a game changer.
And, in addition to earning new organic traffic, you’ll also be taking it away from a competitor.
This creates a powerful competitive advantage because web traffic is a zero-sum game. The traffic you take from competitors can help to increase your revenue while choking out theirs.
That’s a great way to gain market share.
Get Ready for Battle
There will also be a core group of topics that your competitors are all constantly fighting over.
What you’ll want to do here is identify any weaknesses in the content that outranks yours for these topics, and work to improve yours until it is superior. You should then create related content, published as subpages to further support it.
For example, a home builder might create a page intended to rank for “Tampa custom home builders,” which explains various components of their homes, such as stucco, windows, and roofing.
They may then create a subpage for roofing, in which they explain roofing in great detail. They may even create a subpage of that, explaining the individual roofing options in greater detail.
https://verygoodbuilders.com/tampa/https://verygoodbuilders.com/tampa/roofing-options/https://verygoodbuilders.com/tampa/roofing-options/asphalt-shingles/
This approach can help provide important context to search engines while providing valuable information to visitors. But, like most SEO tactics, it’s easy to take it too far.
In rare cases, some websites may need more than three labels of subpages, but they will be few and far between. In most cases, three will be plenty.
Obviously, these subpages can help you to earn traffic for topics that they cover. But they also help to support their parent pages both by demonstrating a hierarchy to search engines and by providing a relevant piece of useful content to link back to the parent page from.
More Resources:
- Competitor Keyword Analysis: 5 Ways to Fill the Gaps in Your Organic Strategy & Get More Traffic
- SEO Competitive Analysis: The Definitive Guide
- How to Spy on Competitors with Python & Data Studio
Image Credits
Featured Image: Created by author, June 2019
All screenshots taken by author, June 2019
Sorry, the comment form is closed at this time.