Every once in a long while you come across a tool that just gives you goose bumps, you are instantly infatuated.

The Search Based Keyword Tool (SbKT) was that for me. The data it brings together and the transparency it brings is just so. . . sexy.

Let's see if you feel that by the time you are done with this post. I can tell you know that you'll never think of Paid Search the same way!

[ UPDATE: Recently Google has folded the Search Based Keyword Tool into the core AdWords Interface. You can now access the keyword tool from within AdWords by clicking the "Keyword Tool" link from the Opprotunities tab. You can also access the Keyword Tool outside of AdWords by clicking here.]

One of the most common phenomenon is the long tail of search. Yet precious few people understand this and even fewer actually are able to use it to their advantage.

A quick recap: The Search Long Tail.

If you have not had a chance please give my long tail of search post a quick review.

The essence of the phenomenon is that your website most likely has a “short head” (say ten or fifteen keywords that drive massive number of Visits) and usually a “long tail” (hundreds of thousands of key words and key phrases that drive five, ten, fifteen – few – Visits).

The really cool thing about the tail is that the thousands of queries in the tail are generic ("category") queries which tend to bring "impression virgins " to your site. People who are new to your franchise, people who have not made up their mind.

Find them first and you get first dibs on convincing them.

My Long Tail: The Story in Numbers.

Here's a real example, data for this blog:

87,861: # of Visits in the month of March. 40,662: # of Visits from Search (!!). 26,137: # of distinct keywords and key phrases that drove those 41k Visits!

Astonishing is that not? For a single topic blog! So what of the head and the tail?

13: # of search key words that brought 100 or more visits (the head). 26,124: # of search key words that brought 2 or 5 or 10 or a small number of visits each (the tail).

If you are not yet confused by all the numbers, please stick with me, here is another way to think about it.

13 head keywords brought 5,128 Visits to the site. 26,124 tail keywords brought 35,534 Visits to the site!

[This is the prime reason I tell Analysts, Marketers, CEO's, my children not to just obsess about the head!]

Clearly even a one topic (Analytics) blog can have a verrrrry long tail.

I can only imagine how long your tail is. You have a real business. You have thousands of pages. You have tons of products you sell, a diverse set of services and so on and so forth.

The Problem?

Despite the massively yummy opportunity, most people have a hard time monetizing the long tail because:

* All they know is their brand terms, so they take the easy way out.

I hate to say this but there is laziness involved, we rarely bother to look beyond the top 10 keywords. * They think broad matching on their key terms will take care of the tail. The reality is the best broad matches will pick up 500 or a 1000 queries, the tail has hundreds of thousands. * You can't possibly take every word in the text on your site and dump it into AdWords (or AdCenter or YSM).

Finding relevant long tail search queries is tough, and doing that intelligently is even harder.

The Solution?

The Search-based Keyword Tool (SbKT) from Google.

It uses two sources of data that Google has (both anonymous, non-PII) to help you identify long tail keywords that are being searched on www.google.com that are relevant to your business.

The first source of the data are the logs that store the queries that are done by users on google.com (logs that have this data in aggregate, not by individual histories – you'll see the latter is not necessary).

The second source of data are the logs from googlebot that indexes the world wide web and all the pages (and your site). These logs understand what your site is, what content is on it, what you sell, service, do on those pages.

SbKT simply brings these two sources together. When combined it provides you with:

A list of keywords and phrases in the search long tail that are relevant to your website (where relevance is defined by the content on your site).

It is a very simple idea, yet it is powerful because now you don't have to guess what the opportunity is for the long tail on google.com.

Google SbKT Reports & Analysis.

SbKT is a fairly easy to use tool, there really is not much to it.

This screen is what you'll see when you login. . . .

You have several options but first thing you can try is to simply type the name of your website and hit enter.

In two seconds. . . . Bam!

Ok so the color boxes won't be there. : )

Let me walk you through what you'll see.

The area inside the Red Box shows all the key results you need: key words and other data.

The Violet Box shows how many relevant keyword suggestions were returned, 65,863 for this site.

The data in the Blue Box allows you to filter the results by various product categories. Example, here, Apparel, Beauty & Personal Care, Sports & Fitness, Finance etc etc.

The Orange Box is the other way to filter the data, by specific product categories. Sony, Microsoft, HP, Samsung, Nokia etc in this case (so you can easily move away from the 66k results and focus just on results for Nokia, which in this case were 890).

Finally the Purple Box, at the very top, contains other filters (cost, competition, keywords etc etc).

Let's walk through the report itself.

Keywords, Monthly Searches, Competition.

The first column shows the specific keyword / key phrase that the tool recommends for your site. These are the long tail keywords identified after the analysis of Google's search query logs and the Googlebot log data.

The second column shows you how many searches were done for that keyword in the last month (+ or – 10%). It is important to know these are not match typed in any way, these are actual search queries of google.com users.

Finally the third piece of data is how much competition there is for an ad for that term (in your country typically, so you can change and choose to get data just for a specific country).

Suggested Bids, Ad/Search Share, Your Webpage.

Sugg. bid is the price you can expect to pay for showing your ad in the top three position on the Google search results page.

Ad share is % of time an ad from your website shows up for the query (due to broad-match). Search Share (the second percentage) is the % of time your site shows up in the first page of the organic results!

Think of both numbers, for Paid and Organic, as akin to impression share (or to those of you in the Retail Industry: "Share of Shelf")

Finally column three shows which page on your website is relevant for that keyword.

This last part is very important because you don't want to buy keywords willy nilly, you don't want irrelevant ads or, worse, irrelevant landing pages. SbKT solves this problem by telling you exactly which page on your site is the optimal landing page for a given long tail keyword.

You can click on the Export button and save the resulting Excel file. There will be two bonus items in there: 1) SbKT will attempt to classify your keywords into logical Campaigns and 2) into logical Ad Groups, thus saving you some time.

Net, Net.

You know:

exactly which keywords to buy to grab all the impression virgins how much share you have today of that traffic how much you can expect to pay for a top 3 paid search listing and exactly where to send this traffic so they have a relevant experience (which translates to a higher conversion rate)

Delightful. Data driven paid search campaigns!

SbKT Bonus Features / Analysis.

There are several deeper and cooler things you can do with SbKT.

Question: I want to have a post spring break sale on my GPS related inventory, how can I find the most relevant keywords and key phrases to purchase?

From the left navigation, in this example, click Consumer Electronics, then click GPS and do a merry dance. . . .

What you are looking at are the keywords related to the GPS category only and only the ones that are relevant to your business.

At the far left, not in the image above, you'll see the unique landing page on your site where you should send this traffic (in case you buy these keywords).

Question: I just saw a tweet that Olympus is going to out of business. How can I identify search keywords to unload all my Olympus inventory?

From the Categories choose Consumer Electronics, then Photo & Video and then Digital Cameras. From the Brands (bottom left) choose the brands Google has identified relevant to content on your site (Olympus).

There you go, a relevant list of keywords for you to consider.

Question: Budgets are really tight right now, can you help me find long tail keywords relevant for my business but that cost less than 30 cents?

In the top bar click on the plus sign next to Advanced Search. . .

You'll see a bunch of lovely options, in this case you'll choose Suggested Bid and type in the amount that you are interested in. . . .

When you hit enter SbKT will take the 222,720 long tail key phrases it had identified for you and show you the 75,464 that are relevant for your business – all of whom have a Suggested Bid of under 30 cents for a top three search listing.

As you see above, you can control the amount of competition you want. If I choose Medium and High ("bring it on!" :)) I get a list of 35,486.

You can also use other filters productively.

Win – Win for Organic and Paid Search Marketers.

If you are a Paid Search Marketer (PPC) then use SbKT to identify relevant long tail keywords and their corresponding landing page. Use Suggested Bid to focus on maximizing your budget and give your CFO estimates of cost (rather than guesses that you probably do today).

If you are a Organic Search Marketer (SEO) then Google has just provided you with a list of landing pages on your site and the keywords that are relevant to those pages. If you are not getting enough Organic share (ps2 above) then you have your work cut out for you.

For either individual there is data now to work more efficiently to improve search share.

Who Can See What?

Before you get too excited let me rush to add that you can only see data for your company using SbKT, i.e. the adwords accounts you have access to.

For other sites your view will be limited to only the top 100 long tail terms. You can see 100 in All Categories or in any Sub Category.

Tips for Maximizing The Long Tail Value.



[Image Credit: Hugh MacLeod]

#1: Don't get hung up one or two or ten keywords. Remember you are dealing with thousands of keywords.

Focus on the criteria that is important to you (low cost, high competition, particular categories or brands, a cluster of products), then put 'em in ad groups and experiment.

#2: Don't spend time torturing every single keyword. You can't. Or you'll never go home.

You are aiming to capture two, ten, fifty visits. Maximize for the portfolio.

#3: Remember all rules of measuring success for Search campaigns apply to the long tail (it must produce bottom line impacting results).

But you have a chance to think in a more sophisticated way about measuring the long tail. Please see this post: Measuring Value of “Upper Funnel” (Long Tail) Keywords

#4: SbKT works most optimally for mid and large sized websites (lots of pages, lots of products, lots of lots).

Notice the irony in that? :)

Closing Thought.

The data you get through the Search Based Keyword Tool are actual search queries that are actually being typed by people using www.google.com.

The information from the tool should be used to create effective Organic and Paid search strategies to ensure you get your share of the relevant traffic.

Just because you don't show up does not mean that your competition won't.

Good luck!

Oh and in case you did not realize it, this is Web Analtyics baby!!

Ok your turn now.

Have you used Google's SbKT? What's your experience like? If you have not used it, do you think you'll find it to be of value?

Got any tips on monetizing the long tail? Got horror stories?

Please share your critique, feedback, cheers and boos. Thanks.