Clean Rankings Report in Google Analytics

Joost de Valk has a post up explaining how to track Google search ranking in Google Analytics. The SEO ranking

Tweet This

Written by Damon on January 13, 2009

Joost de Valk has a post up explaining how to track Google search ranking in Google Analytics. The SEO ranking hack (Dutch) is by Andre Scholten, he guest posted on Yoast for English speakers.

I added a comment to the post showing how to use this technique to also track Yahoo and MSN rankings as well.

But the ranking report diplays page 1 results as unset, (page: ), page 2 results as (page: 10) or (page: 11). SERPs from subsequent pages are similarly awkward.

If you don’t mind writing a few extra filters, you can display search rankings more intuitively. Here’s how.

Clean Ranking Report

Basic Instructions

If you are reasonably experienced with Google Analytics, you should be able to get by with the basic instructions. If you are new to Google Analytics, there are very thorough detailed instructions further down the post.

  1. Create a profile for your search rankings filters.
  2. Create filters that include only organic traffic and only traffic from Google, Yahoo, MSN, and Live.
  3. Create a Custom > Advanced filter to extract search engine rankings using the following settings:
    1. Field A-> Extract A from Campaign Term: (.*)
    2. Field B-> Extract B from Referral: (?|&)(start|first|b)=([^&]*)
    3. Output To-> Constructor: $A1 | page: $B3.
  4. Create a Custom > Search and Replace filter to format page 1 results on the User Defined field using the following settings:
    1. Search String: page:$
    2. Replace String: page: 1.
  5. Create Custom > Search and Replace filters to format page 2+ results on the User Defined field using the following settings:
    1. Search String: page: 1[0-1]$ for page 2, page: 2[0-1]$ for page 3 etc…
    2. Replace String: page: 2 for page 2, page: 3 for page 3 etc…
  6. Check that the Search and Replace filters are in numerical order.

As you can probably imagine, step 5 can get time-consuming if you are dealing with a lot of sites. I tried number of different arithmetic ways of formatting the results that would have made set up simpler, but none of them worked. Right now, it seems you can’t perform mathematical manipulation of numbers in Google Analytics.

I couldn’t figure out any easier way of displaying the SERPS but if you want to experiment yourself, the Replace String field from the Custom > Search and Replace filter requires a valid regular expression and seems more likely to allow basic arithmetic.

If you plan on using this hack, please be sure to write a whiny blog post about how Google Analytics should let you:

  1. export profiles to other accounts,
  2. allow filters to perform basic arithmetic,
  3. make this hack standard on all accounts.

Maybe if enough of us whine, someone will listen.

Detailed Instructions

Careful step-by-step instructions for people who aren’t very familiar with Google Analytics.

Create Profile for Search Ranking Filters

You need to create a separate profile for the search ranking filters because  applying the filters will exclude everyone except for organic search visitors from Google, Yahoo, and MSN.

  1. On the Overview page, select Add Website Profile.
  2. On the Create New Website Profile page, select Add a Profile for an existing domain.
  3. In the Select Domain box, choose the domain you want to track.
  4. In the Profile Name box, enter Organic Search Rankings (or some similar name).
  5. Enter Country and Time zone as appropriate.

Filter Organic Traffic

Include only organic traffic.

  1. On the Overview page, select Filter Manager.
  2. On the Filter Manager page, select Add Filter.
  3. On the Create New Filter page in the Filter Name field, enter Organic Traffic (or other similarly descriptive name).
  4. From the Filter Type box, select Custom Filter and choose Include.
  5. From the Filter Field box, choose Campaign Medium. I’m really surprised that Campaign Medium works here because all the other campaign options refer to PPC and other links that you can tag with traffic parameters.
  6. In the Filter Pattern box, enter organic.
  7. In the Apply Filter to Website Profiles area, select Organic Search Ranking and click Add to apply the filter to the profile.
  8. Click Save Changes.

Restrict to Google, Yahoo, and MSN

Include only the organic visitors from Google, Yahoo, and MSN.

  1. On the Filter Manager page, select Add Filter.
  2. On the Create New Filter page in the Filter Name field, enter Search Engine Visitors.
  3. From the Filter Type box, select Custom Filter and choose Include.
  4. In the Filter Field box, choose Referral.
  5. In the Filter Pattern box, enter (google|yahoo|msn|live).[a-z]+. This regular expression means
    1. (google|yahoo|msn|live) match one of google, yahoo, msn, or live
    2. . match a literal period (.)
    3. [a-z]+ match 1 or more lower-case letters (for the TLD).
  6. In the Apply Filter to Website Profiles area, select Organic Search Ranking and click Add to apply the filter to the profile.
  7. Click Save Changes.

Extract Search Ranking from URLs

Extract the term and results page parameters from the referring search engine’s query string and output it to the User Defined report.

Google uses q to identify the term and start to identify the results page in the query string.

  • Page 1:  http://www.google.com/search?client=opera&rls=en&q=seo&sourceid=opera&ie=utf-8&oe=utf-8
  • Page 2: http://www.google.com/search?hl=en&client=opera&rls=en&hs=QB0&q=seo&start=10&sa=N
  • Page 3: http://www.google.com/search?hl=en&client=opera&rls=en&hs=tB0&q=seo&start=20&sa=N

Yahoo uses p to identify the term and b to identify the results page in the query string.

  • Page 1:  http://ca.search.yahoo.com/search?p=seo&fr=yfp-t-501&toggle=1&cop=&ei=UTF-8
  • Page 2:  http://ca.search.yahoo.com/search?p=seo&ei=UTF-8&fr=yfp-t-501&xargs=12KPjg1ihSroGmmvmnEOOIMLrcmUsOkZ7Fo5h7DOV5CtdY6hNdE%2DIfXpP0xZg6WO8T7xvSy7HBreVFdJGu277WVk0qfeK%5FGr%5FM6c6Vk9AoNdukWnl3tOJ%5F0%2Dmdm6d%2DDA%2E%2E&pstart=6&b=11
  • Page 3: http://ca.search.yahoo.com/search?p=seo&ei=UTF-8&fr=yfp-t-501&xargs=0&pstart=1&b=21

MSN uses q to identify the term and first to identify the results page in the query string.

  • Page 1: http://search.msn.com/results.aspx?q=seo&FORM=MSNH11
  • Page 2: http://search.msn.com/results.aspx?q=seo&first=11&FORM=PERE
  • Page 3: http://search.msn.com/results.aspx?q=seo&first=21&FORM=PERE2
  1. On the Filter Manager page, select Add Filter.
  2. On the Create New Filter page in the Filter Name field, enter Search Ranking.
  3. From the Filter Type box, select Custom Filter and choose Advanced.
  4. In the Field A->Extract A box, select Campaign Term and enter (.*).
  5. In the Field B->Extract B box, select Referral and enter (?|&)(start|first|b)=([^&]*).
    1. (?|&) match either a literal question mark(?) or an ampersand (&)
    2. (start|first|b) match one of start, first or b; Google, Yahoo, and MSN use these to identify starting point in search results
    3. = match an equals sign (=)
    4. ([^&]*) match everything until an ampersand (&).
  6. In the Output To->Constructor box, select User Defined and enter $A1 | page: $B3.
    1. $A1 get the values from the 1st set of parenthesis in the Field A-> Extract A box
    2. | page: $B3 write a pipe character and page followed by the value from the 3rd set of parenthesis in the Field B-> Extract B box.
  7. In the Apply Filter to Website Profiles area, select Organic Search Ranking and click Add to apply the filter to the profile.
  8. Click Save Changes.

Format Page 1 Results

The query string parameter identifying the results page parameter for Google, Yahoo, and MSN are all unset on the first page, so you need a special rule to format page 1 results.

  1. On the Filter Manager page, select Add Filter.
  2. On the Create New Filter page in the Filter Name field, enter Page 1.
  3. From the Filter Type box, select Custom Filter and choose Search and Replace.
  4. In the Filter Field box, select User Defined.
  5. In the Search String box, enter page:$.
    1. The dollar sign means match page: where it appears at the end of the string. Without it, the filter would match every occurance of page:.
  6. In the Replace String box, enter page: 1.
  7. In the Apply Filter area, select Organic Search Ranking and click Add to apply the filter to the profile.
  8. Click Save Changes.

Format Page 2+ Results

Match results pages starting at result #10 or #11 and change to page 2.

Google starts counting results at 0, so page 2 results start at #10. Yahoo and MSN start at 1, so page 2 results start at #11.

Off topic: you can hand edit the query strings to start anywhere you like letting you put result number 5487 first. So don’t trust screenshots showing top results in Google & imitators.

  1. For every search results page you want to track, on the Filter Manager page, select Add Filter.
  2. On the Create New Filter page in the Filter Name field, enter Page 2 (or 3 for page 3, 4 for page 4 etc…).
  3. From the Filter Type box, select Custom Filter and choose Search and Replace.
  4. In the Filter Field box, choose User Defined.
  5. In the Search String box, enter page: 1[0-1]$ (or page: 2[0-1]$ for page 3, page: 3[0-1]$ for page 4 etc…).
    1. page: 1[0-1] match page 10 or 11
    2. $ where it occurs at the end of a line.
  6. In the Replace String box, enter page 2 (or page 3 etc…).
  7. In the Apply Filter area, select Organic Search Ranking and click Add to apply the filter to the profile.
  8. Click Save Changes.

Check Filter Order

Filters in the wrong order can mess up your results. Of particular concern are results from pages 10 or 11, 20 or 21, 30 or 31 etc…

Put your page: x rules in numerical order. If your page 10 or 11 rules comes before your page 2 rule, then page 10 or 11 results will be converted to (page: 10) or (page: 11) which will then match page:10 for Google or page: 11 for Yahoo and MSN. These will then be converted to page 2 when they pass through the Page 2 Search and Replace filter messing up your results.

Otherwise, filters are probably fine in any order because they act on different fields, in ways that are not mutually exclusive, or on fields that aren’t set until they pass through a filter. But I can confirm that the following order works:

  1. Organic Traffic
  2. Search Engine Visitors
  3. Search Ranking
  4. Page 1
  5. Page 2
  6. Page 3 etc…

To set filter order:

  1. From the Overview page, click Edit for the Organic Search Ranking profile.
  2. On the Profile Settings: Organic Search Ranking page, scroll down to the Filters Applied to Profile section and click Assign Filter Order at the top right of the section.
  3. Adjust filter order as necessary.
  4. Click Save Changes to finish.

View the Results

It takes about a day for Google to update your profile with the new filters.

To view the report:

  1. In the Overview page, select the Organic Search Ranking profile.
  2. In the left sidebar, click Visitors to open the menu for all visitors pages.
  3. At the bottom of the list of visitors pages, click User Defined to see the results.

When you are done with this, you might want to check out my post on segmenting the ranking report by search engine.

51 replies to “Clean Rankings Report in Google Analytics”

  2. Damon

    Thanks,
    I noticed a lot of people were interested in formatting the reports from the Yoast comments and I figured I’d contributed enough in the comment section to justify writing my own post.
    Let me know if you have any problems.
    d.

    Reply ↓

  3. Post Author Damon

    Thanks,
    I noticed a lot of people were interested in formatting the reports from the Yoast comments and I figured I’d contributed enough in the comment section to justify writing my own post.
    Let me know if you have any problems.
    d.

    Reply ↓

    1. Post Author Damon

      @Andre
      Sorry, I was planning on linking to you but forgot. I edited the post to give you proper credit.
      @Michael
      I hope you finally get this working. Thanks for the link

      d.

      Reply ↓

  5. Michael

    Hi Damon,

    thanks for doing this excellent piece of hard work explaining it all so clear. I will translate the most important things in german and place into my Blog. For all english speaking people I do already link to this post. Thanks for bringing clear facts onto one page. In Yoasts Blog nobody knows anymore what is right.

    Thnaks, Michael

    Reply ↓

  8. Tony Stocco

    This is perfect. I was extremely confused reading all the information at Yoast.com – Seems like all of the tutorials refer to other posts to make it work. Impossible to follow.

    Thank you for organizing everything in one, easy to follow post!

    Reply ↓

  13. Rob

    I keep getting page . I looked up the keyword myself and it’s serp is on the 1st page. Should the results actually say page 1 or does page . = page 1?

    Reply ↓

  14. Post Author Damon

    Rob,

    If you didn’t do the formatting filters, then a blank page is what you should expect.

    If you did the formatting filters, then carefully double check the Page 1 Search and Replace filter first.

    Also if you started with the Yoast filters and switched to mine, you need to check the earlier filters because I had to do things a little differently to get the formatting to work.

    Reply ↓

  15. Rob

    I did find the Yoast page but I did your install because I preferred the results to be labeled correctly. My search results end up looking as follows:

    bottle stopper pattern | page: .
    My site is the 8th link down on my datacenter.

    My search and replace for filter page 1 =

    search : page:$.

    Replace: page: 1.

    Reply ↓

  17. Rob

    Yeah, I checked that when I originally did the filters. Ahh well, I’m just going to delete them all and redo everything. Maybe an empty space got inserted when I was copying and pasting the code in.

    Reply ↓

  18. Post Author Damon

    These filters only work on new data, so be patient. You might have fixed the problem, but you won’t know for sure until tomorrow.

    If you want to redo everything, create a new profile and parallel set of filters.

    Reply ↓

  19. Larkfield Agent

    Hi Damon,

    I went little further and created two segments so I can monitor how much traffic is coming from page 1 and how much from pages 2++.
    It is very easy:
    For Page 1 traffic:
    > Create an Advanced Segment (name it: Page 1 Traffic)
    > dimension or metric : place User Defined dimension
    > Condition: Ends with
    > Value: page: 1

    For Page 2++ traffic:
    > Create an Advanced Segment (name it: Page 2++ Traffic)
    > dimension or metric : place User Defined dimension
    > Condition: Does not end with
    > Value: page: 1

    Apply both segments to your reports and voila..

    Your tracking along with these two segments have helped me immensely to get a better picture of what keywords are used by my visitors at Larkfield Properties.

    I appreciate your time and I look forward to more tips.

    Reply ↓

  21. Larkfield Properties

    Hi Damon,

    Initially, I tried Yoast’s method, but my stats were pretty incosistent and most of them were showing (page:).
    It was also showing some traffic coming from bookmarks, which is irrelevant.

    While reading the comments there, I came upon your link, which perfectly cleared all the dust from the stats. 🙂
    Who says commenting in other blogs doesn’t bring traffic?

    I also think that my solution is much simpler and it’s quite logical. Plus it works the way it should.

    Reply ↓

  22. seo

    “The dollar sign means match page: where it appears at the end of the string. Without it, the filter would match every occurance of page”

    Would you please explain this ?

    Reply ↓

  23. Post Author Damon

    @seo

    The dollar sign is a special character in regular expressions that anchors a regular expression to the end of a string. If you want to match an actual dollar sign, you need to escape it with a slash $ (I can never remember which slash to call forward and which to call back, but that’s the right slash to use).

    Without the dollar sign the page 1 filter would match all results and cause the subsequent filters to fail.

    An unformatted page 2 result, for example, looks like page: 10 and would get changed to page: 1 10 (remember the page 1 filter matches page:, replaces it with page: 1).

    When the page 2 filter looks for page: 1[0-1]$, which means match page: 10 or 11, it will fail because of the extra 1 inserted by the page 1 filter.

    I hope that clears everything up for you.

    Reply ↓

  24. seo

    Thanks for this great tutorial.

    For filter to format page 2+ results, the replace String should be : page: 2 or

    page 2 ( without the colon)

    Also, if I apply filter for Google and organic results only, where can see the reports for this type of filter.

    Reply ↓

  25. Post Author Damon

    It doesn’t really matter.

    If you want to see your results displayed with a colon, then add the colon. If you don’t care for the colon, then leave it out.

    The earlier filters need to be more exact, but the page filters are just formatting so use whatever formatting you like in the replace string (but follow my instructions carefully with the search strings if you don’t know what you are doing).

    The reports will be in the Visitors > User Defined page whether you apply it to just Google or Google, Yahoo, and MSN.

    Reply ↓

  26. seo

    Thanks.

    I have applied the filters and will wait for a day to see the results.

    Regarding filter order, I clicked on Edit and changed the filter order to this –

    Organic
    Google Only
    Search ranking
    page1
    page2
    page3

    But in the filter manager view, the order is

    Google Only
    Organic
    page1
    page2
    page3
    Search ranking

    I hope i have not messed up.

    Reply ↓

  28. seo

    I have applied all the filters and after a day checked to see no rankings. It just displays URL, pages/visit, their bounce rate ( in the visitors >User defined section ) but no mention of ranking. Where have I gone wrong ?

    Reply ↓

  29. Post Author Damon

    It’s kind of difficult for me to tell what you’ve done wrong without more information. I advise you to carefully double-check your filters. Make sure you haven’t accidentally copied extra characters, like a sentence-ending period. Start with the Field B->Extract B box and the Output To -> Constructor of the Search Ranking filter.

    If you can’t find anything wrong, disable the Search and Replace filters. You should get something that looks like this with lots of page not set and page 10, 11, 20, or 21:

    1. keyword 1 | page:
    2. keyword 2 | page: 10
    3. keyword 3 | page: 11

    If it doesn’t output pages like above, then the problem is in the Search Ranking filter or before.

    If you does output pages like above, then the problem is with your Search and Replace filters.

    Reply ↓

  30. Mika

    Thank you for taking the time to create the Detailed Instructions for us people not so familiar with workings of Analytics. Trying to figure out what to do from the Yoast post and comments was getting difficult.

    Reply ↓

  31. Dan Murray

    Thanks for the post and making it easier to understand than other posts.

    If we want page 3/4/5/6 etc results do we need to keep repeating the steps we did for page 2. So a new filter for page 3 with page: 3[0-1]$ and then another filter for page 4 page: 4[0-1]$ etc?

    Thanks,
    Dan

    Reply ↓

  32. Daniel Murray

    Damon thanks a ton for this! Really helpful and the best advice I’ve come across for adding these filters. Got my first days stats back today after adding it and it works a treat 😉 Cheers!

    Dan

    Reply ↓

  34. Post Author admin

    Umm. The report isn’t supposed to work with non-organic traffic. There’s a filter that specifically excludes all traffic that isn’t organic.

    Reply ↓

  35. newbie

    Hi, I’m really new to Google analytics and recently I found this post http://www.websharedesign.com/display-search-engine-rankings-seo-in-google-analytics.html
    Could you tell me what’s the different between your method and the method in this link?
    Would the user defined report display same keyword list as the keyword report?
    If I create new profile to the same domain as my main profile, will I lose all traffic records?

    Sorry for many questions. I’m a newbie. 🙂
    Thank you

    Reply ↓

    1. Post Author Damon

      @newbie

      That filter you referenced is for a new Google query string that shows the exact ranking, as opposed to just the page.

      I also blogged a solution showing you how to get the ranking from the new Google query string, but it looks like they beat me to it.

      Because the new query strings aren’t universally adopted, there really isn’t much point in implementing the filter except out of curiosity (and so that you know when it becomes worthwhile).

      Reply ↓

  36. newbie

    Hi Damon,

    Thank you for the answer and new blog but could you please explain to me about the new profile?

    If I create new profile to the same domain as my main profile, will I lose all traffic records to the main profile?

    Or in my understanding, the user defined report will not affect the ability to track the traffic in keyword report, am I correct?

    Thank you again.

    Reply ↓

    1. Post Author Damon

      The purpose of the profiles is so you don’t mess up your data. Anytime you want to use GA filters, create a profile to house your filtered data and keep your main profile unadulterated.

      So, to answer your question, the new profile will not affect any reports in your main profile.

      Reply ↓

  37. Bernardo

    Hi Damon,

    I have configured the filter to show results until page 5, because I think most visits will come from those pages. However, I am getting results such as page:70 in my report. What does that mean?

    Thank you

    Reply ↓

    1. Post Author Damon

      Bernardo,

      The reason you are getting that is that you haven’t implemented the filters past page 5.

      Whenever you see something like that just drop the 0 (or 1) and then add 1. So your example is from page 8.

      Reply ↓

    1. Post Author admin

      Yes.

      What you’re really seeing is the number for the first result that that particular page. Their query string just says start showing results at result number (10 or 11). Google starts counting at 0 while Yahoo and MSN start at 1 which is why sometimes page 2 is 10 and sometimes it is 11.

      Reply ↓

  39. James

    Hi,

    Great filters, although I still seem to be getting a lot of results listed as (not set), any ideas how to fix this?

    Thanks

    Reply ↓

  40. Damon

    @James

    You probably have a problem with the filters that exclude non-organic traffic or traffic that doesn’t come from google, yahoo, live or msn.

    Check the detailed instructions for these steps when trouble-shooting.

    Reply ↓

  41. James

    Hi Damon,

    I don’t have any exclude filters applied, just an ‘include’ for organic traffic.

    The ratio of (not set) results is about 90%.

    Thanks

    Reply ↓

  42. DVogel

    Damon, thank you for covering this topic so fully; your level of detail and ongoing additions and improvement to your code is very helpful and appreciated.

    Thanks to your detailed instructions, I was able to setup the filters. However, most likely to an error on my part, I’m experiencing a couple problems I assume to be connected. Since you can’t see my code, I’m just going to describe the symptoms, in case they tell you where I probably went wrong:

    1) As James described above, about 90% of the User Defined visits have a value of (not set)
    2) All of the keywords that are correctly tracked are from page 2 or deeper (I have filters setup through page 10).

    To me, it looks like the (not set) probably include all my missing page 1 results. So, I’ve triple checked my Page 1 filter code, and it appears the same as your instructions.

    Is there anything you would recommend I look at or change based on these symptoms?

    Thank you for your time and effort!

    Reply ↓

    1. Damon

      Sorry for taking so long to respond, I haven’t been using the ranking report and just haven’t gotten around to checking it out.

      Google has been experimenting a lot with different query strings lately, it seems that 90% of Google searches don’t include ranking information anymore.

      There was some rumbling a few months back about GA adding ranking info by default, so it might not be a problem for long, but at the moment there is no way to track Google ranking with GA for most searches.

      Reply ↓

  43. Lilly

    Damon, thanks for your detailed instructions which I appreciate very much. I was wondering if there is a way to set filters of specific keywords. For example, I need to know the organic ranking for my website when a user searches on the keyword “red shoes”. And I have 500 keywords that I want to check for. Is it possible to setup 500 filters? Is there a limit on filters in Google analytics?

    Reply ↓

Leave a Reply

Your email address will not be published. Required fields are marked *