All too often, SEO & site development efforts are out of sync. It’s a common complaint among SEO’s that a client will come to them with a Web site in hand and say “optimize this”. Whereas SEO’s would prefer it if the client would ask first and say “what content do we need to develop in order to have a site optimized for our target terms?”. It doesn’t have to be a chicken vs egg scenario though.

Read the rest of this entry »

When it comes to SEO, Google Webmaster Tools and I are Best Friends Forever.

Fist Bump - courtesy Time

I know at the end of the day, dude always has my back. He’s fluff to my peanut butter, Hutch to my Starsky, my broheim. He’s brutally honest; he’s one of those friends that tells you like it is, even if it’s bad. And I admire that!

Let’s get down to it: 10 reasons why Google Webmaster Tools (GWT) is my BFF (if you can’t follow this, there’s a list of straightforward reasons why I like GWT after this list):

1. He helps me rep-re-zent.  All I want is a little R-E-S-P-E-C-T.  I want to be known!  GWT tells me (“Index stats”) whether or not I can be found in his index. He should be braggin’ about me!  If not, then there’s something awry.
2. Mo Pages, Mo Problems.  GWT lets me know if he’s having problems getting to know me.  Am I slow in getting back to him (“URLs timed out”)?  Did I tell him I was going to meet him somewhere and not show up (“HTTP Errors” and “Not Found”)?  He’s a guy that keeps a journal so he knows exactly how many times he’s expected me to be somewhere only to not find me there.
3. He has has his ear to the ground.  He tells me what people are saying about me (“Top Search Queries”).
4. Helps spread the word if I want to be called “Christian” or “Chris” (“Preferred Domain”).  I mean, calling me both names is technically the same.. but what name would I prefer to be called?
5. He wants to know all. about. me.  So, instead of relying on our conversations for him to piece together what I’m all about, I just can say hey, can I just give you this full biography of who I am (“Submit a sitemap”) and just call that good?  He’s a crazy cat though – he only likes this biography in a certain format (.xml)… he’s weird like that (just playin’ GWT!).  He also is somewhat of a grammar snob, letting me know if there are any problems with the biography.
6. He’s detail-driven.  He’ll let me know if I have any issues (“Content analysis”).  So I didn’t have a title tag here and I had a long META description there – he gets up all in my grill and tells me about it!  Lay off, my man!
7. Dude is a n.e.r.d.!  We’re not talking about manga or anime or anything: he loves statistics (“Statistics Overview”).  I mean, it’s cool and all… but sometimes he starts talking about who craves my attention and (“Subscriber Stats”) even channels his ‘inner-self’ (“What Googlebot sees”) and really opens up about what he sees in me.  I mean, it’s flattering… but dang, he can be harsh!
8. He loves gossip.  He craves it, in fact.  And he will tell me whenever someone happens to talk about me (“Pages with external links”).  He’ll even note when I’m talking about myself (“Pages with internal links).  “Why would Christian steal from the Yankees?”
9. When people ask about me… GWT lets me tell him what I would like him to say about me (“Sitelinks”).  So, say someone asks about me… I can tell GTW that I would like him to let that individual know about my funny personality, perhaps a small synopsis of who I am, or otherwise.
10. Sometimes, I like to keep some details of my life private… like my love for Mandy Moore.  Or my liking of the Los Angeles Lakers.  I can tell GWT to never tell anyone of these certain aspects of my life (Generate robots.txt).  Well, I guess they’re not very private now..

And there you go.  That’s a short list of why Google Webmaster Tools and I are BFF.  Now, for the more serious readers – here’s the list again, but without all the BFF stuff

1. Index stats – says if your site is included in Google’s index.  Also if Google has accessed your home page.
2. Web crawl errors – tells you if Google is having trouble crawling and indexing your site.
3. Top search queries – gives you an quick synopsis of the top search queries people are searching on that access your site.
4. Preferred domain – specify your canonical here to tell Google if you would like your site to be http://www- or http://-.
5. Submitting a sitemap – submit your xml sitemap and see if there are any errors Google has found.
6. Content analysis – lets you know if any of your pages lack title tags or have long (or short) META descriptions.
7. Statistics – gives you some stats such as how many subscribers your site has, crawl stats and even “What Googlebot sees”.
8. Links – lets you know how many reported external links are pointing to your site (and which specific pages are being linked to) as well as your inlinks.
9. Sitelinks – you can somewhat control what sitelinks Google shows for a branded result of yours that has sitelinks.
10. Robots.txt generation and analysis – you can generate a robots.txt file on GWT and also analyze it to check if there are any errors.

Google Webmaster Tools is an invaluable tool that allows you to really see in-detail what Google thinks of your site.  If you have problems – don’t fret!  Start correcting them one-by-one and before you know it, you’ll have a website in good standing with virtually zero crawl or sitemap errors. That will make it a lot easier to get great results from Google.

Greetings humans!  My name is Robots Dot Text.  You can call me robots.txt (ah hah!  Clever, eh?).  As you can see, I am mainly part text document (hence my bland-looking figure; apologies for not being part Word document or Photoshop image), part “dot” (it is hard finding suits that fit a dot-like figure) and part robot.  I am quite a character!

Think of me as a liaison between your website and a search engine robot/spider.  I am like your website’s personal butler: I greet web robots at the door and give them directions as to where they may proceed within your site.

Usually, when you humans have guests over, you do not necessarily like others to go into your human bedroom (unless you are into that thing… not that there is anything wrong with that!). In the same way – you can instruct me as to where you want me to keep the web robots from visiting on your site.  You do not want them to see a page for some reason?  I can make sure that happens.

First and foremost, you may be asking yourself, “Why do I need a personal butler for my website?”  Well, then I may offer a retort of: “Why do aquatic vertebrate animals need water?”  Hah hah hah hah!  Best practice-wise, every site owner needs a personal butler to greet web robots at the door and tell them to not spider specific pages.  Pages that you should typically tell me to not let robots access are your privacy policy, terms of use and contact form fillout pages.  I would think having your “money” pages show in search results would be much better than your boring privacy policy, yes?

These web robot guests are quite the fickle bunch.  They only seem to respond to a particular greeting.  Think of it as a secret “bro-shake” that we do before they know they should comply with my requests.  It goes like this:

User-agent: *

That is my universal greeting to all web robots.  I can also greet individual web robots if you would like me to.  Here is my personal greeting to my ‘brother-from-a-search-engine-mother’ Googlebot:

User-agent: Googlebot

If I were to say this, only my web robot friend Googlebot would respond to the directions that follow.

If you were to say, “Hey Robots Dot Text, I do not want any web robot guests visiting (and consequently indexing) my bedroom,” or a single-page on your site, I would then instruct the “User-agent: *” this way:

User-agent: *
Disallow: /upstairs/bedroom.html

Web robots are totally cool with me telling them where they can visit in your website and where they can not.  They are quite proper, like me, and will not look down on you if you only want them to see a section of your site.

Let us say, for example, that you do not wish for your web robot guests to see your bedroom… but thinking about it, you do not want them to visit the entire upstairs which includes your “man-cave” (hey, that is what you call it, not me!) and guest bathroom.  This is acceptable as well; I can tell your web robot guests not to visit (and subsequently index) upstairs, or an entire sub-directory of your site, I would then instruct the “User-agent: *” this way:

User-agent: *
Disallow: /upstairs/

If I tell your web robot guests this, they know that visiting upstairs (i.e. your ‘upstairs’ sub-directory) is most definitely not allowed!

I can also promptly tell all web robot guests who come over to turn away and come back at a better time.  This is best when you are performing any maintenance at your place (like a redesign).  Do not feel bad about turning away your web robot guests during this time; it is best for them to see your place when you have everything organized and looking sharp!  If you would like me to tell your web robot guests not to visit (and subsequently not index) all of your pages of your site, I would instruct the “User-agent: *” this way:

User-agent: *
Disallow: /

Those are the basics of my butlerly-duty.  I am hoping that the word I created in the previous sentence makes sense for you since it is not a real word in the English language.  Entschuldigung! Hah hah hah hah!

Well, I am off to greet some more web robots.  I tell you: some of these guests come back almost daily!  It is tough work being me.  But it is quite rewarding!

At Amplify Interactive, we look at statistical history when drawing comparisons to current analytic and pay-per-click numbers.  The search behavior of the world populace is a wild and wacky one; there are dips and gains.  Most of the time, you can predict lower or higher search volume for any given month for an industry-set of search terms by looking at the previous years’ data (I stress “most of the time” here because sometimes it does not line up that way).

Comparing Last Month to This Month Doesn’t Give You the Big Picture

That being said, it’s nothing new for a client of ours to request a comparison of data of this month to last month.  So, for example, we would get a request to compare March numbers to February.  While I understand the desire to see this comparison (“Did we improve month-to-month?”), this measure doesn’t tell us a whole heck of a lot about whether we’re seeing improved site traffic / PPC numbers / etc.  Why doesn’t this month-to-month comparison work very well?

Yearly search trends for “homes for sale” – follows a fairly close yearly pattern

Look at the example above. If you were to compare November to December without looking at yearly data comparisons, you’ll wonder why December looked more positive for your PPC and analytics numbers.  Looking at things yearly, you can see that almost every year that’s quite a search volume increase for “homes for sale” from November to December.

Obviously, a lot of other factors may come into play about why you had gains / losses for your site analytic key performance indicators (KPIs) or your PPC stats.  These factors may include:

• Organic search rankings
• Paid search rankings
• Offline marketing initiatives (look for increases / decreases in ‘direct traffic’ referrals)
• Click fraud instances effecting your PPC campaign, boosting your bounce rate and lowering your average time spent on site / page views per visit metrics
• Other elements (your site traffic may dip during the NCAA basketball tournament, perhaps major news events affect search behavior, news about more help for first-time home buyers may boost real estate-type search queries, etc.)

Look at the Big Picture

To sum it up, don’t short-change your site analytic or PPC performance comparisons.  If you’re providing month-to-month comparisons, that’s fine – but look at the big picture when providing analysis.  Look at historical performance and compare time frames from year-to-year to see if there’s a upward (or downward) trend.  Think about other factors like whether their organic search ranking presence improved or if an offline marketing initative was launched.  Simply put: short-sighting your comparisons is limiting your effectiveness to provide a detailed analysis for statistical comparisons.

The following goes over some best practices you should be, well, practicing if you’re involved in a site refresh (redesign) project.  We hit on some steps to take to make sure your rankings carry over to your new site as well as measures to take to ensure that your ‘beta’ site doesn’t work against you.  Then we hit on some quick “lessons learned” items that you should go over before you finally flip the switch.

301 Redirects

When performing a site redesign, you want to make sure all of your old pages have a 301 redirect in place to its URL on your new site. It should be on a page-by-page basis; you do not want to just do a blanket 301 redirect for all pages to go to your root domain. If the page URL is staying the same, you do not need to use a 301 redirect. If you are planning on not having a page you had on your previous version of your site on the new version of your site, then you should set up a 301 redirect for that old page to go to the home page.

The Wrong Way

Your old red widgets product page redirects to your new home page:

http://www.widgetwarehouse.com/redwidgets/ >> 301s to >> http://www.widgetwarehouse.com/

Your old widget resources page redirects to your new home page:

http://www.widgetwarehouse.com/resources/ >> 301s to >> http://www.widgetwarehouse.com/

The Right Way

Your old red widgets product page redirects to your new red widgets product page:

http://www.widgetwarehouse.com/redwidgets/ >> 301s to >> http://www.widgetwarehouse.com/products/red-widgets/

Your old widget resources page redirects to your new widget resources page:

http://www.widgetwarehouse.com/resources/ >> 301s to >> http://www.widgetwarehouse.com/widget-resources/

Block Search Spiders From Your Beta Site

The beta version of your new site should be blocked from any search spiders so they can’t spider that content. If search spiders are able to access your beta site and spider that content, this could become a huge problem with search engines displaying incorrect results and could potentially hurt your rankings.

You can implement several different measures to ensure spiders do not spider your beta site:

A robots.txt File

A robots.txt file is a text file that’s placed on your root domain that tells search spiders which directories they should not spider.  You can disallow specific pages or entire subfolders. A robots.txt file that is disallowing several different elements looks like this:

User-agent: *
Disallow: /features_terms_service.php
Disallow: /temp/
Disallow: /beta/

As you can see, each specific page or entire subfolder call-out needs to be on a new line.  You can create a robots.txt file with the help of Google in your Google Webmaster Central account.

A Robots Meta Tag

On each beta site page, you can include a meta tag that tells search spiders to not index the content on the page and to not follow any links on the page. This meta tag is placed in the code right along with any other meta tags in the section for the page.

Each beta site page needs to have this robots meta tag. The specific tag to use is:

<META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW">

We recommend adding this meta tag on every page of the beta site just to be sure robots aren’t spidering any of that content.

Lessons Learned

A couple of things go over before you flip the switch for the new site:

• Make sure all of your new pages do not have a robots META tag. It’s easy to keep this tag in place and forget about it when you’re moving your beta content over to the new website
• Update your robots.txt file
• Flip the switch during an “off” time. This typically means you would want to make your site transition in the late evening Pacific time / early morning Eastern
• Pump some extra money in your “Branded” PPC campaign just in case there are any snafus
• Keep an attentive eye on your analytics to see if there is an influx of visits to your 404 error page. If there is, dig deeper to see where these visits are coming from (an internal page or external) and fix the problem
• Create a custom 404 error page with a “mini site map” that points to the major sections of your site

What To Expect

Expect your search results to just be a bit funky the first six weeks after flipping the switch. It takes time for search engines to spider all of your new content and get rid of your old data. During this time, you’ll see some irregularities with your search rankings; old results will show up for a search and the next might yield a result for your new site.

Resources

Robots.txt File – http://www.robotstxt.org/robotstxt.html
Robots Meta Tag – http://www.robotstxt.org/meta.html
301 Redirects – http://www.google.com/support/webmasters/bin/answer.py?hl=en&answer=93633