September 18th, 2008 Add Your Comments Bookmark and Share

Back in July of 2007 when I started themakemoneyonline.net I was very new to all things WordPress. I’d only ever used it once before, adding a blog to an existing website. That site was hosted on an IIS server so I was never really fussed about the SEO factors of using WordPress, I just needed a quick and simple content system. So in terms of getting a WordPress driven website to perform in the search engines this was my very first.

I do like WordPress. So much so that I’ve gone on to create probably another dozen websites using it as well as moving several old websites over. Yesterday though I discovered a problem that has been holding me back and it is something so simple that I’m positive there are plenty of people who have done the same.

Duplicate Content
Duplicate content is a simple 2 word phrase that can strike the fear of god into SEOs. WordPress’s biggest weakness (out of the box) is the high possibility of picking up a duplicate content penalty. Or at the very least having the wrong content indexed. This is where the robots.txt file helps. At the moment mine looks like this:-

User-agent: *
Disallow: /cgi-bin/
Disallow: /go/
Disallow: /wp-admin/
Disallow: /wp-includes/
Disallow: /author/
Disallow: */page/*
Disallow: /wp-images/
Disallow: /images/
Disallow: /backup/
Disallow: /banners/
Disallow: /archives/
Disallow: */trackback/
Disallow: */feed/
Disallow: /*?*

User-agent: Googlebot-Image
Allow: /wp-content/uploads/

User-agent: Mediapartners-Google
Allow: /

User-agent: duggmirror
Disallow: /

There is one vital thing missing…I don’t exclude tag pages! It’s because when I first started using WP there were no tags. Since then they’ve been added and I’ve started tagging posts across the majority of my WordPress websites. Depending on your template but for the vast majority, a tag page is given much more authority than the actual post itself (it’s all to do with the tag cloud in your template). This results in lots of non keyword targeted tag pages that although indexed don’t rank anywhere whilst your targeted posts end up ignored. It’s a lose-lose situation.

I recently started a new blog where this became apparent. Within the first week I had 20 pages indexed in Google. The home page, 2 posts and 17 tag pages. The home page brings a tiny amount of organic traffic (it’s still very new). The 2 posts that are keyword targeted and indexed are already in the top 3 results of Google for their keywords and bringing targeted traffic. However the tag pages that have been indexed are nowhere in the first 3 pages of search results, even when you search for their title. The posts that contain the content similar to what is on the tag pages aren’t indexed at all.

The really simple solution here is a 5 second fix. All I need to do is add “Disallow: /tag/” to my robots.txt file.

Related Posts

Paul B