So I purchased hosting, copied all the files and uploaded them on to the server, edited the web.config (it was .net) and it all appeared to work without too much trouble.

Next I added a new design and everything seemed to run smoothly

After that I verified the account in Google Webmaster Tools, sat back and waited to see what happened. I checked back and in a few days or so all the pages bar 1 had been crawled and indexed even without any links pointing at the site.

So what happened, why where all the pages indexed in Google, bar the most important homepage?

I checked the source code in the browser for non index elements, checked for any canonical mistakes, checked the history of the site to see if the site had been black listed, but couldn’t figure out the issue. I didn’t think the fact that the content had been on another domain and was non on a new and weak authority site would stop it from being indexed, particularly as only 1 page hadn’t been indexed.

So after a week of scratching my head I decided to add the site to Google Webmaster Tools and verified that it was mine. Next I popped into ‘Diagnostics’ and then clicked on the link ‘Fetch As GoogleBot’ to see if Google was picking up a problem that I wasn’t. When I clicked ‘Fetch’ an error message came back with ‘Fetch Status’ unreachable.

So there it was, GoogleBot was obviously having a problem accessing the page, yet when I checked it in every browser going and in mobiles it worked fine.

Next stop was to download the user agent plugin for Firefox. Switching the User Agent to Googlebot 2.1 and trying to access the site brought up the issue, a nasty error message.

Within the source code of one of the template pages that I had written a very long time ago and not checked when quickly moving the site over, it showed that if the Googlebot spider visited the site then a certain bit of code would be run. The file path to the database file into which the information was to be stored had been hard coded, and so when Googlebot visited the old site the code ran fine but with the new site the file path was incorrect and the code didn’t!

Anyway lesson learned, check to make sure Google is seeing what you want it to see, particularly if it’s a new site.

The weekend, possibly a Saturday night and a date in January would suit.

Update – Meetup is on Saturday 21st January @ 5pm in Eglantine Inn on the Malone Road.

However there’s alot of poor nasty looking link farms, that just don’t offer any value whatsoever or have any unique content.

Recently I was looking at a website for a client. His site has been SEO’d by the notorious ‘It’s Cold Outside’ company. There’s a whole host of disgruntled customers on the web that have complained about keyword stuffing, excessive link exchanges, cloaking and other tactics that are outside Google’s guidelines.

Well when I looked at the client’s site, and checked his back link profile with Yahoo Site Explorer it was quite clear that this client’s site was on one of their link farms. I looked a bit deeper into the farm to see whether it was worth the investment that he was currently paying. I went through a 100 or so of the links that Yahoo listed and checked them in Google using the site operator. Website after website had been de-indexed in Google.

This link farm had obviously been discovered and penalised. The value of the client’s presence on these sites was absolutely nil. No doubt the company in question would obviously have been aware of this, yet the monthly payments were still recurring. Why should the client still be paying to be on a failed link farm?? I presume that any additional SEO they offered to make up for this, just amounted to an automated report with very very little work involved. I couldn’t see any continuing on site work, in fact even basics such as stats/analytics tracking weren’t given as far as I’m aware.

So what does this tell us:

1. In the long run simple link farms and schemes aren’t a great strategy, they’re a house built on sand. If you’re using an SEO company that is more interested in maximising their return and is quite happy to cram your site together with totally unrelated sites, then you’ve got no long term winning strategy.

2. If you’re a business person then make sure you know what your SEO is up to and you’re not getting ripped off. Clients were paying £100 a month recurring to be on this link farm. Although not a big budget, with every link in this farm devalued and deindexed in Google that is a complete waste of money. Talk about not getting your money’s worth, and the SEO company in question is presumably doing very little.

3. Even though many link farms go undiscovered for years chances are they’ll get walloped by Google.

4. Automated SEO solutions tend to be cheap and nasty for a reason. Not saying that they don’t work, indeed this farm probably worked for a while. Build your own custom link profile and it’ll have a better chance of lasting, don’t follow what everyone else does, and where everyone else is, otherwise it’s more likely to fail.

5. SEO isn’t just about ranking for your trophy term. Being top of Google for 1 keyword term isn’t the be all and end all of SEO. A more rounded internet marketing approach is better and covers more bases.

6. It’s important to note that being on this link farm didn’t have a negative effect on the client’s web positioning at all. All the constant forum questions about can being listed on a link farm hurt your rankings, well from this case study it’s pretty obvious that Google is clued up on this, and no negatives were accrued to the client’s site. The old adage that there’s not much that a competitor can do to take out your site holds up in this case.

The reason why this client didn’t complain was that he thought he was getting his money’s worth. Reality was that he was in a low competition market and that without the SEO’s efforts he presumably would have been no worse off ranking for the primary keyword.

How can you check if your backlinks count for nothing?

First of all pop over to Yahoo Site Explorer, and type in your domain name and click the explore button.

Next click the ‘Inlinks’ button, and from the dropdown ’show inlinks’ select ‘except for this domain’.

You should see results like the screenshot below.

Copy the domain name, head over to Google and type in the domain with the site operator e.g. site:domainname.com – If nothing is returned then chances are that this has been deindexed. Repeat this. If you notice most of the domains aren’t showing then best speak to your SEO as it looks like you have a problem.

For further reading check this out:
http://www.ukbusinessforums.co.uk/forums/showthread.php?t=98143&page=1
http://www.ekmcommunity.com/showthread.php?5366-Its-*********-SEO-Beware!
http://aerin.co.uk/how-to-avoid-the-seo-link-building-scam/

On the forums you’ll always get the usual throwaway one liner response that directory submissions are a complete waste of time, and money in the case of paid ones.

Some of the arguments go along the lines that years ago Google clearly devalued this style of site when it took a swipe at some directories engaged in tactics outside Google’s guidelines and reduced their page rank, rankings, the number of indexed pages they had and their ability to pass authority.

Other arguments stress how Google removed their crystal clear recommendation to submit your site to Yahoo and DMOZ and other relevant industry related directories.

So why did Google remove the recommendation and why did they take firm action against some high powered directories?

I’m guessing the reason Google acted was more to do with them trying to put the genie back into the bottle. The amount of poor quality general web directories that sprung up with the intention to pass page rank and anchor text relevancy rocketed almost into orbit.

It seemed like every web script wonder set up a general web directory running off a fairly cheap PHP script and poor template design and began running free and cheap directories with the sole purpose of ranking sites and creating manufactured links. Google’s recommendation of submitting in a selective manner merely fuelled the frenzy.

Arguably Google’s action taken against directories was merely against the abuse of their guidelines, not against the directory style site itself.

With Google’s crackdown some SEO’s abandaned their crap directories and general directory submission fell out of favour, with many moving onto the next method and dismissing directories as an ineffective link building method.

So are they worthless? And if not why not?

Absolutely not. Like I said there are quality directories, gems that just need to be unearthed. General directories that provide some editorial review, that don’t sell links for page rank, that aren’t covered in adsense, that don’t approve all and sundry, that are run and promoted with a passion and committment, can still provide and send quality signals to Google. Niche, regional and industry related directories can also be very valuable resources and places to be listed on and and in many cases haven’t been abused as much as the cheap general web directories form.

From my own experiences niche coding directories such as aspin, hotscripts, 411asp.net provided excellent traffic, paying customers for products, deep links and rankings. Almost everything you could wish for in a directory.

In some cases directories can outperform your site, especially if your site is newish, just doesn’t have enough clout or suffers from a poor structure. If a directory is going to outperform your site, then the next best thing is to get listed in the directory so that when someone visits that directory through a search on Google for your main keyword they’ll then pass through the directory and your listing and onto your site. All you have to do is run a local search on Google for a product/service and notice the amount of directories, though this may be to a lesser extent now with Google places merging with the organics.

Like I said, you should never ignore directories that appear at the top of Google for your keyword searchs. Though this might seem obvious it’s not to some. One of my directory sites sits right at the very top of Google and has done for the best part of 2 years now. It’s a fairly moderate term, though I’d like to think it’s a quality niche directory. Anyway there are a number of companies paying up to £2 for a pay per click in this market, while at the top of the organics (a market where there are no Google Place listings) most of these companies don’t bother with a 15 minute free submission that could save them hundreds and hundreds of pounds in advertising revenue and get them relevant traffic. It’s a no brainer, yet most of the companies don’t bother. It almost seems criminal.

Like so many things in SEO, experiences and opinions are made without testing, without qualifications, without context and this simply clouds the picture and message. It’s important to be aware that some people group all types of directories together, making no distinction between poor quality general web directories, niche industry directories, regional directories and so on. Directories are just bunched and labelled together and dismissed out of hand. Others will make a distinction between ‘web directories’, niche, regional and business/industry directories. So on forums if someone refers to ‘web directories’ as being useless they may not necessarily be referring to or dismissing quality niche, local or indeed all web directories en masse, they may simply be referring to the mass of crap web directories on the net.

Directories still provide a backlink to your site, many are still follow, and many directories will pass page rank and anchor text relevancy. Getting listed can improve your overall link profile, crawl rate and send a clear trust signal to Google.

I don’t advocate the mass directory submissions to general web directories that comes with services such as Directory Maximiser. For buttons you’ll get a run of hundreds of directory submissions to unmanned directories with little value. There are many SEO’s that suggest this method still works, and that quantity and mass quantity at that still does work, though no doubt that’s in fairly weak markets.

Do you still submit businesses to General Directories?

Yes, I still selectively look for quality general directories, business directories, regional and niche directories. I think they can still provide a backbone of a linkbuilding campaign and aren’t as time consuming as other linkbuilding tasks.

Aren’t these very expensive, is Yahoo directory worth the money ($299)?

The Yahoo directory doesn’t provide a great deal of traffic for businesses in the UK and Ireland. Some clients I had were lucky to get more than 5 visitors a year, and the SEO value was debatable. So if you are on a limited budget then creating content, getting a content writer to help you and trying to rank for keywords might be a better investment.

If you have a bigger budget, beyond that of most small businesses, then I’d probably recommend it but I don’t think it provides great value. I believe the UK & Ireland version of directory itself has now closed.

Is Yell worth the money?

If you have the money and have Google and other advertising and marketing well covered, and have a product that someone might use Yell for like a trade then possibly. As far as I remember Yell standard listings rotate and prices vary depending on the category. In the listings that I have managed there has been little traffic that comes through them. I suspect that the traffic isn’t great quality based on the bounce rates, and that some of it is casual competitor traffic. I think there would be more value pumping your money into a Google PPC or organic campaign if you were on a restricted budget if there was still room for improvement in Google.

If you’re already listed in Yell you can always use your analytics software and work out how much a cost or conversion costs and tally this up against Google Adwords or other forms of online marketing.

I wouldn’t entirely dismiss some of the bigger free UK directories such as the FreeIndex & Hotfrog. The 10 – 15 minutes that it takes to fill these in can provide a return for small businesses. In low competition markets you can dominate the first page for some keywords with these business listings.

Is Submitting to directories that charge, a paid link of the type Google dislikes?

From my understanding if the payment is charged for legitimate reasons such as business expenses, reviews, and editing costs as opposed to being a payment solely to pass page rank and relevancy with the sole purpose of improving your rankings on Google then it’s fine. Google has other forms of spam and abuse to monitor and paid directories seem to be exempt or not at the top of their priorities. You can watch the video on what Matt Cutts has to say.

Final Thoughts

I doubt you’ll find very few SEO’s that would overlook directory submission as one of their backbone strategies for starting off a link building campaign. For me a quality site is a quality site whether it’s a directory, blog of informational site. A quality web page is always worth getting listed on. Rather than give you a list of directories, learn how to spot the signals that make a site a quality site and webpage, and you’ll have a fair idea whether a directory is worth the submission and money.

The signals are still the same as they always were. Can you add 1 or 2 keywords into the title, are the links do follow, are pages structured and themed correctly, are the pages indexed and what’s the cache cycle, are the descriptions quality, is the page title in line with your site, are the links direct and does the site have the added bonus of sending you potential buyers. Directories for some SEO’s may have gone past their prime, but there’s still a bit of life left in them.

The facility is likely to be every students dream, with the ability to type in whatever can be found in the fridge, and lets Google concoct a delicious quick and easy meal to make Jamie Oliver jealous.

According to the search giant, about one percent of all searches performed on the search giant each day are for recipes. If you consider that they handle about a billion queries per day, that’s 10 million recipe searches. That’s a massive amount of hungry mouths looking for food online.

The search also lets you look at other additional data surrounding the recipe. You can find recipes from particular chefs, or indeed things as abstract as recipes for religious festivals.

From a technical perspective, Google have once again shown their expertise in the parsing of complex data, and the data mining and collation to launch this product is immensely complex. It also hints at the increasing move towards using structured data such as RDFa, and microformats that Google in the past while have begun to ask for from publishers. As and when they want to find out the information to create a new search feature, typically this is the format they ask for, which is more easily parsed by their bots.

Without this additional markup on the page, Google finds it immensely difficult to make a forray into a particular search vertical, and generate revenue from it. Other information they are clearly looking to receive include people (which Facebook and LinkedIn are both using to provide data) and events, hinting that a Google people search and Google What’s on may be services they are planning on somewhere down the line. Stay tuned for that one, undoubtedly one of the eagerly awaited services, with the most far reaching implications is just around the corner.

That said, Recipe search is a particularly nice improvement to Google search, and can be really useful to amateur chefs and pro’s alike. Take a look a the video below to find out more:

Today’s guest blog comes from Paul Anthony, who works as Internet Marketing Manager for Northern Ireland’s largest property website, PropertyPal.com. His role within the company includes managing all search engine optimisation, search engine marketing and social media activity.

PropertyPal.com is Northern Ireland’s number 1 property website, displaying details of more homes for sale and rent than any other website. It has around 95% of all properties for sale and rent, and at any time stocks over 40,000 properties to buy or rent. Including at time of writing approximately 1200 houses for sale in Bangor.

Should Google really lower the importance and ranking weight given to a keyword rich domain name, just because some SEO’s go fishing for keyword rich domains to use and abuse?

And are keyword rich domain names really as powerful as they’re made out to be?

This article is based on my experiences and opinions with keyword rich domain names, particularly over the last few years when I’ve acquired a small number of them for building genuine informational sites on them including this one.

The article isn’t about the advantages and disadvantages of using keyword rich domain names versus brand names, or using keyword rich domain names as part of an overall strategy of creating satellite sites to capture keyword related search and driving that traffic back to the main site, or 301 redirecting keyword rich domain names back to the main site, or how restricting a keyword domain can be. It’s primarly concerned with keyword rich domain names as a sole ranking factor, taken in isolation.

Low Level Competition Markets

Every man and his dog knows that in low competition markets keyword rich domain names are all you need to rank at the very top of Google, that’s a fact. Go and do some keyword research and find a low level keyword market, buy the exact domain, put up a site, give it a bit of time and although you might not get alot of traffic you’ll rank no1 for that specific term in a short time. Low amounts of traffic can work and convert very well sometimes particularly in a small niche, but that’s for another article.

Keyword Rich Domain Names in More Competitive Markets

The stiffer the competition, the more difficult it becomes to rank a keyword rich just on the domain name alone.

In a stiffer market I have an exact match keyword domain name, coupled with site content it ranks down on page 18. This is a competitive market, in the web design field but it’s not ultra competitive. So if an exact match keyword rich domain name was the golden ticket then I would have expected it to rank slightly higher than this, it didn’t. With a matching home page title, and decent content the site is no where to be seen.

Keyword Rich Anchor Text

As everyone in web development knows Google’s ranking pays particular importance to the keywords used in links (see section 2.2). So if I was to fire some links at this domain in this stiffer market, particularly with the anchor text being the same as the keywords I was targetting then no doubt it would shoot up the rankings. Nevertheless it would take effort.

Another aged exact match keyword rich domain name that I own, in a fairly competitive market ranks on page 2 of Google despite having a decent amount of links with keyword rich anchor text. The links aren’t great links though as they’re primarily article marketing links, but they should be enough for a page 1 ranking based on the authority of the current number 9 & 10 listings.

Keyword rich domain names aren’t the sole ranking factor, there are obviously other important ranking SEO factors at stake here.

A quick search on Google for keyword rich domain names, shows an article that Jill Whelan wrote way back in 2002, when guess what, keyword rich domain names where the topic of the conversation. What I take from this article, is that in many of these low level competition markets, a keyword rich domain name isn’t even needed to rank, an exact match keyword rich page title would suffice on it’s own.

Keyword Rich Domain Names and Relevance

Google looks at relevance, so why shouldn’t it pay that much attention to a domain name, particularly an exact match. And if it ignores or lessens the value of the domain name, then what’s next, Google ignoring the page title because SEO’s know that this is obviously one of the most important ranking factors. Should Google also ignore keywords in the file structure?

Whether you should go for a brand name or a domain name is for a different article, but keyword rich domains definitely help, though the further up the competition chain the more factors and SEO you’ll have to work on and not just rely on the domain itself.

Search on Google, for instance look at competitive terms such as car insurance – no keyword domain names here. Despite the lack of keyword rich domain names, I’d take a bet if a site given all the same SEO factors and everything being equal, age etc that a keyword rich domain name would be easier to rank.

I’m firmly of the opinion that they aren’t too strong. I think they work in low level competition, where a strong page title might even be enough. They are relevant and a strong signal as to the content and theme of the site, though not the quality. And if it’s you’re sole ranking strategy in a more competitive market then you’ll find out it isn’t the silver bullet, though it may help, brand considerations appear to keep things in check.

Am I right or wrong? Anyone else got any thoughts and experiences on the power of keyword domains? Randfish thinks keyword domains are far too strong, and from inside information believes that Google will turn down the dial on the importance of them, personally I don’t think Google needs to.

Whether this is a change to be rolled out permanently or just a test, I’ve no idea. It’s very early morning here, I was just trawling through Google and noticed the changes and once I got over the shock/surprise I decided to write a quick blog post.

For some localised searches it looks like Google has merged Google Places with the Organic Results though not for all the limited searches I’ve done. First Google did away with the 10 box listing, then they culled it down to the 7 box. Now in certain searches that 7 box listing have been removed, moving up the organic results to the top. The map is now placed on the right hand side, and there’s 4 or 5 pure organic listings. Who said Google organic listings were dead?

So this looks like a big win win for local sites that ranked well organically and that have a Google Places account.

So What happens to those that did well in Google Places but didn’t rank well organically?

Well one of my recent clients ranked in a 2 box listing for their main term, and on page 2 organically. Now the client appears to have been wiped off the first page of Google, leaving them with just a Pay Per Click listing on the 1st page.

The first page of screenshot for the search for ‘wedding photographer Liverpool’ above looks less cluttered, now it’s down to 10 or 11 results for certain searches in the main center column.

Different Displays in Different Markets

On another travel related search Google Places is still there but the Google Places listings are formatted slightly differently and the map has been removed to the side.

This is certainly one to keep an eye on. No doubt there’ll be more changes to come. Happy Halloween from Google, hopefully these changes won’t spoil the celebrations.

That’s where advanced search operators and combinations come in handy.

With the right combination of search operators and keywords, you’ll never have to dig through the SERPs for the gems again.

Search Operators and How They Work

There are tons of search operators out there, each one refines the search results just a little differently. Here are some of the ones you’ll find you use the most:

• ["keyword"] – Quotes tells Google to look for that exact word or phrase
• [keyword site:example.com] – searches that site for a particular word
• [keyword site:.co.uk] – searches only .co.uk sites for your keyword
• [keyword1 -keyword2] – Returns results that include keyword1 but NOT keyword2 (exclude)
• [keyword1+keyword2] or [""keyword1 keyword2""] – only returns results that include the phrase exactly how you typed it
• [keyword1 OR keyword2] or [keyword1|keyword2] or [keyword1 keyword2 OR keyword3] – Returns results with either of those results. For results that include both keywords, leave out the ‘OR’.
• [keyword1 * keyword2] – if you’re looking for information, but aren’t quite sure about the terms, you can use a wildcard and Google will return various possibilities. For example, [top * web design companies] also [keyword1*.co.uk]
• [~keyword] – Brings up similar terms and synonyms
• [link:example.com] or [linkdomain:example.com] – Brings up pages linking to that URL though note Google will only show a small subset of links
• [inurl:keyword] or [intitle:keyword] or [intext:keyword] or [inanchor:keyword] – Only brings up results with the keyword in the url, title, text, or anchor text

Examples include:

1. Look for guest blog opportunities for an SEO Consultant – [seo consultant "guest blogger" OR "guest blogger wanted"]
2. Find blogs in the Irish market related to web design – [intitle:web design inurl:blog site:.ie]
3. Find web design sites that are built on wordpress – [intitle:web design "powered by wordpress"]
4. Looking for a photographer in Ireland – [photographer +Ireland site .ie inurl:blog inurl:photography]
5. Test out competition in your market – [intitle:web design inanchor:"web design"]

Using Combinations of Search Operators

The search operators listed above do a great job of refining the search results, but if you’re still finding lots of junk, you’ll want to use combinations of the operators listed above.

So, if you want to search for certain types of sites linking to a particular URL, you can use [link:example.com site:.gov] or for those using certain anchor text [link:example.com inanchor:keyword] or [link:example.com site:.gov inanchor:keyword]

Want to find out who links to other sites, but not yours? Try [link:example.com -link:yoursite.com]

Looking for a place to submit your posts? Try [intext:"submit a new story" intitle:register]

Maybe you want to look for niche directories? Use your keywords with common phrases found on these sites. Try variations of [keyword +"intitle:directory"] and [keyword +"submit url"].

Find Dofollow Blogs

A large number of webmasters and SEOs are always on the hunt for relevant dofollow links. So, how can you find them? Simple! [keyword inurl:ifollow*.gif] or [keyword ifollow*.gif] in Google Image Search will reveal all the relevant blogs using the “U Comment – I Follow” badge. Alt tags leave a footprint. If you go to Google image search and type in your keyword plus “U Comment I follow” with and without quotes you’ll find a list of do follow blogs.

[keyword +"Commentluv Enabled"] or other variations such as [keyword +"Enable Commentluv"] will look for relevant sites with the Comment Luv Plugin. ([keyword +"Comments protected by Lucia's Linky Love"] works for that plugin too!). [keyword +"last blog post"] will help you find blogs that post a link to your last blog post (not necessarily dofollow). You can do the same with press releases by using [keyword intitle:press +release "allowed html tags"]. Use [keyword +"KeywordLuv"] for the Keyword Luv plugin. You could also come up with a search to scout for the “Top Commentators” plugin.

There are many more options out there and it’s always best to test them out and see what works for you and what doesn’t. Ann Smarty has a great piece on using the Google wildcard operator for link building and a good collection of link building search queries as well.

Other reading includes:

http://www.seomoz.org/blog/long-list-of-link-searches
http://socialseo.com/getting-crafty-advanced-search-operators-to-find-the-best-backlinks.html
http://searchengineland.com/21-link-builders-share-advanced-link-building-queries-29848
http://soloseo.com/tools/linkSearch.html
http://www.seobay.com/tools/link-suggestion/

Advanced Search Operators Are Not Just For Link Building and SEOs

It doesn’t matter if you’re looking for information, working on a school project, or simply looking to satisfy your curiosity, advanced search operators and footprints are a great way to find what you need without wasting time.

Simply by using the right combination, you can find virtually anything without having to dig. What combinations have you found helpful?

RMS Titanic (Credits)

I noticed that one of the sites had gone from the very top of the results to page 1 – 2 and sometimes page 3 for one of the markets that I keep a close eye on. The site seems to fluctuate if not on a search by search basis then a day by day basis. The positions are fairly stable for other sites as it’s not a fast moving and nothing exciting really happens.

My first thought was that something had been changed on the site, possibly a tweak to the page title, and that the site would bounce back right up to the top, so I thought nothing of it.

A couple of weeks later and the site was still bouncing about from spots 11 to 30 so I decided to keep a closer eye on it.

I hadn’t visited the site, just looked at it in the SERPS. Next I thought that maybe a few of the site’s links had been dropped, but when I did visit the site, the reason the site had dropped in Google stuck out like a sore thumb.

In it’s own block totally separated from content was a text ad. With keyword rich anchor text and totally unrelated to the site there’s no doubt that Google would view this as a paid link. Not even a no-follow. What’s worse, there were more than 1 text links.

I decided to look for a footprint, and searched Google for text to see if this had been a large volume campaign by the buyer. Yes the text of the ad was appearing on other sites, and other unrelated sites. I just quickly worked my way through these and it was noticeable that some of these had taken a hit and dropped down the search engine ranking positions not for their brand but for the keywords they targetted.

One other of the sites in question that I looked at in a bit more detail had bombed into no mans land for it’s own unique page title. The site had decent authority but didn’t rank anywhere near where you’d expect it to.

Moral of the Story

Moral of the story, don’t sell links on your main business site. Know the risks involved and what it might mean for your business. For 20 – 30 quid a month, if you offer a service and rely on the business generated from your website’s rank then you are running a risk that doesn’t way up. If you are going to tempt fate then at the very least make sure it’s related.

Interesting the company in the competitive market that was obviously purchasing links ranks in the top 3 in Google.co.uk for the term that they were targetting. The big company that possibly made approaches to these website owners have broken Google’s guidelines, and yet the small business owner, the guys that rely heavily on the business that Google Search offers have been deemed to be the offenders and hit the hardest.

Recently I was running an eye over the SERPS and checking out the rankings of one or two local companies. One of the listings stuck out like a sort thumb. All the other listings on the first page had page title’s with their company names, targeted keywords and a fairly comprehensive description. The site is question just had a short URL reference. Figure 1 below (just a mockup I stitched together) shows the difference between 2 listings. To hide the company’s embarrassment I’ve replaced the URL in the first one with my own, but you’ll notice the difference in a listing and a short reference.

Figure 1

A quick check of the source code revealed 2 interesting and conflicting lines in every pages’s code.

<meta name="robots" content="noindex,nofollow" />
<meta name="robots" content="index,follow" />


In a nutshell the first line basically tells Google to de-index your listing, and the second line instructs Google and other search engines to index the page. Judging by how Google followed these directions, the first was given priority and the site was de-indexed. Arguably when there a conflict’s and an obvious indexing mistake has been made Google should ignore both directives and include all pages in it’s index.

What Went Wrong

The site in question was running on Wordpress. My guess is that the site had been redeveloped, and that the privacy settings on Wordpress had been set to dissallow search engines from indexing the site while it was being redeveloped online. When the new site was moved across they forgot to take the privacy setting of, and the noindex remained. I’m not quite sure how the additional robots tag got in the code, maybe it was inserted manually.

Anyway, I fired them off an email but it wasn’t acted upon.

A couple of weeks later when I rechecked I noticed that the penalising code was still there. The homepage was still showing a short URL reference, all the other pages had dropped from the index, and the home page was no longer on the first page of Google, it had slipped down to second page siberia. Traffic through the search engines now must be a trickle.

I’m not a web developer, how can I tell on my own site?

On your browser toolbar, click view source and look for the robots tag, like the code shown above. If it’s noindex then this might start the alarm bells ringing, time to follow up with the developer.

What Can Be Learned

• Following a redesign or just going live, check that there is nothing stopping your site from getting indexed correctly.
• Keep an eye on Google Analytics or your stats software for any unexpected dips in traffic.
• Check through the SERPS occassionaly for your main keywords.
• Run a site command, site:yoursite.com and inspect your site’s listing. This is also great for noticing other things including duplicate content.

On a final point Google, Yahoo and Bing all showed a short URL reference. Ask was the only engine that didn’t even list a URL reference, it totally de-indexed the site in question. Not entirely sure what is the correct interpretation of the tag, but arguably with noindex the main engines shouldn’t even show a short reference.