On July 17th, Plussem.com launched their website offering to provide fake +1s for your website for a fee.

Like buying links, this service offers fake user endorsement of your site – potentially improving your ranking in the Google search engine.

It’s hard to know if Google will be able to detect this without knowing how Plussem actually generate the +1s? 

I’m sure Matt Cutts and the Google web spam team are working on ways to detect and ignore these fake endorsements, but if the spamming grows, it may affect how much +1s affect the ranking signals.

When Google added Instant Previews to user search results, there were questions as to how and when the images were created:

Google have now added an additional feature to the popular Webmaster Tools (under Labs) to help website owners find issues which may be affecting the preview image generated by Google.

The tool gives you the option to specify any web page on your site and then generates a preview on the spot.   It compares the preview to the actual site side by side:

As you can see above, there are some small discrepancies (some borders/menu elements are missing), however this appears to be fairly random?  

Repeatedly hitting the compare button gives different results – possibly indicating a bandwidth issue when downloading pictures?   Maybe because so many webmasters are hitting the new tool at the moment!

Below the comparison, the tool provides a list of errors encountered when processing the site.    In the above example, the missing bitmaps are listed as “fetch failure”, which is odd as the images download fine for normal browsers.

This also raises the question – if the image on the left of the actual site is also generated in real time for the comparison – how come it succeeds to download all the pictures, whereas the instant preview doesn’t?

Google states that the image preview is generated using a WebKit engine which doesn’t support flash or Silverlight which may account for differences between the real site and the preview.

Check out the Google webmaster central blog for the official Google announcement.

History of Robots.txt

Photo Credit http://www.flickr.com/photos/tamaleaver/

If you’ve never heard of robots.txt files before, they’re simply a  text file placed on a web server containing a set of rules to control how web spiders crawl a site.

Developed in 1994 by Martijn Koster, robots.txt files can now be found on most websites, however they’re rarely viewed by humans.

However did you know there’s a lighter side to these simple files?

For example, Malcolm Coles recently made a discovery when analysing the Daily Mail’s robots.txt file. Hidden inside was this secret message:

# August 12th, MailOnline are looking for a talented SEO Manager so if you found this you’re the kind of techie we need!

According to Malcolm as of August 25th, his post had received 5,500 views, so I suspect the position at the Daily Mail was quickly filled!

.

Google  Zombies

Photo Credit http://www.flickr.com/photos/kwl/

In October 2008, Google slipped a hidden message into the rules of their robots.txt file for Halloween:

User-agent : zombies

Disallow : /brains

The change was covered by Google’s Web Spam expert Matt Cutts in his personal blog  

http://www.mattcutts.com/blog/google-protects-itself-from-zombies/

Robot Reforms?

Photo Credit http://www.flickr.com/photos/dcjohn/

Now moving on to politics, did you know that when the Obama administration moved into the White House, they pruned the www.whitehouse.gov robots file down from nearly 2400 lines down to just:

User-agent: *

Disallow: /includes

Today the file now contains just the following:

User-agent: *

Crawl-delay: 10

Sitemap: http://www.whitehouse.gov/feed/media/video-audio

As a comparison, you can view an archive of the original Whitehouse Robots.txt

Asimov’s 3 Laws of Robotics

Photo Credit http://www.flickr.com/photos/gladius/

Quite a common tactic with robots files is to include Asimov’s 3 Laws of robotics.   Both Yelp.com and Last.FM have included Asimov’s 3 rules for Robots in their files:

Yelp.com

#

# 1. A robot may not injure a human being or, through inaction, allow a

# human being to come to harm.

#

# 2. A robot must obey orders given it by human beings except where such

# orders would conflict with the First Law.

#

# 3. A robot must protect its own existence as long as such protection

# does not conflict with the First or Second Law.

Last.fm

Disallow: /harming/humans

Disallow: /ignoring/human/orders

Disallow: /harm/to/self

Hidden Whisky

Photo Credit http://www.flickr.com/photos/bignavijp/

Another find by Malcolm Coles!   This time on the Whyte and Mackay website.    W&M decided to run an online promotion in October 2010, giving away 250 bottles of 30 year old whisky worth £150.

The special bottles were hidden in stores across the UK, however the first person to read the robots.txt and send an email also won a bottle (don’t bother it’s already been won!).

You can see the robots.txt with the competition at Malcolm’s blog:

http://www.malcolmcoles.co.uk/blog/whisky-hidden-in-robots-txt-file/

Bender and Gort

Photo credit http://www.flickr.com/photos/josek/

Finally,we come to the robots.txt file for the site www.reddit.com.    Hidden in the text are instructions to two popular robots, Futurama’s own Bender (shown above), and Gort from the 1951 film, The Day the Earth Stood Still.

(photo credit http://www.flickr.com/photos/narcosislabs/)

Let us know in the comments if you find any other funny or odd robots.txt files!

Yesterday Yahoo! announced a new service called “Yahoo! Clues”.    The service provides trending information on Yahoo! searches.

You can access the beta using clues.yahoo.com

Yahoo! goes further than Google by giving you statistics based around demographic, income and location.

You can either see current trending topics or enter your own search terms to see trend patterns.

Google have recently announced that they’re promoting a new open source image format known as “WebP”.

Speed

Google’s main drive for pushing this new format, is because it uses their open source VP8 encoding technology to shrink images to use hypothetically 40% less space than a JPEG.

Over the last year we’ve seen several moves by Google to convince webmasters to speed up the web, however this has to be their most ambitious yet.    To gain traction, they’ll need support from all the latest browsers, and also from graphic design tools.

Advantages for Googlebot

As well as the known SEO advantages of speed on the web, there is another small feature of WebP which could be aimed at improving Google’s Search Algorithm.    The WebP format is designed to include meta-data about the image.   This could mean that images using this format are easier for the Googlebot to interpret – giving it a greater understanding of the scope of the page.   Instead of looking at HTML title tags, Googlebot could look inside the image to determine the context of the image.

Further Resources

Google WebP Blog Announcement

Google WebP Project

APIs are a great way of developing custom SEO applications.   By tying together these building blocks, you can quickly and easily develop tools such as our example free SEO site comparison tool.

Here’s our Top 10 list of SEO APIs:

#1 SEOMoz API

SEOMoz – a leading SEO Software company with years of SEO consulting background have developed their own link graph of the web – known as Linkscape.    They offer free access to this link graph via their API.   Offering access to back links, page authority and page ranking statistics, SEOMoz is number one in our top ten list.

#2 Alchemy Keyword API

Alchemy provide an advanced API based around term/keyword extraction.   Using advanced linguistic algorithms, the API is able to extract common phrases and is even able to infer the topic of the content.

#3 Google Search API

As the #1 search engine, Google provide numerous APIs to most of their online tools.   The AJAC Search API provides access to Web Search, Local Search and Multimedia Search.

#4 Yahoo Search API

Always popular in SEO, the Yahoo Site Explorer tool provides reliable backlink data on websites.   You can access this data via the Yahoo API.  The future of this tool could be limited due to the integration with the Microsoft Bing engine.

#5 Bing API

Microsoft’s search engine is growing in popularity and becoming increasingly advanced.   Microsoft have realised the potential of APIs by providing access to Web, image, InstantAnswer, Phonebook, RelatedSearch, Spell and more.

#6 Majestic SEO API

Another link graph of the web is provided by Majestic SEO.   Majestic have used their own crawling technology to provide an extensive data.   Through the free API you can gain access to the number of referring domains and the number of external backlinks to a site.

#7 Twitter API

Twitter, the popular Social Networking application provide extensive search access to their data and trends via free API calls.

#8 Compete API

Compete, a web analytics company, provide free API access to their site traffic and history statistics.   Compete also provide their own trust data based on surveys and analytics tracking.

#9 Backtype API

Backtype indexes comments from blogs, social networking and other social media.   You can access the resulting data via their free API.

#10 Amazon Alexa API

Alexa have been analysing site traffic for years via their free browser toolbar plug-in.   They’ve teamed with Amazon to give API access to this data.   Unfortunately, unlike the above APIs, access isn’t free however calls are very cheap.

If you’ve been watching Twitter over the last 24 hours, chances are you’ve seen at least one post from Twifficiency.    

The service analyses your followers, who you follow and your tweet history to generate a “Twifficiency score”.

As you can see from the following Google Trend – the growth of Twifficiency traffic has exploded over the last 24 hours:

Now that so many people have fallen for the novelty, people are starting to have second thoughts about the service – is it secure, could it be a scam, have I compromised my twitter account?

Where Did Twifficiency Come From?

Well firstly, here’s a bit of background on Twifficiency.    It was developed by a (now rather shocked) 17yr old web developer from Dundee in Scotland.   James Cunningham (@jamescun) describes himself as :

“17 year old Coder, designer, gamer and wannabe-entrepreneur. Fascination with random facts. Doesn’t always make sense to anyone other than myself.”

Is it secure?

Looks like it – yes!   According to James, Twifficiency is based around the open source oAuth which means that he has no access to your Twitter password.

If you’ve used Twifficiency and want to remove the connection to your account, use the following page on Twitter:

http://twitter.com/settings/connections 

It’ll be interesting to see what becomes of James and Twifficiency since becoming so popular on Twitter!

Rand Fishkin over at SEOMoz has just posted an interesting article about how Google appear to be showing increasingly spammy entries in the SERPs.

While I agree with Rand’s conclusions, I thought it’d be interesting to compare the results with Bing so see if things are as just as bad?

Rand suggested that the following search terms provide spammy results when searching in Google:

So I decided to run the same queries through both Google and Bing, then compare the results.

What I found interesting is that when there isn’t much of an intersection in the results, it’s usually due to Bing listing more local (and less spammy) results than Google.  

Take for instance the “Birthday Party Supplies” query.   Most results returned by Bing were to local suppliers.

The following Venn diagrams illustrate the overlap in the results.   Google results are shown as yellow and Bing results are shown as orange.

SEO Software

Starcraft 2 Strategies

Birthday Party Supplies

Currency Trading Online

Tennis Racquet Review

s

Leather Crafting Supplies

Nanny Services

Home Business Ideas

French Doors

Vietnam Tours

Antioxidant Supplements

Home Espresso Machine Ratings

Following the recent changes to the BBC news website there has been some talk in the SEO community about the affect this has had.

A tweet from @tomcritchlow of Distilled this morning had me wondering how the transition to the new site was going:

I first noticed that the PR of the new www.bbc.co.uk/news website is currently 0 – though this is understandable as the PR updates are fairly infrequent.   It’s interesting though that a 301 redirect doesn’t transfer the PR value of the original source page?

Next – I thought I’d take a look at the cached page that Google have for the new home page to see how recent it was.   Here’s the surprising result:

Odd?   Are Google not indexing the new home page?   A check of their robots.txt file didn’t show any problems?

Looking at the robots.txt file – I noticed that the news site has it’s on sitemap:

http://www.bbc.co.uk/news_sitemap.xml

Looking through the contents of this file – it looks like there are entries for most languages on the site – but oddly I could not see English in the list?

Is this an oversight or are the BBC just relying on in-links and internal linking to ensure that Google indexes the site.

Here’s the (albeit rough) stats from Google on the number of pages of each site in their index:

Quite a difference!   It’ll be interesting to follow this over the next week to see how the figures change.

What was also interesting was the pages shown in the index.   For the old site, the home page and the top category pages are showing top of the list:

However for the new site – only deeper article pages are shown, which matches what I saw earlier with the home page not showing in the Google cache.

We’ll have to keep an eye on the results over the next few days to see how the statistics change.    It’s certainly a brave move on behalf of the BBC to undertake such a widespread restructuring and maybe the results will help others in similar situations?

As one of the most popular news sites on the Internet, the the BBC News website has recently been updated.  

What’s interesting about this update though is that a lot of the changes seem to be SEO driven.   In November last year, BBC editors wrote a blog post explaining how they were already making changes specifically for the benefit of Search Engines.

Site Evolution

Lets start by looking at the evolution of the BBC News website over the years courtesy of the Wayback Machine:

BBC News Website 1998 -1999

BBC News Website 2000 – 2001

BBC News Website 2002 – 2003

BBC News Website 2004 – 2006

BBC News Website 2007 – 2009

As you can see, the layout has stayed fairly static over the years, with the main changes being to fonts, color schemes and additional graphics.

New Design

OK – so now let’s take a look at the new site:

The first thing you notice is that the categories are now across the top of the page, rather than down the left.   This gives the page additional real-estate for larger graphics, and more importantly longer headlines.

I mentioned earlier that the editors had blogged last year about how the site changes were SEO based.   In a post by Steve Herman he explains how they decided to make the headlines on article pages longer for the benefit of search engines (chasing the long tail?)

With the extra space gained by moving the categories to the top of the page, they can now extend the headlines as they did with internal articles.   There are also theories that links higher up the page are given more weight than those lower, in footers or sidebars (as on the original layout).  

Redirect

The next change is quite subtle and I doubt a lot of people will notice – the original home page at http://news.bbc.co.uk now 301 redirects to http://www.bbc.co.uk/news.     Is this a sign of the BBC trying to improve their www. site by sending to link juice and PR it’s way?

Social Media

The BBC have now embraced social media and have added the usual share buttons to all articles as you can see in the screenshot below.

BBC Editor, Steve Hermen said last November “A growing number of users come to stories on the BBC site from places other than our own front page – for example search engines, other sites, personal recommendations, Twitter or RSS feeds”.   

This change will make it much easier for readers to share news articles on the major Social Networks.

Link Sculpting

There’s more emphasis on links to other (in particular – popular) articles now – an example of which you can see below:  

More page space has been dedicated to related articles and analysis and there’s also a push to include more media with additional links to image galleries and videos.

By creating additional inter-article linking, is this a sign that the BBC are trying to spread the flow of link juice throughout the site?

Summary

Whether you like the new look or not will be a personal choice, but as you can see from above, it looks like the BBC is taking SEO seriously.  

So seriously, that their willing to use SEO as a reason to change the look and feel of a site which has been popular with readers for many years!