Remove Spam from Google Blog Search

Even if Google Blog Search doesn't have too many interesting features, I still use it more often than Technorati because it's faster, it's not down for hours, it's much more comprehensive and it has features not available in any other important blog search engine. I still use Technorati for finding backlinks, because Google does a poor job in this area (compare Technorati with Google Blog Search). Unfortunately, Google Blog Search indexes a lot of spam posts that steal content and use it for lucrative purposes.

Google has two features that reduce the number of splogs (spam blogs) from search results. Like in web search, there's a duplicate filter that removes some of the posts that are almost identical. But it doesn't exclude all of them and it doesn't find posts that duplicate articles from news sites like Business Week.


The second feature is the option to sort results by relevancy, which is enabled by default. It may seem counterintuitive to sort blog search results by relevancy and not chronologically, but that's a great way to filter splogs or at least move them at the bottom of Google's search results. Google uses a lot of signals to rank blog posts, including PageRank, the number of feed subscriptions or the amount of duplicate content. But if you sort the results by relevancy, you'll find both recent and old posts and that's not always the optimal solution. A better way is to restrict the results to a recent period of time in the sidebar (to the last day or the last hour, depending on the volume of posts).


If you see a "References" link after the snippet, that's an indication that Google found (a significant number of) backlinks, so the result should be a little more reliable.

Many blogs use Google Alerts to pollute the web and make money, so you could also add [-"google alert"] to your query (a search for "google alert" returns more than 200,000 results). A lot spam blogs are hosted by Google's Blog*Spot, so removing the posts from blogspot.com could increase the quality of your results, but also remove non-spammy blogs like this one or Google's official blogs. I also noticed that many spam blogs use the .info TLD. A recent study showed that, when searching for commercial keywords, 75% of the results from blogspot.com and 68% of the results from .info sites are spam.

It's also a great idea to restrict the result to English (or another language) in "Advanced blog search".

So here's a summary:

1. sort the results by relevancy
2. restrict the results to a recent period (last day)
3. restrict the results to English (or another language)
4. if you really have to sort the results by date, remove the posts that follow a spammy pattern (for example, add -"google alert" -site:blogspot.com -site:.info to your query), but make sure you don't remove important results
5. check the posts that contain "References"

Google should do a better job at detecting spam in Blog Search results and identifying results from sites that happen to have feeds, but they're not blogs. It should also make it more difficult for spammers to use sites like Blogger or Google Alerts to pollute the search results.

The Supercomputer that Connects Everything and Everyone

One of the best articles ever written about Google is "Google. Who's looking at you?" from the The Sunday Times. The article tries to understand Google's latest projects by looking at what Google really wants to do in the distant future. And while reading the plan, you'll realize that Google's power from today doesn't mean anything compared to the envisioned plans. Here are some interesting quotes, but the article is too good not to be read entirely (my emphasis):

[Craig] Silverstein, now director of technology, explains that, from the earliest days, Brin and Page envisaged a super-connected computer. "The vision of search has always been broader than has been portrayed in the press," he says. (...) He recalls one example that shows that Brin and Page imagined that one day even the smallest "stuff" would be online. "When we were doing the first research, we used to eat in Whole Foods [an organic supermarket chain]. We talked about using search to find out what aisle the salt is on. Instead of having to look at the big signs at the top of each aisle, you could use a search engine to tell you where in the store everything is, and maybe graph it out for you."

Brin and Page were obsessed with recording, categorising and indexing anything and everything, and then making it available to anyone with internet access because they genuinely believed - and still do - that it is a morally good thing to do. It may sound hopelessly hippie-ish and wildly hypocritical coming from a couple of guys worth £10 billion each, but Brin and Page insist they are not, and never have been, in it for the money. They see themselves as latter-day explorers, mapping human knowledge so that others can find trade routes in the new information economy.

Google's plans have three parts: bringing everything online and to make it searchable (universal search), indexing personal information about users (personalized search) and creating new things online using Google's web-based software (cloud computing). The idea is to bring everything online, with various levels of privacy, making it accessible using search and useful from Google's web apps.

Instead of worrying that they are going too far, Google's top team talk, with poker faces, about a "300-year mission" that will eventually see almost everything - including, perhaps, one day you and me - linked to the web and searchable online. (...) Google's techno-dream comes in three bytes. The first is loosely referred to as "universal search". Scribbling frantically on a whiteboard, Mayer, Google's head of search products and user experience, says the web is currently "very limited and primitive". It consists mainly of words, images and some music, mostly created in the last few years. There is much, much more that could - and should - be online. At its simplest level, this includes every film, TV show, video or radio broadcast ever made; every book, academic paper, pamphlet, government document, map, chart and blog ever published in any language anywhere; and any piece of music ever recorded. Google is currently developing new software that will scan millions of new sources of information to give richer search results. (...) Mayer and co argue that to be true to its mission statement of "organising all the world's information and making it universally accessible and useful", Google should be about more than searching for words, images and music; it should be about finding objects and, eventually, people. Any item that can be fitted with a radio-frequency identifier - an electronic tag called an RFID - can be linked to the internet over local or national WiFi networks.

But once you have everything online, how do you link the objects from Google's index with those who need to find them? How do you understand what they want to find if you don't know anything about them? In most cases, the same question shouldn't generate the same answer for everyone.

The second part of Google's techno-dream is "personalised search". (...) Google wants us to sign up for iGoogle on our PC, and also to install it, along with Gmail, Google Maps and Google Earth software, on our mobile phone, so that it knows not just who we are but where we are in the world, 24 hours a day, thanks to the satellite-positioning chips starting to be included in mobile phones. "Our goal is that you can, if you want, search for anything, anywhere, any time," says Douglas Merrill, 37, Google's chief information officer.

To make the information useful, you may need to use it in some context or share it with other people. Google's apps should make this happen from any computer or device connected to the Internet.

The final piece of the Google future is called "cloud computing". Instead of using the internet to search for information that we then copy and use to work on documents stored on the hard drives of our computers, using the software on those computers, Google wants us to create all our documents online, to work on them online using Google's web-based software, and to store them online on Google's vast global network of servers.

By looking at these placemarks from Google's future map, it's clear that search is the most important Google service and every new addition expands the index with more information. It's also obvious that search will propel Google's apps, which become some collectors of search results and channels for communicating them.
If this will happen and Google will become the universal ministry of search and communication, the future will tell. For now, Google seems the only company that has the resources and the desire to conquer the world at the informational level, but that's no guarantee for success.