Google WHOIS OneBox

Google shows a special OneBox when you search for "whois", followed by a domain name: for example, [whois google.com]. The OneBox shows the date when a certain domain was created and date when it will expire. It seems that the only provider of information for this OneBox is Domain Tools.

Google launched a similar feature four years ago, but it was removed really fast because it scraped data from Network Solutions without permission. "Google quietly launched a service allowing visitors to look up data on domain name owners from public databases - collectively known as Whois - run by registrars worldwide. Although largely unpromoted, the service generated enough traffic to surpass Network Solutions' (NSI's) daily Whois use limits, which aim to stop spammers and other undesirables from harvesting information about its customers."

This is not the only direct answer displayed at the top of Google's search results: there are many OneBoxes that show maps, stock quotes, weather information, local time, books, definitions or facts.

{ via Matt Cutts }

Update: after 3 or 4 page views, DomainTools shows this message "You have reached your daily lookup limit as an guest user. Please login or register". Maybe Google should partner with companies that have less restrictions.

Yet Another Google Video Redesign

Google Video redesigned its homepage and now only includes a list of "hot" videos. "The Google Video home page allows you to browse and play hot videos directly from the home page, making it easier for you to discover popular, interesting videos from across the web. The hot video list is compiled by looking at a variety of signals including videos that most shared, viewed and blogged about."

The video watch page has also been redesigned and it's more flexible: you can hide the right sidebar, minimize the list of related videos and use pagination to read a long list of comments.


Videos from third-party online video sites are still displayed in an annoying frame, but Google Video's bar has been moved at the left of the page to leave more room for the videos.


The search page has two new options for displaying results: in a grid and in TV view, that lets you watch videos while also being able to see the list of results. An interesting new option in the advanced search page lets you search only closed captioned videos. Another good thing is that you can now watch videos inline for some new video sites: DailyMotion, Revver, Guba, Crackle, not just for YouTube and Google Video.


The updates make Google Video more user-friendly and easier to use, even if mixing a video search engine with a video hosting site makes the user experience confusing.

Watch Restricted YouTube Videos

I've noticed that an increasing number of YouTube videos are restricted to a limited number of countries, probably because the company that uploaded them doesn't have global distribution rights or because it wants to use different marketing strategies in other countries. Even if YouTube says that "this video is not available in your country", you can actually see it using a very simple trick: replace http://www.youtube.com/watch?v=VIDEOID with http://www.youtube.com/v/VIDEOID (VIDEOID is the 11-characters video identifier).


Example of video blocked outside US (Madonna - 4 Minutes): http://www.youtube.com/watch?v=I9ciR9qR1dU
To see it, paste this in the address bar: http://www.youtube.com/v/I9ciR9qR1dU

Update (one day later): YouTube fixed the player and you can no longer bypass the country restrictions using this trick. Youtubegir has a nice proxy for YouTube videos.

The same trick works if you don't want to log in when you get this message: "This video or group may contain content that is inappropriate for some users, as flagged by YouTube's user community. To view this video or group, please verify you are 18 or older by logging in or signing up."


http://www.youtube.com/v/VIDEOID sends you to the player used by YouTube when you embed the video into a web page and this player doesn't perform country verifications and can't detect if you're logged in.

Finding the Right Signals to Rank Search Results

Udi Manber, VP for search quality at Google, talks in an interesting interview about search, personalization and the influence of social networks on finding the information you need.

Search has always been about people. It's not an abstract thing. It's not a formula. It's about getting people what they need. The art of ranking is one of taking lots of signals and putting them together. Signals from your friends are better signals, stronger signals. On the other hand, many searches are long-tail kinds of searches. If you're looking for what movies to see tonight, your friend can probably give you the best information. If you're looking for the address of the business, the Web as a whole can give you better information. If you're looking for something obscure about anything, again the web can give you much better information. It depends on the type of search you do and how to take all those signals and put them together.

The ranking algorithm is a recipe that takes into account many factors: an absolute value for the importance of a page, the relevance of the web page to your query, the location connected to the web page, the recency, the percentage of people with similar interests that found the web page helpful, the density of information etc. It's important to find the right signals, their importance and the contexts where you should apply them, but Google relies on engineers to adjust the recipe. At some point, Google will have to come up with an algorithm that automatically identifies potential signals.

"I found this surprising. Google manually comes up with tweaks to its search engine that only apply to a small percentage of queries, tests the tweaks, and then tosses them into the relevance rank? (...) Frankly, I thought Google was beyond this. Rather than piling hack upon hack, I thought Google's relevance rank was a giant, self-optimizing system, constantly learning and testing to determine automatically what works best," wrote Greg Linden in "The perils of tweaking Google by hand".

{ via Slashdot }

Google Maps Predicts Traffic Conditions

Google Maps can now predict traffic information for any day of the week and time of the day, based on past conditions. By default, if you click on the Traffic button in a supported area from the US, Google Maps shows real-time traffic information. "Comprehensive traffic data is available in over 30 major US metropolitan areas (including Los Angeles, New York, Chicago, and others), with partial coverage available in many more," according to Google Maps help center. There's also a traffic layer in Google Earth and Google Maps Mobile, but these applications don't include yet traffic prediction.


{ Thanks, Michael. }

Subscribe to Authenticated Feeds in Google Reader

Google Reader is one of the many online feed readers that don't support authenticated feeds. This special kind of feeds requires a username and a password before displaying the content to protect sensitive information. An example of authenticated feed is Gmail's feed for unread messages, but you'll also find password-protected feeds for internal bug reports, private email distribution lists, etc.

FreeMyFeed wants to solve this problem by creating feeds that don't require authentication. The site acts as a proxy between the original feed and your feed reader, while promising that your credentials are safe. "Usernames and passwords are never stored on the server. The usernames and passwords are only parsed to retrieve your RSS feed and then are discarded." Rob Wilkerson explains that the credentials "are encrypted using a rotating algorithm and included in the new URI."


It's not a good idea to enter your username and password in any other place than the site where you created them, but FreeMyFeed could be useful for feeds that are not tied to important accounts. Make sure you don't share any item from the generated feeds.

Google News Quote Finder

Google News can now detect quotes in news articles and attribute them to their authors. If you search for people like Eric Schmidt, Pope Benedict XVI, President Bush, Angela Merkel or Fernando Alonso, Google News will display a relevant quote and a link to other quotes from recent news articles.


You can search inside the quotes, sort them by date, restrict them to the last day or the last week. Unfortunately the links to quote listings aren't very descriptive (here's one example) and there's no option to find the quotes related to a news cluster. By default, Google sorts the quotes by relevance and gives more weight to the quotes that are used often.


{ via Google News Blog }

YouTube Search Enhancements

YouTube's search algorithms are increasingly smarter and borrow a lot of things from Google search: advanced operators, spelling corrections, related searches, query expansions. YouTube detects duplicate videos and shows the most popular copy in search results, followed by a link to the other videos. There's also an enhancement for videos that are split in two or more parts: YouTube displays a list of links to all of the episodes.

An Outdoor Campaign for Google Video

This very interesting outdoor campaign for Google Video Germany used a billboard imitating a real-life video player that captures the life as it happens. The tagline is "any film you can imagine", a simple message that encourages people to search for videos and to upload their own videos. AdFreak has an interesting explanation for the unusual idea: the see-through billboards suggest "that online video presents life in all its glorious randomness".

It's not very clear if the campaign was launched before or after the YouTube acquisition, but the fact that the video embedded below is from YouTube tells a lot about Google Video's success.

Google Earth 4.3 Adds New Navigation and Street View

The latest version of Google Earth brings a lot of interface changes and new features. There's a redesigned and improved navigation control that lets you change the perspective much faster. Here's the description from Google Earth's help center:

"1. Click the north up button to reset the view so that north is at the top of the screen. Click and drag the ring to rotate your view.
2. Use the Look joystick to look around from a single vantage point, as if you were turning your head. Click an arrow to look in that direction or continue to press down on the mouse button to change your view. After clicking an arrow, move the mouse around on the joystick to change the direction of motion.
3. Use the Move joystick to move your position from one place to another. Click an arrow to look in that direction or continue to press down on the mouse button to change your view. After clicking an arrow, move the mouse around on the joystick to change the direction of motion.
4. Use the zoom slider to zoom in or out (+ to zoom in, - to zoom out) or click the icons at the end of the slider. As you move closer to the ground, Google Earth swoops (tilts) to change your viewing angle to be parallel to the Earth's surface. You can turn off this automatic tilt (Tools > Options > Navigation > Navigation controls; Mac: Google Earth > Preferences > Navigation > Navigation controls)."

You can now display the sun by enabling View > Sun or clicking on the sun button from the toolbar. To create time-lapse views of sunsets and sunrises, click on the "play" button and watch the changes.

For some of the imagery, you can see at the bottom of the window an approximation of the date when it was taken. The Street View images from Google Maps are now available in a new Google Earth layer, which is not enabled by default.

Google Earth includes much more models in the 3D buildings layer for cities like: San Francisco, Boston, Orlando, Munich, Zurich. "Google has optimized the loading and performance of 3D buildings. When you first turn on the 3D Buildings layer near a city with models, you'll see simplistic versions of the buildings load up really fast, then they gradually get more solid and load more texture detail," explains the unofficial Google Earth Blog.


Google Earth 4.3 can be downloaded from earth.google.com. Windows users that don't want to install the application using Google Updater can try this direct link. You'll probably notice that the Windows setup is much smaller: the size has been reduced from 12.7 MB to 7.36 MB. Unfortunately, the new version seems to be less stable and it uses more resources, but it's still in beta.

For Google, Online Video = YouTube

Whenever a Google product adds a feature related to video, YouTube comes into play. Google Talk's gadget lets you watch YouTube videos, orkut lets you add videos from YouTube and Google Video, personalized maps can include videos from the same two Google-owned services, content producers that want to add their videos to Google News need to host them at YouTube and now local business owners can add videos to their Google Maps listings, but only if they are hosted at YouTube.

"In addition to using Google Maps to get local business details, read reviews, and check out photos, I can now also get a sneak peek with embedded videos. Local business owners can easily add YouTube videos along with other content such as business details, photos, and descriptions to their listings. To do so, simply upload your videos to YouTube and ensure that the 'embed' option is turned on," explains Google LatLong Blog.


Online video is more than YouTube and Google Video, but Google seems to ignore this. Even if YouTube's US market share is 73.18% (according to Hitwise), it's unreasonable to think that YouTube should aggregate all the videos that are available online. Google should encourage diversity and choices, instead of selecting the most convenient options.

Related:
Promoting your own services in search results: Google/YouTube vs Yahoo/Flickr

Google Updater, the New Installer for Google Software

Last year, I posted that Google intends to install all its applications through Google Updater, the central component of Google Pack. At that time, a small number of people were redirected to the integrated installer, but this behavior has now become a standard practice.

Because some of the files from Google Earth were corrupted, I had to uninstall it. When I went to Google Earth's download page, Google informed me that I have to install Google Earth with Google Updater.


Google Pack's help center gives some reasons why it's convenient to use the Updater, but most of them help Google promote other software. "The Google Updater makes the software installation process more convenient in several ways. First, it installs software easily with just a few clicks. Also, once the Google Updater is installed, you can choose to have a system tray icon notify you when new software becomes available. Finally, the Google Updater provides you with a central place from which you can download more Google software, as well as other software we think you'll enjoy." (my emphasis)

Probably the only reason why I use my computer is to install Google software and this updater finally helps me get things done. If I want to install Google Earth, it's obvious that I should be informed if Google launches other applications and I should be able to install them with a single click. Hopefully, in the next iterations of the Updater, the click will be eliminated and the new software will be installed automatically after analyzing my interests.


I installed Google Earth using the updater and the setup was launched in the background, with the default settings. Google Updater is installed as a system service that starts automatically, places an icon in the system tray and constantly pings Google to see if there are any updates for the Google software installed on your computer. By default, the application installs the updates automatically and can be uninstalled.


Google still offers the chance to install applications without the updater, but the page that points to the direct links is too difficult to find and has an inappropriate title. I'll repost the links here, for convenience.

Google Earth for Windows:
http://earth.google.com/tour/thanks-win4.html

Google Desktop for Windows:
http://desktop.google.com/index.html?rd=f

Google Toolbar 4 for IE:
http://toolbar.google.com/service/tbdl?hl=en&tbdata=T4

This practice is not Windows-only. Google's Mac software is installed only with the updater. "Google Updater is the installer for Google products on the Mac. You can use Google Updater to see which Google software you have installed and to see other Google applications you might be interested in. Google Updater helps keep your software up-to-date by installing updates when they become available. And you can use Google Updater to uninstall Google Software." Probably the most outrageous part from the Mac FAQ is the answer to the question: how do I uninstall Google Updater? "To uninstall Google Updater, you first have to uninstall other Google software on your computer. You can't uninstall Google Updater while you have Google software on your computer because we need it there to keep your software up-to-date."

Maybe Google should focus less on "we" and more on "you". Most Google software already has an option to auto-update and this could be easily added to the applications that don't have it. If the installers are too confusing, Google could simplify them and remove the unnecessary steps. I don't want to imagine what would happen if each application installed a system service for auto-update and used your network connection to constantly check for new updates.

Update: Apparently, I was lucky to install Google Earth in Firefox. If you use Internet Explorer, Google adds the options to install Google Toolbar and to set Google as the default search engine. Both options are enabled by default, so a standard Google Earth installation bundles Google Updater, Google Toolbar and changes your default search engine in Internet Explorer. This is way too much.

Google Starts to Index the Invisible Web

Google Webmaster Central Blog has recently announced that Google started to index web pages hidden behind web forms. "In the past few months we have been exploring some HTML forms to try to discover new web pages and URLs that we otherwise couldn't find and index for users who search on Google. Specifically, when we encounter a <FORM> element on a high-quality site, we might choose to do a small number of queries using the form. For text boxes, our computers automatically choose words from the site that has the form; for select menus, check boxes, and radio buttons on the form, we choose from among the values of the HTML. Having chosen the values for each input, we generate and then try to crawl URLs that correspond to a possible query a user may have made. If we ascertain that the web page resulting from our query is valid, interesting, and includes content not in our index, we may include it in our index much as we would include any other web page." For now, only a small number of websites will be affected by this change and Google will only fill forms that use GET to submit data and don't require personal information.

Many web pages are difficult to find because they're not indexed by search engines and they're only available if you know where to search and what to use as a query. All these web pages create the Invisible Web, which was estimated to include 550 billion documents in 2001. "Traditional search engines create their indices by spidering or crawling surface Web pages. To be discovered, the page must be static and linked to other pages. Traditional search engines can not see or retrieve content in the deep Web -- those pages do not exist until they are created dynamically as the result of a specific search."

Anand Rajaraman found that the new feature is related to a low-profile Google acquisition from 2005.

Between 1995 and 2005, Web search had become the dominant mechanism for finding information. Search engines, however, had a blind spot: the data behind HTML forms. (...) The key problem in indexing the Invisible Web are:

1. Determining which web forms are worth penetrating.
2. If we decide to crawl behind a form, how do we fill in values in the form to get at the data behind it? In the case of fields with checkboxes, radiobuttons, and drop-down menus, the solution is fairly straightforward. In the case of free-text inputs, the problem is quite challenging - we need to understand the semantics of the input box to guess possible valid inputs.

Transformic's technology addressed both problems (1) and (2). It was always clear to us that Google would be a great home for Transformic, and in 2005 Google acquired Transformic. (...) The Transformic team have been been working hard for the past two years perfecting the technology and integrating it into the Google crawler.

It's not clear what are the high-quality sites used by Google for the new feature, but this list includes some good options. Along with Google Book Search, Google Scholar, Google News Archive, this is yet another way to bring to light valuable information.