Viacom Wanted the Source Code for Google's Search Engine, But Obtained YouTube's Server Logs

In the ongoing trial between Viacom and Google, regarding the videos uploaded to YouTube that infringe Viacom's copyright, Viacom really wants to prove that the most popular videos watched at YouTube were from its programs. Viacom even claimed that Google's search results are biased to give better ranking to the infringing YouTube videos, so it asked for... Google's source code (and YouTube's source code too). Here are some excerpts from the rulings:

Plaintiffs move jointly pursuant to Fed. R. Civ. P. 37 to compel YouTube and Google to produce certain electronically stored information and documents, including a critical trade secret: the computer source code which controls both the YouTube.com search function and Google's internet search tool "Google.com". YouTube and Google cross-move pursuant to Fed. R. Civ. P. 26(c) for a protective order barring disclosure of that search code, which they contend is responsible for Google's growth "from its founding in 1998 to a multi-national presence with more than 16,000 employees and a market valuation of roughly $150 billion" and cannot be disclosed without risking the loss of the business.

The search code is the product of over a thousand person-years of work. There is no dispute that its secrecy is of enormous commercial value. Someone with access to it could readily perceive its basic design principles, and cause catastrophic competitive harm to Google by sharing them with others who might create their own programs without making the same investment. Plaintiffs seek production of the search code to support their claim that "Defendants have purposefully designed or modified the tool to facilitate the location of infringing content." (...) YouTube and Google maintain that "no source code in existence today can distinguish between infringing and non- infringing video clips -- certainly not without the active participation of rights holders".

Unfortunately for Viacom and Google's competitors, the request to provide the source code has been rejected. But another request, this time for YouTube's server logs, has been approved.

Defendants' "Logging" database contains, for each instance a video is watched, the unique "login ID" of the user who watched it, the time when the user started to watch the video, the internet protocol address other devices connected to the internet use to identify the user’s computer ("IP address"), and the identifier for the video. That database (which is stored on live computer hard drives) is the only existing record of how often each video has been viewed during various time periods. Its data can "recreate the number of views for any particular day of a video." Plaintiffs seek all data from the Logging database concerning each time a YouTube video has been viewed on the YouTube website or through embedding on a third-party website. They need the data to compare the attractiveness of allegedly infringing videos with that of non-infringing videos.

Google argued that the task requires a lot of resources, since the logging database has 12 TB, and it violates users' privacy. Google has previously stated in a blog post that an IP address without additional information cannot identify people, so it's not personal information. "Therefore, the motion to compel production of all data from the Logging database concerning each time a YouTube video has been viewed on the YouTube website or through embedding on a third-party website is granted."

Viacom wanted other things: the schema for Google's advertising database and for Google Video's database, data about private YouTube videos etc. You can read the entire document as it's pretty entertaining.

Salon thinks that "all's not lost. Google might manage to reverse this decision on appeal, and Viacom, gauging the outrage, could decide to withdraw or limit its request." After all, getting YouTube's server logs just to determine the popularity of the infringing videos is an abuse: YouTube could have offered aggregated data about those videos.

Update: According to Search Engine Land, Google sent a letter to Viacom regarding the removal of personal data.

Given Plaintiffs' stated reasons for seeking information from the logging database -- to conduct proportionality analyses -- potentially personally identifiable information should be irrelevant. Indeed, Plaintiffs have previously represented that they do not desire to investigate users' viewing activities, and Viacom's general counsel is on record today stating that Viacom does not want to receive individuals' usernames and IP addresses. Accordingly, we request that Plantiffs agree that YouTube may redact usernames and IP addresses from the viewing data in the interests of protecting user privacy.

Update 2 (July 15): "We are pleased to report that Viacom, MTV and other litigants have backed off their original demand for all users' viewing histories and we will not be providing that information," says YouTube Blog.

Google Results in Firefox's Address Bar

Ryan Wagner from CyberNet News developed CyberSearch, a Firefox extension that lets you see Google results in the address bar. The extension adds new functionality to Firefox 3's address bar, which already shows bookmarks and other pages from your web history.

By default, CyberSearch is a little inefficient: it performs web searches as you type in the address bar and it shows Google's search results after the local results obtained by Firefox. If you type one or two letters, it's likely that the address bar will only show suggestions from your local history. For more precise queries, you should mostly see Google results. In the example below, I only had to type [new sci] to directly access New Scientist.


If you usually type your queries in Firefox's address bar, you know that sometimes Firefox sends you to the top result for your keywords (e.g.: enter [new scientist]), while in most cases you're sent to the list of search results (e.g.: the ambiguous [scientist]). CyberSearch is redundant for many navigational searches since the top results is the only one that matters and Google sends you to the top result anyway.

But there's a way to trigger the list of search results only when you need it: start your query with the special keyword "goog". Enable keywords in the options (Tools > CyberSearch Options), where you can add your own keywords that trigger results from specialized search engines like Google News or restrict the results to a single domain. For example, I added the keyword "gos" that shows Google results from this blog:


To only show search results when you start your query with a special keyword, go to Tools > CyberSearch Options, and check:

[x] Enable keywords
[x] Show only search results when a keyword is recognized
[x] Don't perform a search unless a keyword has been typed

It's difficult to determine if a result is useful without reading some snippets from the web page, so CyberSearch is great when you want to re-find specific pages from familiar sites.

This extension is based on Searchery and the same idea is used in Inquisitor, a Mac application recently acquired by Yahoo.

Other ways to search more effectively from your browser:
* Outsource Firefox's keywords to YubNub
* Tips for Google Toolbar
* Link to a page using Google AJAX Search

Second-Class Google Citizens

Every time Google releases a new feature for Gmail, Calendar, Google Docs etc. people who use Google Apps are left wondering whether they'll get the new feature. Sometimes they have to guess addresses, like for the new mobile Google Talk. They are supposed to figure out that the Google Apps version of: http://talkgadget.google.com/talkgadget/m is http://hostedtalkgadget.google.com/a/YOURDOMAIN.COM/talkgadget/m.

Besides having to deal with delayed updates and mysterious addresses, Google Apps users usually have at least one standard Google account and it's difficult to switch between the two parallel worlds. I noticed that when you go to Google Sites and you're logged using both a Google account and a Google Apps account, you are asked to choose one of them:


Maybe Google could somehow integrate Google accounts with Google Apps accounts so you can access all the services and get all the new features. The services that are part of Google Apps should have a customized interface and functionality, while the other services should only interact with them so you can, for example, share Google Reader items with your Google Talk contacts.

Google Talk for iPhone

Google has finally launched a mobile version of Google Talk, but it's optimized for iPhone's browser. You can access if you go to google.com/talk on an iPhone/iPod Touch or using this permalink: http://talkgadget.google.com/talkgadget/m (shorter URL: http://tinyurl.com/4vnfcd).


This Ajax version is based on Google Talk's Flash gadget, but it doesn't include tabs, group chat, options to add contacts and send email. What you can do is to update your status, search your contacts and chat with people that are online.

Besides this mobile interface, Google also has a mobile app for Blackberry. That means if you don't have a Blackberry, an iPhone or at least a WebKit mobile browser, you have to find a third-party service.

eBuddy is a service that offers both a basic mobile interface for Google Talk and a mobile application that lets you chat faster with your friends. eBuddy doesn't connect only to Google Talk, it's a multi-network IM client that works with MSN, Yahoo and AIM. Unfortunately, I couldn't find any information regarding the way eBuddy stores or uses your credentials and seeing that eBuddy sends the password in plain text is worrisome.

Another mobile application that lets you chat with your Google Talk contacts is Fring, but it only works on Symbian and Windows Mobile devices. As usually, make sure you trust the service before entering the username and password of a Google account.

Google Street View in France

As a special present for the upcoming Tour de France, Google added Street View imagery for Tour de France's routes. The coverage is really limited, but Google will extend it in the next months, when Street View will be available in many European countries (UK, France, Germany, Spain, Italy and others).


Wikipedia has some details about this year's edition of the largest cycling race in the world. "The event will take place from July 5 to July 27, 2008. Starting in the French city of Brest, France, the tour will enter Italy on the 15th stage and return to France during the 16th, heading for Paris, its regular final destination, which will be reached in the 21st stage."


{ via Zorgloob, a French blog about Google }

Mobile Google Maps with Voice Search

An obvious use of the GOOG-411 service is to add speech recognition to Google's mobile mapping application. Microsoft already has voice support in Live Search for Windows Mobile and that's an important differentiating factor when you are on the go and you need a quick answer.

Google started to test Google Maps with voice search for three Blackberry Pearl models: 8110, 8120, 8130 and only for the US. Instead of typing when you want to search for a local business, hold the left-side key, say what you want to find and then release the key. Google Maps Mobile will display the search results on the map.

"Similar to other voice-recognition technologies, the accuracy of voice recognition will improve over time as more people use the voice search feature," says Google Mobile Blog. In fact, the main reason why GOOG-411 has been developed was to obtain data to train Google's speech recognition algorithms. "Whether or not free-411 is a profitable business unto itself is yet to be seen. I myself am somewhat skeptical. The reason we really did it is because we need to build a great speech-to-text model ... that we can use for all kinds of different things, including video search," explained Marissa Mayer from Google.

Speech recognition could greatly facilitate the way we interact with mobile phones and it's not hard to imagine that voice search will be the main way we'll use search engines in the future. This will change the way people formulate queries and processing natural language will become increasingly important. Microsoft's acquisition of Powerset, a so-called semantic search engine, could improve the way Live Search handles complicated queries, even though it's difficult to scale the computationally-intensive process of parsing text to the web and to all languages.

Taking Features Away From Users

Ryan Singer wrote an interesting post titled "Features are a one-way street". The main idea was that "once your user base has grown beyond a certain point, you cannot take features away from them. They will freak out. Whether the feature is good or bad, once you launch it you've married it."

Google is a company that launches many features and services "in beta", tests them for a while and then decides whether it's a good idea to continue developing them. Some of the Google Labs projects were successful, while others were discontinued or they were integrated in other experiments.

In a talk about innovation, Marissa Mayer mentioned an idea that came from Eric Schmidt: "don't kill projects, morph them". I don't know how many people miss Google Voice Search, a service that allowed you to search Google with a phone call, or Click-to-Call, a simple way to call businesses for free directly from Google Maps, but both services are the predecessors of GOOG-411. The cool Google Deskbar was partly integrated in Google Desktop, the SOAP Search API was replaced by a REST API, Google AdSense Referrals was superseded by the DoubleClick Performics Affiliate Network, while a replacement for Google Answers is being tested in Russia and China.

Other products were discontinued without offering a replacement: Google Video Store, the photo sharing service Hello, Google Browser Sync or the "Send to SMS" feature from Google Toolbar and from Google Send to Phone.


For Google, features aren't a "one-way street" and you never know when a functionality you start to rely on is removed for unknown reasons. That's why I think Google should label experimental features more prominently and it should do a better job at communicating the reasons why a service is dropped.

Google Docs Stats

Update (March 2012): This feature is no longer available. "We're phasing out Google Analytics support in our new editors, but we'll be working on alternatives to provide admins with more information about their doc's viewers."

If you publish documents at Google Docs and link them from web pages, you may wonder how many people actually view them. Google helps you find this information by placing a Google Analytics tracking code that can be connected to your account.

How to get traffic stats for your documents?

Just go to Google Analytics, create a new profile for http://docs.google.com and get the tracking code for that profile. You only need a small piece of the tracking code: the value displayed in bold below, which should be different for your account.

var pageTracker = _gat._getTracker("UA-84813-6")

Once you have that value, go to Google Docs Settings, enable "Track visits to my documents using Google Analytics" and paste the value in the input box for "Google Analytics tracking code".


What happens after enabling the integration with Google Analytics?

After a couple of hours, you should see some traffic data in your Analytics account:


You'll also find a confirmation that the visitor traffic is tracked with Google Analytics after you publish a document:


The list of Google services that integrate with Google Analytics includes: Google Sites, Project Hosting, iGoogle gadgets and hopefully the support will be extended to other services.

{ via Blogoscoped Forum }

The Best Firefox Extension for Google Bookmarks

If you use Google Toolbar for Firefox only because you like the bookmarking feature, there's a much better option: GMarks. While there are other extensions that provide an interface for Google Bookmarks, John Marshall's GMarks adds a lot of value to Google's basic bookmarking service.

GMarks can display the bookmarks almost the same as Firefox's native bookmarks: in a separate menu and in the sidebar. There's also a dropdown that can be added to Firefox's toolbars. The sidebar is useful when you want to perform full-text searches in your bookmarks without having to visit Google's site, while the GMark menu has an excellent option for managing bookmarks. If you click on "Organize bookmarks", you'll be able to import bookmarks, edit or delete bookmarks and create Gmail-like filters. For example, you can automatically add the label Google for any bookmark from this blog.


The extension can show the number of bookmarks for each label, a list of recent bookmarks and the most frequently visited bookmarks. To bookmark a page, use the familiar shortcut Ctrl+D or the GMarks menu.

One of the most frequent user requests for Google Bookmarks is nested labels and GMarks uses a little trick to simulate hierarchical labels: separate the labels with >. For example, if you want to label a bookmark as Shopping and Comparison, where Comparison is a sublabel of Shopping, just enter Shopping>Comparison when you create the bookmark. Since Google Bookmarks doesn't support this feature, the labels will appear to be nested only in GMarks.

And the cool features don't stop here: the extension suggests labels based on the web page's title, you can hide the bookmarks that have certain labels and there's an option for integrating the starred items from Google Reader.

Since the last time when I wrote about GMarks, the extension added the most interesting ideas implemented by the "competition" and managed to become THE Google Bookmarks extension. If there's anything that could be added, maybe it would be useful to completely replace Firefox's bookmarking system by hiding all the interface elements related to the native bookmarks and replicating all the changes to Google Bookmarks locally.

{ Thanks, Dr. Azrael Tod. }

Mozilla's Weave Prototype Updated

After Google announced that Browser Sync will no longer be updated and will stop working at the end of the year, many people looked for an alternative. The closest project to Browser Sync is Mozilla Weave, a Firefox extension that synchronizes local data from your browser with a server so you can access your data from multiple computers. Unfortunately, Weave is still an experimental prototype, so it can be buggy and unreliable.

Weave 0.2 has been released today and it supports new data types: cookies, passwords, saved form data and sessions, in addition to bookmarks and browsing history. After installing the extension and restarting Firefox, you need to create an account and enter a key used for encryption.


Unlike Browser Sync, Weave doesn't synchronizes data in real-time. "After the initial data transfer is completed during setup, Weave connects to the server periodically when it accumulates enough changes that need syncing or when enough time has passed since the last sync (at least once every 20 minutes if any data has changed). Weave also syncs when you start Firefox, when you quit it, and when you click on the "Sync Now" button in the Weave menu or the preferences."

It's worth mentioning that browser data sync is just one of the components that will be included in Weave. "As the Web continues to evolve and more of our lives move online, we believe that Web browsers like Firefox can and should do more to broker rich experiences while increasing user control over their data and personal information. Weave is the Mozilla Labs' project to develop a coherent framework and platform for deeply integrating online services with the browser."

Opera users can already synchronize bookmarks and notes using a feature included in the latest version of the browser. If you only need to synchronize your bookmarks, migrate to an online bookmarking service like del.icio.us, Google Bookmarks, or install plug-ins like Foxmarks (Firefox), Zinmko (Firefox / IE), that synchronize bookmarks and provide a web interface where you can access and share your bookmarks.

Adobe Helps Search Engines Find Flash Files

Adobe announced that it provided some its Flash technology to Google and Yahoo in order to improve the indexing of SWF files and especially Flex applications. "Although search engines already index static text and links within SWF files, RIAs and dynamic web content have been generally difficult to fully expose to search engines because of their changing states."

To solve this problem, Adobe decided to share Flash Player technology that allows search engines to walk through a SWF file and simulate user interactions. "We've developed an algorithm that explores Flash files in the same way that a person would, by clicking buttons, entering input, and so on. Our algorithm remembers all of the text that it encounters along the way, and that content is then available to be indexed," explains Google's Webmaster Blog. The indexing algorithm, which is already live, has some important limitations: it ignores Flash files loaded using JavaScript and it separately handles external resources loaded by Flash files.

I couldn't find too many Flash sites that are fully indexed by Google, probably because we need to wait until Google updates its index.

Google Photos?

Launched in 2006 as an online extension of the Picasa photo management application, Picasa Web Albums didn't become very popular, as Flickr continued to grow. Besides lacking a community of photography enthusiasts and leaving out a lot of basic features like stats or adding a photo to favorites, Picasa Web Albums is not a great name and many might not even know that Google has a photo sharing service.

There are many references in Picasa Web's code which show that the service will change its name to Google Photos. The navigation bar already uses "Photos" as an anchor text, there's a Google Photos Screensaver and the service's blog is titled Google Photos Blog.


Other upcoming features referenced in the code include content licensing options, uploading photos by email and tagging people in your photos.

YouTube's Search Module

Probably as an alternative to the list of related videos displayed in a sidebar, YouTube experiments with a search box placed below the video's description. The search box doesn't load a new page to show the results, as you can see in the screenshot.


You can search for things mentioned in the video or create a small playlist of related videos by adding some of the results to YouTube's Quicklist. Maybe in the future YouTube will open the sidebar modules to developers so you can add useful information related to the current video: music lyrics, popular playlists that include a video or videos recorded on the same location.

Update: Another experiment shows a link for a related video and an option to automatically play all the related videos.

Export Data from Google's Services

Google mentioned in many occasions that it doesn't want to trap users' data and some of its services have started to provide import/export options. Here are some of the ways to export your data from Google's services:

* iGoogle lets you download the list of gadgets, feeds, themes and their corresponding options if you go to the settings page and scroll down to the "Export / Import" section. The resulting XML file can be imported to another Google account and those who are familiar with XSLT could covert iGoogle's data file to OPML, so you can subscribe to the feeds in Google Reader or any other feed reader.

* Google Calendar has options to export your calendars one by one as iCal files, but it's much easier to export all the calendar you created by going to: http://www.google.com/calendar/exporticalzip.

* Google Docs lets you export your documents as an archive of HTML files and images. To export your files, restrict the Docs list to documents, select all the files and choose "Save as HTML (zipped)" from "More actions". Zoho can already import your documents and there's a Greasemonkey script that helps you download all the files from Google Docs, not just the documents.

* Blogger added in the experimental version available at draft.blogger.com an option to export the posts and comments from your blogs as an Atom feed. Hopefully, developers will write scripts that convert Blogger's feeds to the formats accepted by other blogging services.

* Google Bookmarks can be exported as a HTML file, but for some reason browsers can no longer import the bookmarks. The web history or search history can be exported as a RSS feed: http://www.google.com/history/lookup?q=&output=rss&num=1000 (replace 1000 with the number of items you want to export).

* Gmail lets you export your contacts as CSV or vCard, but Google automatically adds all the people you send a reply in the list of contacts.

It would be nice to export all your data from a single page, so you can delete a Google account or switch to a competing service without losing your data.