Google Indexes Videos from All Over the Web

Google Video goes beyond indexing videos hosted by sites like YouTube or Metacafe and starts to add videos from all over the web. Unfortunately, Google doesn't convert the videos from AVI, WMV, MOV or MPEG to FLV so they can be played online, because it's probably illegal to host content without permission*. Instead, Google links directly to the videos, doesn't mention a list of pages that could provide some context and doesn't warn you that the video can't be played online.

For now, there aren't too many non-Flash videos indexed by Google: .edu sites and NASA seem to have priority.

* Caching web pages is a notable exception


And here's an excerpt from a web page that link to the AVI (the fourth search result). Google doesn't link to this page, but you can find it using a normal Google search:


Other video search engines, like Live Search and Yahoo Video, already include videos from the web, but none of them figured out how to get information about videos, rank them and present them properly. And Google Video is no exception, considering that YouTube dominates the search results for almost any query and that the ranking algorithms use information about the number of views and ratings, which is only available for some video hosting sites.

From Sidekick to Google Phone

New York Times has a great article about Andy Rubin, the man behind Google's mobile developments. He's believed to lead a team that develops the Google Phone. "The Google Phone — which, according to several reports, will be made by Google partners and will be available by the middle of 2008 — is likely to provide a stark contrast to the approaches of both Apple and Microsoft to the growing market for smartphones. Google, according to several people with direct knowledge of its efforts, will give away its software to hand-set makers and then use the Google Phone's openness as an invitation for software developers and content distributors to design applications for it," speculates NYT.

Andy Rubin doesn't try for the first time to build a great mobile platform around a device. After leaving WebTV in 1999, he "decided to make a device about the size of a small candy bar that cost less than $10 and allowed users to scan objects and unearth information about them on the Internet." Then he "added a radio receiver and transmitter to the device, which in mocked-up form was about the size of a bar of soap" - the results was Sidekick. "In early 2002, Mr. Rubin gave a talk on the development of the Sidekick to an engineering class at Stanford. Mr. Page and Mr. Brin attended the lecture. It was the first time they had met Mr. Rubin; after the lecture, Mr. Page walked up to examine the Sidekick and found that Google was the default search engine. 'Cool,' he said. At the time of Mr. Rubin's talk, the idea of a hand-held device that included cellphone capability was already in the air, but the recent emergence of digital wireless networks was giving it new life. Mr. Page, in particular, soon became enamored with the idea of a Google Phone and a complete operating system for mobile devices."

Another New York Times article, from 2005, has more details: "The Sidekick was an early favorite of both Mr. Page and Mr. Brin, who wore the units on their belts as all-purpose voice and data communicators several years ago. A Google-branded smart phone has long been a pet project of Mr. Page, and earlier this year Google invested $2 million in a project by Nicholas Negroponte, founder of the M.I.T. Media Laboratory, to develop a $100 wireless laptop. The smart-phone idea, which the company has not talked about publicly, would be a way to extend Google's reach and give it a more extensive connection with its users by offering Google on a multipurpose mobile device."

(Sidekick 3, launched in 2006.
Photo licensed as Creative Commons by Aaron Landry)

Three quotes from the press of that time show that Andy Rubin's device had a lot in common with Google's vision and could foreshadow what we'll see in the Google Phone, which, according to Wall Street Journal, could be announced on Monday. Some key things: platform for developers, always connected to a server and cheap.

Question: With all the challenges in the handheld market, do you think it is a good time to be entering the market with a new product right now?

Answer: I do, I do. I know that a number of PDA companies are having problems. But what we're really talking about here is a converged device--it's a phone with data capabilities. When we're talking about converging devices a lot of people are asking, "Is it a voice device or a data device? Is it a PDA or a phone?" The real question they should be asking is, "Is it a platform for third-party developers?" This is new. This is the first time that a phone has been turned into a platform for third-party developers.

(June 2002 - CNet, Andy Rubin talks about Danger's hiptop device, a.k.a. Sidekick)

"The hiptop does everything wirelessly," [Joe] Britt said. "It's a cell phone and data communicator and does its job by communicating with the Danger service. It also has a Web front end, so you can access your personal calendar through any computer via a Web browser. You can import data through the Web interface and have personal info, like contact data or photos, appear on your hiptop. (...) The hiptop is always on and always on the network. You can do things such as instant messaging anytime."

(December 2001 - MacWorld)

Jim Forbes, industry analyst and producer of DEMOmobile adds, "The industrial design of the Hiptop is certainly innovative; however, the fundamental differentiator allowing Danger to redefine the category is their back-end service that supports the device. This active state technology allows Danger to break the price barrier and deliver a compelling consumer offering. The service does the heavy computing and continually updates the user with new services and capabilities without the need for expensive new hardware."

Danger provides an integrated solution for wireless service operators consisting of a live back-end service, a standards-based platform that uses programs written in Java and Hiptop hardware designs.

(Press release from 2001)

Orkut's Application for Asking Questions

OpenSocial, Google's API for social gadgets, has been launched to the public. The API is supported by many social networks, including MySpace and orkut, but it's still not ready for prime time (Plaxo is the first site that lets you add social gadgets to your profile).

orkut added something new: an application that lets you ask your friends questions. It wasn't developed using OpenSocial, but it's a perfect example of an useful social gadget. You can find it at the top of the feeds page.

Ask Friends (or Mobius, probably a reference to the Möbius strip) has two clever features that make it non-trivial. Based on your question, orkut automatically generates a list of labels, probably from an existing database. Using these labels and other labels entered by you, orkut determines what friends are most likely to have an answer for your question. "The system will choose Orkut friends who may have an interesting answer for you based on their answering history and interests." There's also an option to select the friends that can view and answer your question.

It's a pretty interesting way to target your question and it could be extended to a broader audience, like all orkut users from your country or all orkut users who like Radiohead, not just your friends.

Social networks are containers for web applications. They have details about you and your friends, but this data only becomes meaningful in an application. Your personal data is the input for social applications that aggregate it and derive useful outputs and make your life easier.

"Self expression and communication are often fun and entertaining alone, but OpenSocial is also a platform that can be leveraged to solve real world tasks where the social graph assists us in making decisions. For example, while some might be prone to grab a book at random off the shelf, there are many who appreciate a good recommendation from a friend. With a variety of possibilities in entertainment and interests, it can be useful to facilitate meetings, purchases, recommendations, information management and learning to create a richer, more lasting experience across your application," says Google in an article titled "Social Design Best Practices".


{ via labnol.org }

Google Notebook Adds Labels and Bookmarks

Google Notebook added the most requested feature: labels. Now you can label each note and see all the notes that have a certain label.

The application imports all the labels and web pages from Google Bookmarks and places them in a new notebook titled "Unfiled". Adding a new note in this special notebook actually creates a new bookmark.


The organization of a notebook is more flexible: the notes can be sorted by date, by label and filtered by label.


Google Notebook's extension was also updated and has a new option: select the text from a web page and click on the small star to add it as a note. Now you can use the extension to add bookmarks by clicking on the star when the "Unfiled" notebook is selected.


All in all, the integration between Google Bookmarks and Google Notebook is interesting, but it's not very transparent in the interface. Every bookmark becomes a note in a special notebook, but "Unfiled" should be replaced with "Bookmarks".

Google Notebook will also add offline support using Google Gears in the near future, so you can access your notes and bookmarks even when you don't have an Internet connection.

{ Thanks, David Hartunian. }

Blog Search, Included in Google History

Google Blog Search is now a part of your search history. That means every time you use the specialized search engine for blogs and you're logged in, your query is associated with your account and made available in the Google History interface. You can browse the list of queries and find a web page you visited in the past, but only if it was a Google search result. To augment Google History with all the web pages you visit, you need to install Google Toolbar and enable this option.

Google-Difficult Words

For a small number of queries, Google shows a list of three results for other related query in the middle of the search results page. The "inline revisions" are only displayed if Google is confident that the new query provides better results.

We'll call a word or expression Google-difficult if the search results page shows an inline revision for "dictionary". That means many people who searched for that word changed the query to "dictionary" to find its meaning in a dictionary. Some examples: conceptual, tentative, salient, lucrative. Because Google's results change all the time, a word that was Google-difficult at some point will not lose its quality even if it doesn't return an inline revision for "dictionary".

Can you find other Google-difficult words? Use an English interface for Google.

A Tentative Google PC

Inexpensive computers go hand in hand with the vision of moving applications online. After all, if you spend most of your time inside a browser, you don't need a very powerful computer.

Everex gPC TC2502 has a 1.5 GHz processor, 512 MB of memory and an 80 GB hard drive. It runs a modified version of Ubuntu, called gOS, and comes with free applications like Firefox and easy access to many Google services: Search, Calendar, Docs, Blogger, YouTube. "Imagine a gorgeous desktop that just works. All the web, media, and office software you'll need is included. Imagine an OS with easy access to the best of Google Apps and products, and other popular Web 2.0 applications." It will be available in the US at Wal-Mart for only $199.

According to DesktopLinux, Everex claims the computer was "created as a conceptual Google PC with a conceptual Google OS" and it has Google's permission to use its trademarks. The g from gOS and gPC means "green", as the computer uses an energy-efficient processor, but many people will associate it with Google.

The idea of a Google PC is not new and Google dismissed it in many occasions, but that doesn't stop Google from having distribution deals for its software with computer manufacturers like Dell. As web applications and browsers become more powerful and Internet becomes ubiquitous, the transition to the online software will seem natural.


{ Thank you, Milivella. }

Google Photo Picker

The standard Google dialog for choosing a photo integrates with Picasa Web Albums. If you want to add a photo to your orkut scrapbook, you can select it from your computer, from the web or from Picasa Web Albums. When you upload photos from your computer, they're saved in an album called "Drop Box". For each Blogger blog, Google creates an album that stores the images uploaded to that blog.

That means we could expect to see other Google apps using the same model. Google Docs could have an album that contains all the images uploaded to your documents, spreadsheets and presentations. The Gmail album would include all your image attachments. This way, you wouldn't have to upload the same image multiple times and you could reuse in other Google sites.

It's unclear why Google doesn't provide in the dialog an option to search your images from Picasa Web Albums. You still have to know the location and the name of an image. Google could extend the photo picker to documents and other files, move Google Desktop online and provide a unified way of uploading, organizing and sharing your files.

OpenSocial, Google's APIs for Social Applications

Interactive Friends Graph Map, a Facebook app. Image licensed as Creative Commons by inju.

You probably remember the post about Socialstream, a Google-sponsored project that tried to "rethink and reinvent online social networking". The result was a meta-social network that aggregates data from other community sites using APIs and whose goal "is to present social information in a way that ties it to the person who posted the information, and not the site from which it came."

Since last year, Google was busy developing plans for a set of APIs that would make it possible to communicate with other social networks. Brad Fitzpatrick, who moved to Google from Six Apart, wrote an interesting article "Thoughts on the Social Graph" that tackled this problem. "Unfortunately, there doesn't exist a single social graph (or even multiple which interoperate) that's comprehensive and decentralized. Rather, there exists hundreds of disperse social graphs, most of dubious quality and many of them walled gardens." His solution was to make this database of social connections a "a community asset, utilizing the data from all the different sites, but not depending on any company or organization as the central graph owner."

Even if this will not bring Brad's vision closer to reality, Google will launch tomorrow OpenSocial, "a set of common APIs for building social applications across the web". According to a still-unofficial press release, "OpenSocial gives developers of social applications a single set of APIs to learn for their application to run on any OpenSocial-enabled website. By providing these simple, standards-based technologies, OpenSocial will speed innovation and bring more social features to more places across the web." The APIs give access to a user's profile, their friends, and the activity streams.

The success of Facebook's platform, that has more than 5000 applications, made a lot of social networks consider the launch of similar platforms. But not many developers would develop different applications for each social network, so smaller sites will have less visibility. orkut, Google's social network, has more than 70 million users, but only 18% are in the US. While it would've been easier for Google to just open up orkut, this common set of APIs will make the social applications more valuable because they can run in many other places and can access data from other sites.

Google's social APIs should be available at code.google.com/apis/opensocial (the site is not live yet). The initial social networks and companies that support the APIs are orkut, hi5, Friendster, LinkedIn, Viadeo, Ning, Salesforce, and Oracle. It will be interesting to see if other social networks decide to join Google's efforts. MySpace announced that will open its platform in the next months, but it's unlikely to use Google's APIs.


"The timing of OpenSocial couldn't be better. Developers have been complaining non stop about the costs of learning yet another markup launguage for every new social network platform, and taking developer time in creating and maintaining the code. Someone had to build a system to streamline this (...). And Facebook-fear has clearly driven good partners to side with Google," writes TechCrunch.

"Open Social's API is based entirely on Javascript. If you know HTML and Javascript today, you will be able to immediately use Open Social to turn your web applications and web sites into Open Social apps. You can also use standard web development tools to build Open Social apps. This is obviously a much better way to operate than having to learn a proprietary [mark-up] language or query language," writes Marc Andreessen.

And Google also has a financial incentive to build this open platform. "A person familiar with Google's efforts said that those applications have been far more effective for advertisers on social networks than users' personal pages," reports the New York Times. Google tried to convince "third-party developers with applications on Facebook to run Adsense ads within applications pages".

Update: MySpace joins OpenSocial. "MySpace says they are abandoning their efforts to create their own markup language (which is what Facebook has done) and direct APIs will go exclusively with OpenSocial." Other social sites that join Google's efforts: Bebo and Six Apart. It seems that everybody except Facebook will be in this coalition.

Update 2: "As the most trafficked website in the country and the most popular social network in the world, MySpace is one of the leading forces in the global social Web. We're thrilled to grow our strategic relationship with MySpace by joining forces on this important initiative," said Eric Schmidt. Google's press release is mostly about MySpace and its "commitment to supporting standards that foster innovation in an increasingly social Web". That's a welcome change from MySpace.

Google to Connect to Other IM Networks Using Jabber Transports?

With more storage, IMAP support and a new version optimized for performance, Gmail continues to surprise everybody. And if you think you've seen everything, you're wrong. After looking at the code of the recently launched Gmail 2.0, it seems that Gmail team actually listens to feedback, because they'll implement some of the popular suggestions:

* colors for labels

* detaching messages from a conversation

* Jabber transports (these could be used to chat with people from other IM networks, like Yahoo, MSN, AIM). You could already use these transports to connect Google Talk with other IM networks, but you have to use a third-party server and another IM client to configure the transports. Being able to chat with people from other networks, which are much more popular than Google Talk, will make Google's instant messenger more useful.

An extract from Gmail's JavaScript code:



It's also interesting that the new contact manager from Gmail lets you enter usernames for many IM networks:


What else would you like to see in Gmail 2.0?

Gmail's New Version Is Now Available

The new version of Gmail I was talking about the other day is already available in some Gmail accounts. If you see a link to an "older version" at the top of the page, that means you can enjoy the new features: mail prefetching, updated contact manager and other small updates.


Gmail has a new architecture that improves the performance and the usability. Now you can use the back button in your browser and bookmark URLs from different Gmail views because the URLs change when you go to a different section. The messages are prefetched when Gmail loads so you don't have to wait too much until a message is displayed.

The new contact manager has inline search, better options to delete contacts or add them to a group, the address is now structured and you can enter usernames for different IM networks. You can also export the contacts in vCard format, export the contacts from a group and print your address book so you can use it offline. If you want to see the new contact manager and you don't have the new version of Gmail, don't worry. It's a separate application that can be found at this URL.


Gmail also added an option to create filters based on the current message and a mystery-meat "mute" that brings to the interface an already existing feature. When you hover over a contact name you'll no longer see only the picture, now Gmail shows a beautiful contact card that lets you view the recent conversations with one click.




There are more options for displaying Gmail chat and you can now select a photo for your profile from Picasa Web Albums. Gmail will probably use the same profiles available in Google Maps and Google Shared Stuff.




Here are some comments from those who have the new version.

Chris: "Just experienced the new interface. The load time for the inbox takes a bit longer, but my goodness are the instantaneous load times of the pre-fetched emails ever convenient."

Anonymous: "The new interface seems quite a bit slower to me. Prefetched messages do load more quickly, but switching to labels/folders that have many messages now takes quite a bit longer. (E.g. switching to All Mail or Sent Mail from the inbox seems to be quite a bit slower now.) The new contacts manager is surprisingly unstable for a Google web product as well."

Gmail's interface is almost identical, but the team promises to add more features in the future. "The Gmail team has been working on a structural code change that we'll be rolling out to Firefox 2 and IE 7 users over the coming weeks (with other browsers to follow). You won't notice too many differences to start with, but we're using a new model that enables us to iterate faster and share components."

Bottom line - Gmail 2.0
- limited testing: Firefox, IE 7 (only some users)
- new JavaScript architecture
- look for: permalinks, new contact manager, photo picker, mail prefetching
- there aren't many new features, but I think these are the first steps for the social Gmail

{ Thanks, Jason Persampieri. }

How Gmail Blocks Spam

While Gmail doesn't filters all the spam messages that could reach your inbox, it certainly does a better job than other webmail apps like Yahoo Mail or Hotmail. Gmail's filters are constantly improving and an important ingredient of their effectiveness is the use of community signals. Every time you click on the "Mark as spam" button, Gmail uses that information to block similar future messages not only for you, but for all Gmail users. But spam is also evolving and it's harder to block, especially when it uses images and literary texts.

"Many Google teams provide pieces of the spam-protection puzzle, from distributed computing to language detection. For example, we use optical character recognition (OCR) developed by the Google Book Search team to protect Gmail users from image spam. And machine-learning algorithms developed to merge and rank large sets of Google search results allow us to combine hundreds of factors to classify spam," explains Google. "Gmail supports multiple authentication systems, including SPF (Sender Policy Framework), DomainKeys, and DKIM (DomainKeys Identified Mail), so we can be more certain that your mail is from who it says it's from. Also, unlike many other providers that automatically let through all mail from certain senders, making it possible for their messages to bypass spam filters, Gmail puts all senders through the same rigorous checks."


The spam in Gmail is also the subject of a promotional video that encourages to use Gmail if you want to "get back your time". The video reminded me of an old email account from my ISP: it received so much spam that it was almost impossible to find the genuine mail and the 10 MB of storage were insufficient to collect all this junk.


Maybe Google should make greater efforts to filter spam in other services, like Google Groups, Blogger or YouTube. The experience from Gmail could be useful.

The Growing World of Google Gadgets

Most people call them widgets, but Google wanted to be different: the first name was modules, but it was changed to gadgets. They're wrappers for small pieces of content from the web that can be added to a web page or to the desktop. "Google Gadgets are mini-applications that work with iGoogle, Google Desktop, or any page on the web. They can range from simple HTML to complex applications, and can be a calendar, a weather globe, a media player, or anything else you can dream up."

"[Gadgets] are so easy to create that they are a good starting point if you are just learning about web programming. The Google Homepage API consists of a few simple building blocks: XML, HTML, and JavaScript." This is a description from December 2005, when Google opened the personalized homepage to developers.

Since then, Google's gadgets evolved a lot: they have more functionality, can be added to web pages and Google Desktop, became a new form of advertising and are able to recreate a web page from scratches. But gadget also became synonymous to extension, add-on, plug-in at Google. The concept was extended to other applications: Google Maps (mapplets), Google Calendar (web content), Google Toolbar (buttons), Google Search (subscribed links) and more.

Google uses similar directories for all these different kinds of gadgets:

* iGoogle gadgets (more than 20,000)

* Google gadgets for web pages (a subset of the iGoogle gadgets)

* Google Desktop gadgets

* Google Toolbar buttons

* Google Maps mapplets

... but also for other types of content:

* Google Desktop plug-ins (extend the indexing capabilities)

* Google Earth KML files (overlays)

While most iGoogle gadgets can be added to web pages, Google Desktop gadgets can be added to iGoogle only using a browser plug-in and mapplets work only inside Google Maps. There's still no single standard for gadgets and you need to write different code, depending on the medium where the gadgets run.

It would be nice to write a single code that displays status data in Google Toolbar or in a mobile app, a simple interface in iGoogle, a richer interface in Google Desktop's sidebar and could also integrate with Google Maps or Google Calendar. A mail gadget could show the number of unread messages in Google Toolbar, their subjects in iGoogle, their content in Google Desktop, the locations mentioned in the messages at Google Maps and the events at Google Calendar. You would only need to add the gadget once and see it in different incarnations, depending on the context.

The Next Version of Gmail Will Be Faster

At the Analyst Day event from October 24, Google didn't announce a lot of things, but it revealed what we can expect from a new version of Gmail that will launch soon:

* prefetching messages. When you load a page that contains a list of messages, Gmail will fetch them in advance so that when you need them, they'll be available instantly. Google said that Gmail will have a completely new JavaScript architecture that will bring a much better performance.

* a new contact manager that will be shared with other Google apps (Google Docs, Google Calendar etc.)


* integrating features from other Google products (an example of an existing feature was the "Open as a Google Document" links placed next to Microsoft Office attachments).

Google didn't mention anything about an interface change, but a recent discovery showed that Gmail also prepares a new UI. Maybe the features I anticipated last month will still make their way in this new version of Gmail.

Related: Google Analyst Day 2007 @ YouTube

Update (a day later): Gmail 2.0 is here.