Let's take a look at a simple example of a site that doesn't have a custom 404 error page (they're very hard too find these days, so most sites won't fall in this category). If you try to go to news.speeple.com/sunflowers, here's what you see in IE7: a page with useful suggestions like "Retype the address" or "Go back to the previous page".
This is actually a page created by Internet Explorer and you can disable it in the advanced settings, by unchecking "Show friendly HTTP error pages". Here's the page returned by the server, which is displayed in most browsers (Firefox, Opera, etc.):
The latest version of Google Toolbar has a feature disabled by default that replaces IE's error pages with more useful suggestions: the site's homepage or subdomain, some search queries that could help you locate the right page. The idea is that you probably clicked on a bad link or the page was relocated without using a redirect. In this case, Google's query segmentation is not perfect, but it usually does a pretty good job at transforming a URL into an useful query. To obtain the suggestions, the toolbar sends the URL to Google's servers, so this feature has privacy implications. More exactly, the suggestion page is obtained from:
http://linkhelp.clients.google.com/tbproxy/lh/fixurl?sourceid=navclient &hl=en&sd=com&error=http404&url=http://news.speeple.com/sunflowers
Google Toolbar only displays that page for default error pages (that have less than 512 bytes), DNS errors and connection failures. The feature can be enabled from Google Toolbar's settings by checking "Browse by name in the address bar", a feature that also performs searches when you enter keywords in the address bar.
So which of the three pages is more helpful for someone who ends up on a non-existing page from a site that didn't bother to create a custom 404 error page?
Related:
Matt Cutts' reaction
Google tries to fix broken URLs
Browsing the web using Google Cache