The feature is disabled by default. "We realize that not everyone wants their subscription list to turn into a multi-colored extravaganza, so we've made it into a setting that you can access from your subscriptions menu," explains Mihai Parparita.
Google uses a special service for obtaining the favicons instead of loading the images directly. For example, Search Engine Land's favicon is generated like this:
http://s2.googleusercontent.com/s2/favicons?domain=searchengineland.com