To test the face detection algorithms, Google started with Manhattan, where Google replaced the Immersive Media imagery with newer and better panoramic images. "This effort has been a year in the making - working at Street View-scale is a tough challenge that required us to advance state-of-the-art automatic face detection, and we continue working hard to improve it as we roll it out for our existing and future imagery." The results are pretty good, although not all the faces are correctly detected and people can be identified even when their faces are blurred.
As Avi Bar-Zeev suggested, a better idea would be to remove people and cars from the images. "The main reason for removing people and cars is the same reason you'd want the base Google Earth imagery to lack clouds. These things tend to block your view. You can't really look behind them in a simple (essentially 2D) panoramic image. They only represent one snapshot in time vs. a broader/more virtualized essence of the place. They make it confusing to add dynamic versions of the same things on top of what's permanently baked into the imagery."