"Push e-mail is used to describe e-mail systems that provide an "always-on" capability, in which new e-mail is instantly and actively transferred (pushed) as it arrives by the mail delivery agent (MDA) (commonly called mail server) to the mail user agent (MUA), also called the e-mail client. E-mail clients include smartphones and, less strictly, IMAP personal computer mail applications," explains Wikipedia.
Instead of periodically fetching new messages, mail clients receive notifications as soon as there's a new message. Applications no longer waste bandwidth and you're able to read a message shortly after it was sent.
"Using Google Sync, you can now get your Gmail messages pushed directly to your phone. Having an over-the-air, always-on connection means that your inbox is up to date, no matter where you are or what you're doing. Sync works with your phone's native email application so there's no additional software needed."
If you're already using Google Sync to synchronize your calendars and contacts, it's easy to edit the settings and enable it for mail, as well.
Unfortunately, there are some downsides to this feature, so you should only enable push support for Gmail if you really need it. "Depending on your device, any push connection may use more power than fetching content at intervals. In some cases the difference in power usage may have a noticeable impact on your phone's battery life," mentions Google.
Google explains the difference between using Google Sync, IMAP and Gmail's web application for mobile phones. "Google Sync synchronizes Gmail, Google Calendars, and contacts with your mobile phone. You can use the Google services directly in the built-in Mail, Calendar and Contacts application on your phone. Google Sync offers some advantages over using IMAP on your phone. Most importantly, it allows for content to be pushed to your phone within seconds after changes occur on the server. On the other hand, using Gmail Drafts from your phone is not possible via Google Sync. On iPhone and iPod Touch devices, the Gmail web application offers the most comprehensive features and the best experience for most users. But for users who prefer using the native Mail application, Google Sync offers an alternative way to access their Gmail account."
While Google Sync is also available for Blackberry, Nokia S60 devices and other mobile phones, push Gmail only works for Windows Mobile and iPhone devices that support ActiveSync. At least for now.