But there are also Flash players that let you listen to any MP3 file available online: one of these players was created by Google to use it for Gmail's MP3 attachments. A Greasemonkey script (you need Firefox + Greasemonkey to install it) can add the player next to each link to an MP3 file. Or at least to a file that has the .mp3 extension.

You could also add this code to your site to include the player (replace [MP3_file] with the proper URL of the MP3 file):
Now if only we could have a player for other file formats like Ogg Vorbis or Windows Media Audio...