Slacker Astronomy
I've been listening to the Slacker Astronomy podcast for a few weeks now. It's really nice to have some hard science presented in an enjoyable and humorous context.
Noteworthy shows:
If you've ever wanted to know how we know the age of the universe and that the big bang happened, listen to this.
The Big Bad Astronomer - Show #5 (MP3 extended-version (longer, but more interesting))
An interview with the creator of Bad Astronomy.
I use Doppler as my client. The betas for Doppler are distributed as podcasts (bitcasts? codecasts?), so new versions just show up on your machine, and you can install them as you wish.