When Google added IMAP support they turned every label into a folder, including the spam label. This makes total sense, but I get far more spam than real emails and when I use IMAP I have to download all of those spam messages.  Or do I?  I just stumble upon a Gmail Labs extension called "Advance IMAP settings" which gives you the ability to control which labels are downloaded via the IMAP protocol.  Install the extension and go to Settings > Labels.  Each label/folder has a checkbox next to it called "Show in IMAP".  Uncheck the one called spam and you're done.  No more waiting for your spam to download.