Here are Eric Shupps top five tips for improving web part performance (feel free to add your suggestions to the comments):
Take Out the Garbage
One of the most common mistakes that new SharePoint developers make is failing to properly dispose of the objects they create. This is especially true of SPSite and SPWeb objects, mostly due to the fact that these are used everywhere and it's all too easy to initiate them then forget they are there. The trouble with such objects is that they are almost completely invisible to ASP.NET's trusty garbage collector. This recycle-happy process is quite good at showing up on time and getting rid of messy memory fragments lying around but it can't toss out what it doesn't see. Each of those SPSite and SPWeb objects lurking in the shadows consumes several megabytes of memory; get a few hundred of them loitering about and your application pools will have a real vagrant problem. To ease the strain, be sure to call the Dispose() method on all persisted objects or, better yet, wrap them in Using(…) statements and let the framework do your dirty work for you.
Get Out of the Loop
SharePoint has a pretty robust drivetrain, capable of hauling hundreds of thousands - even millions - of list items over hill and dale without breaking much of a sweat. But slap a foreach loop on a list with a few thousand items in it and watch all that well-oiled machinery sputter to a crawl. Iterating through large lists using an indexer is like pouring sand in your gas tank – it's hard to get anywhere running on sludge. The simple fact is that lists weren't designed to withstand this kind of abuse. All those announcements, issues, discussions, contacts, tasks and what not scattered about that big corporate intranet are stored in a single SQL table being accessed continually by a whole bunch of built-in procedures. Looping through all those items just to find the few you are looking for has to contend with every other read/write operation in the pipeline. Instead, use CAML queries or web services to retrieve a collection of just the items you want and leave the rest alone. Don't get drawn in by all those shiny indexing examples in the SDK; they may look good but you're going to pay a steep price at the pump.
Make a List, Check it Once
While it may be OK for Jolly ol' Saint Nick to be making lists willy nilly and checking them all twice, we SharePointers need to keep our data traffic down to a minimum. This means making as few calls to list objects as possible. Every time a list object's items are accessed it requires a round trip to the database, adding unnecessary overhead and latency to the equation. Instead of retrieving list items using an index value, such as SPList.Items[i], first get the collection of list items (SPListItemCollection items = SPList.Items;) then retrieve an item out of that collection (items or items["Title"]). This isolates the request to an in-memory object and prevents SharePoint from having to make successive call to the database, which has much more important things to do than hang around waiting for someone's custom code to nag at it over and over again. That's just bad manners.
Pimp Your (Over)Ride
One Web to Rule Them All
For reasons that defy biology, SPWeb objects seem to reproduce in code faster than rabbits in springtime. How many times have you been strolling down a bright sunny Region and run smack dab into SPWeb after SPWeb, all chewing happily away on the same list? Pause for a few seconds to shoo them away and when you turn around five more pop up. Trouble is, SPWeb (as mentioned earlier) is no happy-go-lucky Easter Bunny – that little hopalong can chew up a whole bunch of w3wp's in nothing flat. So what to do? Well, start by invoking SPWeb objects only when the context changes (i.e. accessing lists in two different site collections). Then start passing those objects around instead of invoking new ones every time. Finally, if all the code being executed is confined to the web the user is currently browsing, use SPContext.Current.Web instead of creating a new SPWeb object from scratch. And for heaven's sake, don't let those SPWebs get too close to each other in the same method – you'll be chasing those little rascals down until next winter.