I've been sitting in a lot of code reviews and code inspections where somebody suggests a change to optimize something. The response from the author is almost always; but that's premature optimization! End of argument... But I almost never hear somebody doing the opposite. I came to think about this again a few days ago when it happened again and just a few hours later I read this post about the subject.
Premature optimization is not black or white. Some things we know are bad for performance (like memory allocation and copies in a high performance C program) and nobody questions if you optimize that without actual proof it's a problem. As the post above describes it's all about paying a tax up front that is big enough to avoid a large fine later. I don't think the idea behind the quote about premature optimization is about making no optimizations. It's about making a reasonable amount of optimizations. Kind of the same kind of trade-off you have to do when it comes to writing tests. Is it worth adding a test for code that you know will pass just because to document the expected behavior? Sometimes yes, but not always as this post points out.