Baby Steps
Stuart responded to my post "Show Me" saying that, basically, he's in the same boat as (based on my personal experiences) most people producing software: what I call programming by accident. Specs are few and far between. He does actually have a tester, and that tester is "very good at finding ways to make our code fail", but she doesn't know how to program so automated tests are out of the question. Stuart knows there's a better way only because he has been blog-reading; the rest of his small group has no idea anything is wrong.
Stuart asks what he can do to help his company baby-step its way to doing things right. Good news, Stuart: it's easy to do!
He has already taken the first step: realizing he/his company has a problem. He has taken the next step too: starting a search for solutions.
A good place to start is keeping track of your bugs. Please use something that was built for this purpose. Don't use a spreadsheet. Don't use email. There are lots of good programs available for every price range. You can get away with a home-grown database, but you will almost certainly come to regret it before long. You don't really want to build a UI for custom queries and reports. Trust me on this.
Now that you're logging each and every bug, start analyzing them. You will find that certain areas of your app are bug farms; focus on these areas and clean them up. You will find that certain types of bugs occur over and over; figure out why. Figure out what needs to change to squash them once and for all. Periodically go though every closed bug and try it on your current build. You will find regressions; determine why each bug was reintroduced and change your process to keep it from happening again.
Start writing your test cases down. Ad-hocing tests is a good way to start testing, but you don't really know what testing you are and aren't doing if you aren't recording your test cases. For automated tests, the code itself is sufficient, but manual tests need to be written down somewhere you can find them again. Organize them in a logical fashion, but don't stress over this to start. Their organization is easy to change later on when you have a better understanding of what exactly you have to organize.
Start writing specs. Write them yourself if you have to. Who writes them is much less important than recording what a feature is expected to do, how it is expected to behave, and most importantly what the user benefit is. A spec doesn't need to be a lengthy document so long as it clearly records this information. In fact, all those test cases you are now keeping track of can serve as most of the spec quite nicely! (Just remember to write the user benefit for each feature down somewhere.)
James Whittaker's books How To Break Software and How To Break Software Security are excellent introductions to testing. Cem Kaner's book Testing Computer Software is a classic. Brian Marick's website testing.com has lots of good information, as does StickyMinds. The XProgramming and Agile Alliance are good starting points for learning about XP and other agile methodologies, all of which tightly integrate testing into the development process. And these are just the tip of the iceberg.
Invest in your tester. You're twice blessed: not only do you have a tester, but by your account she's quite good. Do everything you can think of to help her get better. Buy her a subscription to testing magazines. Introduce her to the testing-focused blogs and websites. Find out whether she's interested in learning how to program. If she is, send her to a intensive training program. Pay for her to enroll in programming courses at your local colleges. Take an hour each day of your time to teach her. Give her an hour each day to teach herself. If she isn't interested in learning to program, find out why not. Ask her whether she's tired of running the same tests over and over and over. Ask her whether she would like a simple way to ensure fixed bugs never reoccur, and to instantly know when they do. Show her how much more time she'll have to find new and interesting bugs if she automates the ones she already knows about.
Start writing tests yourself. Just because your tester can't write automated tests doesn't mean you can't either. Write unit tests. Write integration tests. Write user acceptance tests. They don't even have to be automated! (In fact, I would advise *not* automating at first. Test manually for while and get a feel for what exactly you want out of an automated test framework. Then pick one to use or build your own.)
Use your tester. She can do so much more than just make your code break! Have her teach you how to test. Walk her through your design before it's coded and let her find weak spots and problems. Ask her to pair with you when you write your unit tests.
As you start to get "normal" testing, expand into all the other kinds of testing: Internationalization. Localizability. Accessibility. Security. Performance. Stress. Start with one, get good at it, then move on to another one.
Start tracking metrics about your code. What makes a good metric is a religious debate, but the ones you start with don't have to be good. Track numbers of bugs per feature and overall. Track customer complaints. Track code churn (numbers of changes per file/method/feature). Whatever you pick, find out where you are now, decide where you want them to be, then make a plan for getting there. If a metric turns out to not be useful, dump it and choose another one. Remember that metrics should work for you, not vice versa.
Use metrics once you have some. "Oh look: that module that we unit tested the stuffing out of had one-tenth the number of bugs as the other modules." "That feature where we had a well-defined spec to work from made it through the ship cycle in record time because we worked out most of the issues in the spec rather than in testing." "Unit testing that module took twenty hours but saved us fifty hours in debugging, giving us a net gain of thirty hours." "Since we started pairing with our tester to write unit tests our defect count has dropped to ten percent of previous levels." Management usually likes to hear these kinds of things. <g/>
You don't necessarily need permission to do any of this, either. I'm not suggesting lying to management or anything, but do you ask permission to have a coworker help you debug an error? If it results in a more stable, more reliable, more usable product, I bet they won't care how you made it happen. If they do get curious, explain the changes you're making and the effects they're having. If they're skeptical, you'll have the data to back you up. If they still tell you stop it, then quit and find a job with someone that will let you do things the right way. Life is too short to waste time supporting people that don't want quality.
*** Comments, questions, feedback? Want a fun job on a great team? Send two coding samples and an explanation of why you chose them, and of course your resume, to me at michhu at microsoft dot com. I need a tester and my team needs a data binding developer, program managers, and a product manager. Great coding skills required for all positions.