Testing And Customers: Getting Answers from Customers

Bwill asks “Do you have any favorite techniques to obtain answers to your questions
about customers? How do you convince yourself that your answers to those questions
are 'good enough'?”

My favorite techniques include a variety of data gathering tactics that include both
1st, 2nd and 3rd hand techniques. 

1st Hand includes my own understanding of the feature, how I want to work with and
interact with the feature.  Here I make the assumption that I’m a potential customer
of the product and have some interest in it or else I wouldn’t choose to work on improving
it.  Although its important to recognize that your use/perceptions are skewed
from your personal involvement in the product itself you can use your own assumptions
as starting points for further investigation.  Your own opinions however could
prove to be weighted least in the end as you start to understand work-styles and perceptions
that fall outside your own patterns.

My 2nd hand data gathering involves anything that creates a quality connection between
myself and real (intended target) potential customers that don’t work first hand with
the product being tested.  I read newsgroups about my own product as well as
those for competing products.  People that are loyal to competing products are
more willing than you’d imagine sharing with you what key scenarios you might be missing. 
I will create my own usage surveys targeted towards the feature in question. 
These, for example, led to a bunch of changes to the design of the VS Start Page. 
These surveys, in addition to gathering lots of raw data, served as great conversation
starters with customers that care about your product.  In these threads you can
dig deeper than you intended with the broad survey to see why people answered one
way or another.  Eventually you form, what some people would call, your customer
counsel of close key influencers you can bounce ideas off of.  A lot of these
people have a direct connection to me through IM.

3rd hand is information about customers that may be spoon-fed to you through designers,
usability engineers, co-workers, the media, etc.

I find the 2nd hand information to be the most valuable of the three.  This is
because I can use these techniques to be my verification method for information gathered
through 1st or 3rd hand approaches. 

Good Question,

Josh