The return of A-Z Testing in Production – STPCon Fall 2012

Download the Slide Deck: PPT PDF The folks at STPCon had asked me to give an encore presentation of my March 2012 talk A to Z Testing in Production: TiP Methodologies, Techniques, and Examples. The new talk has some substantial updates including more focus on the use of Big-Data and a simplified categorization of test…


A Summary of the Amazon Web Services June 29 Outage

  Summary of the AWS Service Event in the US East Region …from Amazon <>   The event was triggered during a large scale electrical storm which swept through the Northern Virginia area   Though the resources in this datacenter, including Elastic Compute Cloud (EC2) instances, Elastic Block Store (EBS) storage volumes, Relational Database Service…


A to Z Testing in Production: TiP Methodologies, Techniques, and Examples at STPCon 2012

Download the Slide Deck PPT PDF Listen to the audio (requires free registration with STP – they’re good folks, so go ahead and register) A to Z Testing in Production: TiP Methodologies, Techniques, and Examples Testing in production (TiP) is a set of software testing methodologies that utilizes real users and production environments in a…


Failure is Always an Option

Design Strategies for Cloud Hosted Services The title phrase was made popular by Adam Savage of MythBusters fame (pictured to the right… the bunny is made of C4).  However it is also motto to live by when designing and deploying services to run on a Cloud platform.  We of course do not accept failure of…


Testing in Production (TiP) – It Really Happens–Examples from Facebook, Amazon, Google, and Microsoft

  Testing in Production (TiP) really happens, and yes that it s a good thing! Here are some examples from Facebook, Amazon, Google, and Microsoft. The Methodologies This is not a complete list of TiP methodologies, but explains the ones used in the examples given A/B Testing (aka Online Controlled Experimentation): I explain this in…


Testing with Real Users – User Interaction and Beyond, with Online Experimentation

My talk at the Better Software Conference was a lot of fun.  It was well received.  If you want to check out the slides, take a look here. Evidence shows than more than half of the ideas that we think will improve the user experience actually fail to do so—and some actually make it worse….


Software Testing Cage Match: vs. Microsoft

While I previously made some comparisons between and Microsoft’s different approaches to software testing in Building Services: The Death of Big Up-Front Testing (BUFT)?, I think now would be a fun and interesting time to do a deeper dive on this. Before I joined in 2005 as an SDET, while I was interviewing…


What The Heck Is Cloud Computing? – A Brief Re-Look

In a previous post, “What The Heck Is Cloud Computing?” I broke the cloud up into three categories.  however I do not think I did a good job of delineating these categories with examples.  So here goes: IaaS: Infrastructure as a Service   Storage: Amazon S3 (BLOB storage), Amazon SDB (Table Storage), Microsoft Windows Azure Simple…


Building Services: The Death of Big Up-Front Testing (BUFT)?

 Let’s start with some definitions: Death Demise, end, termination, the state of no longer existing BUFT “Big Up-Front Testing” an activity or “Big Up-Front Test” a group that carries out that activity (the latter sounds cool, like “Big Pharma”) It’s an acronym of my own invention. At least I believe this to be the case…


Feeling TiPsy…Testing in Production Success and Horror Stories

One topic I hope to return to many times in this blog is that of TiP, or Testing in Production.  For a good introduction to this check out Ken Johnston’s blog entry TIP-ING SERVICES TESTING BLOG #1: THE EXECUTIVE SUMMARY which is an updated version of the executive summary of the ThinkWeek paper that I…