Drilling for Gold

I was watching Gold Rush last Friday night on Discovery channel. The crew felt that they needed assurance that there was some gold in the ground below them. As one of the guys remarked, "Unfortunately, there is no arrow on the map that says, 'dig for gold here'".

In fact, one veteran gold miner said, "No drill, no dig."

So, they hired a guy to bring a drilling truck out to the site to drill all the way down to bedrock. Then, they removed the dirt from the drill bit, placed it into individual buckets (one per hole), and panned the dirt for gold.

By doing this, they confirmed there was gold in the ground where they were currently digging, and no gold at all in their "plan B" location.

Does this sound familiar? They were "testing" for gold by taking samples. The results of their tests were very valuable in guiding the next steps of their work. The risk is that it costs about $1,000 per day just in fuel costs to run the earth movers.

I think this is a really great analogy of what we do as testers, especially in black-box, top-down testing. The defects are like the gold. In fact, a software defect is more valuable than one ounce of gold when you consider the cost savings of post-implementation re-work! Then, when we learn from the defects and improve our practices, the defects have even greater value.

The main difference between gold mining and testing is that we're not always sure when something is a defect, but you can't miss gold.

My approach in top-down testing is to cover the critical workflow processes (scraping the surface). When defects are found, drill down to find others. This form of sampling makes use of the principle that defects tend to cluster.

Like gold mining, in testing, there are no indicators that say, "Look here for defects." We have to sample wisely to gain insight where to focus our efforts. There's gold (or defects) in them there hills!