Wednesday, February 04, 2009

Five Ways to do More With Less in Software Testing

It seems that many test managers are in the squeeze of tight testing budgets or no budgets at all. That means there is a lot of interest in doing more with less. The economy is bad, but we still have the need to do software testing well.

While I understand the need for efficiency in today's economy, this is not a new message. When I first started training in software testing back in 1990, there was a recession underway. There was also a quality crisis as U.S. manufacturers were battling against higher quality foreign imported items which included everything from VCRs to cars.

We learned how to recoup for the cost or quality with high-quality methods. This is what Phillip Crosby wrote about in his book "Quality is Free". In IT, however, the message of the economy of high quality has been lost over the years. In fact, in recent years I have found that senior IT and business management in many of the companies I have worked with will spend lots of money and take lots of risks with little to show for it.

So, for this window of time (hopefully a small stretch of time), the kinds of things I'll be discussing here have a home in the minds of people. However, the principles are profitable in any economy and we need to remember them.

#1 - Know What You are Doing

The number one value of training and skill building in my opinion is to learn the best and most efficient ways of doing things. After all, someone with no training could attempt just about any job but the results wouldn't be very good in most cases.

In software testing, there is a popular misconception that anyone can test. I have learned that many people have the ability to test, but there are skills that must be developed before someone can do a decent job of testing. And, some people just don't have the mindset or patience to be a good tester.

Training is how you learn what to do at the right times. Repetition is an important part of the training process. So, a three-day class once a year isn't enough to build skills.

Training can be expensive, but doesn't have to be. There are effective options:

* Self-study on the web and by reading books
* Team study such as reading and discussing the same book for a month or more
* Design and conduct your own in-house mini-course
* Have someone share a great new technique or tool at each test team meeting
* Webinars and teleconferences
* E-learning - its better than it used to be!

#2 - Reuse What You Use

This is really basic, but can be a big timesaver. If you have a good test plan or test strategy for one project, make a template and use it as the basis for other projects.

This idea can extend to detailed test design and test automation.

Perhaps the hardest part of reuse is managing the items as they are shared. This is the situation when people share and reuse test cases and test automation. There must be control over the items.

#3 - Define Tests Efficiently

There is an economy of testing. More tests are not necessarily better. In fact, each test case you have is one more you must maintain and perform. If a test case isn't adding specific value to your understanding of the software under test, you should consider why it is in the test.

Many people are surprised that once they start to analyze combinations of tests, how few tests they actually need.

While not every project is a good application of pairwise testing, it is a good technique to explore and try.

#4 - Automate Well

There is a great potential in good test automation. Some have realized this value is actual practice while others still struggle. In my experience, test automation is one of those areas where you can spend a lot of money in a hurry with little to show for it.

There are some free test tools such as FitNesse ( that can be helpful, as well as scripting languages. Other free tools are at

It's good to learn the lessons of automation on the free and less expensive tools before moving on to the more robust and expensive tools.

#5 - Learn From Your Defects

Each defect tells a story. If you take the top 10 defects (as measured by the frequency and impact) in the previous month and perform root cause analysis to learn why the defect occurred, you will make significant strides toward improving your processes. The main investment in this effort is time. It will also take management support to fix the processes. However, these are things you can often do with very little direct cash outlay and they yield big value.

#5.5 - Bonus - Write and Use Checklists

Checklists are easy and inexpensive to create. They help improve processes and prevent mistakes. They also add consistency as to how things are done. If you don't believe me about the value of checklists, ask a pilot.


Going back in time again, one of the first books I read when starting out as a consultant in the early 90's was Jerry Weinberg's, The Secrets of Consulting. One of the laws is the "Law of Raspberry Jam", which basically says the wider you spread something, the thinner it gets.

Like jam, you can only spread people and tests so far before they lose their effectiveness and value.

We should be lean and mean in our testing because we just don't have the luxury of time (and right now, of money). So it's a good thing to learn efficient ways of testing. Just remember there are limits.

To hear my podcast on this topic, visit