Randy Rice's Software Testing & Quality Blog

Webinar Recording - Everything You Wanted to Know About ISTQB Advanced Certifications

2012-01-22T17:03:00.002-06:00

Thanks to everyone that attend the webinar on Friday.

Here is the recording:

http://www.anymeeting.com/riceconsulting/EA50DB838849

Here are the slides:

http://www.softwaretestingtrainingonline.com/cc/public_pdf/Everything%20You%20Wanted%20to%20Know%20About%20ISTQB%20Certification.pdf

There was a question about the need to read/understand code for the Advanced Technical Test Analyst certification. Here are two resources for those that would like to know about coding:

Here are my ISTQB Advanced course links:

http://riceconsulting.com/home/index.php/Events/istqb-advanced-level-public-courses.html

http://riceconsulting.com/home/index.php/ISTQB-Software-Tester-Certification/istqb-advanced-study-groups-forming.html

http://www.mysoftwaretesting.com/ISTQB_Advanced_Test_Analyst_Course_Public_Course_p/ctaltapub.htm

Syllabus:

http://www.astqb.org/documents/AdvancedSyllabus2007.pdf

Thanks again!

Randy

Webinar - Everything You Wanted to Know About ISTQB Advanced Certifications

2012-01-11T14:58:00.001-06:00

Many people who hold the ISTQB Foundation Level certification are curious about what is involved in the ISTQB advanced level certifications. This session will walk you through the structure of the advanced certifications, what you need to know, the pre-requisites, ways to prepare and the value of holding an advanced certification. There will be plenty of time for questions along the way.

I hold all three ISTQB advanced certifications and will explain the things you need to know about ISTQB advanced certification.

Time: 1/20/2012 1:00 PM (UTC-06:00) Central Time (US & Canada) for 60 minutes

Register: http://www.anymeeting.com/PIID=EB54DE818249

I hope to see you there!

Randy

Book Review - Making it Big in Software

2012-01-05T22:59:00.005-06:00

There is a great "secret" I tell software professionals. That "secret" is that if you want to rise to the top of your field, it's not that hard to do because so few people do the simple things to rise to the top. You will learn those things in this book.

Making it Big in Software brings a great perspective to the idea of breaking through to becoming an elite thought leader in the software profession. The first thing that caught my eye was the stellar nature of the people Sam Lightstone interviewed for this book. These include James Gosling, the inventor of Java, Steve Wozniak, inventor of the Apple computer, Grady Booch, co-founder of Rational Software, and many other luminaries in our field.

Lightstone anticipates the question most people have right out of the gate, "Why bother?"

"But with long hours, considerable stress, and no guarantees, the obvious question is whether it’s even worth trying to make it big. I believe the answer is unequivocally yes. The most compelling reason is that, in most cases, you have to show up to the office and work like a lunatic anyway—it’s really not optional (if you want to eat). So if the difference between being a midlevel career programmer and making it big is an incremental strategic investment of time and energy, then it’s more than worth it for you and for your family. In the long run, the benefits are significant: a more satisfying career, greater influence and impact within your company and the industry, more fun, and more money. And while there may not be less “crap” to do, at least it’s strategic work rather than “grunt” work."

Some of the enticing things about making it big are:

Fun and interesting work
Corporate and industrial influence
The betterment of society
Freedom to work on what you want, when you want (Lightstone makes clear that this is what you want to work on, not how much you have to work!)
Fame
Travel

(As an aside, I humbly say that as a minor software testing celebrity, I experience these things on a regular basis and it is good. While travel can become a chore, it is cool to teach in Rome twice a year.)

Software testers and QA professionals will appreciate Chapter 2, "What Good Software is Really About." This chapter is a concise case for better software and why we need to refine our craft, profession, or whatever you call what we do as software people.

Lightstone lays out all kinds of practical, real-world advice, such as "What to Look for in a Company." I like this list and it reinforces a key idea that you do not have to strike out on your own to make it big.

"1. Is this a company that has experience in building professional, high-quality systems?

2. Are there really talented people here I can learn from?

3. Is the position I’m being offered one that is interesting, with long-term growth potential on something I can believe in?

4. Do they have savvy business executives who really understand the business requirements for success and have a track record for delivering it?

5. Does the company have clarity of vision for the product it produces?

6. Is there an independent research arm?

7. How does the company innovate, and how profound has their innovation been?

8. Is the work environment pleasant and flexible, and does it suit my lifestyle?

9. Does the company seem stable? Do I believe it will still be around in ten years?

10. Is the pay in line with industry standards?"

The book goes back and forth between interviews and practical guidance, which is a good thing. The interviews let you get inside the heads and hearts of these gurus, while the guidance gives you a plan of attack.

Good economy or bad economy, it doesn't matter in terms of the importance of standing out and making your mark. Economies rise and fall. We all need to learn to not let our jobs distract us from our careers. This book helps light the way to do that. I highly recommend it to anyone in the software field!

Killer Weeds and Killer Defects

2012-01-05T21:22:00.011-06:00

Recently I was listening to a radio interview about genetically modified crops. During the discussion, the guest mentioned the fact that farmers are starting to see “killer weeds.” These weeds are resilient to known herbicides which makes them very difficult to control or eradicate.

Being the tester I am, I immediately thought of the “Pesticide Paradox” that Boris Beizer wrote about in the 1980’s in his book, Software Testing Techniques. That principle says that just like bugs that become resilient over time to pesticides, software tests can become ineffective at finding new defects.

Another way to express this principle is to realize that your tests will grow weaker over time in terms of finding new defects.

My way of saying this is that when you repeat tests with the same conditions and test data, the tests become more confirmatory in nature than being discovery-oriented.

Similarly, there is the minefield example in which we can see that the safest way to cross a minefield is to walk in the footsteps of someone who has been successful previously in crossing from one side to the next. Seeing software defects as a mine, if we go down proven paths we don’t hit the mines.

Of course, in testing our goal is to hit the mines. As I say, “Failure is not an option, it is an objective.”

This discussion about killer weeds got me thinking about some other tie-ins with software testing and the “killer defects” we deal with.

Weeds are Ubiquitous

I don’t have a green thumb, I have a killer thumb. I’m sorry, but plants and I don’t get along that well. I have been successful in growing some things, like tomatoes, but my climate is harsh and the main things I can grow are grass and weeds.

However, I seem to have no problem with growing weeds. Have you ever noticed how resilient weeds are? They can grow in the cracks of a sidewalk!

Similarly, software defects are everywhere. Those of us in the weed control business of software (testers), have an abundant supply of defects to find. But like the killer weeds, it seems that no matter how often we spray, or pull, or mow, the weeds return.

Even when I pull the weeds out by the roots, they return because the seeds from neighboring yards (and my own yard) float into the yard or garden. Before long, there they are again.

What are the seeds of your defects and where do they originate? Inadequate requirements? Bad code? Inadequate testing?

The good news is that in software development we can actually control the seeds of defects at their source. It’s not always easy. In fact it hardly ever is easy, which leads me to the next point.

Weed Control is Maintenance

I really wish I could just spray or weed once a year, but that doesn’t work. In Oklahoma last summer, we had 63 days that were 100 degrees or more, plus we were in a very severe drought. Basically, most of the vegetation died, then caught on fire. I told people, “Welcome to Hell.”

Every blade of grass in the vacant lot next door to me was brown. There were inch-wide cracks in the clay soil that had the same consistency of bricks. There were also bright green weeds! Yes, the weeds thrived even in the absence of water in soil as hard as bricks.

Have you ever asked why, despite all your testing and other efforts, the defects seem to occur? The best answer I have is that people are human and humans make mistakes. These mistakes can be misinterpretation of needs, incorrect implementation of solutions, or just forgetfulness.

Another reason for mistakes may be the rate and nature of change. The faster we go, the easier it is to forget something or to simply become careless.

Perhaps the number one thing I hear testers complain about is maintaining their tests. This causes me to ask, “Where did this false expectation start that says tests are maintenance-free?”

The fact is that most tests will require maintenance, just like the software we are testing. However, the thing we deal with as testers that is unique is that changing one line of code may require changes hundreds of tests.

Some may say that this is why agile testing is great. I agree that agile approaches help. However, even in agile there are tests you want to remember and repeat. Yes, these may be automated. However, that may not matter in terms of maintenance since changes ripple through automated tests as well as manual ones.

Timing is Critical

Since I am horticulturally-challenged, I hire my friend Marty to spray and fertilize my yard five times a year. The first spraying in late winter is a pre-emergent. This treatment is very important to kill the weeds while they are still in the seed stage. Failure to catch weeds early means you play catch-up the rest of the year!

In software projects, early defect prevention and detection methods are key to stopping defects before they “bloom” and become larger and nastier. Activities such as reviews, walkthroughs and inspections can be your pre-emergent detection of defects.

Know Your Weeds

Marty doesn’t spray for all possible weeds in my yard. Instead, he sprays for those weeds common to our area. If rogue weeds appear, he sprays them individually as needed.

If you have ever ventured out to the hardware or garden store to buy weed killer, you know that there are many kinds, each oriented to specific types of weeds – grassy weeds, broadleaf weeds, and so on. There are treatments such as Roundup that will kill every bit of vegetation, but his is not a good option if you want to keep the good plants. That is overkill except when you want to clear out everything.

In testing, we need to know the types of defects we most often encounter. We also know that defects tend to cluster and we also know that people fall into patterns of repeating mistakes.

If we orient our tests to find the defects we most commonly encounter, that’s a good first pass. In the past, I have called this “starting a bug collection.” Now, I can also call it “a weed collection.”

Then, we can drill-down down as needed to branch out and find new types of defects.

Conclusion

Basically, we can pull weeds (a.k.a software defects), or we can work on preventing them. Pulling the weeds is hard work and seems to never end (sound familiar?). Preventing weeds takes discipline as well, but spraying in advance is easier and cheaper in the long run. Similarly, preventing defects with process improvement and finding defects early with reviews pays off in the long run.

If you know your source and types of defects, you can test wider and then focus on trouble spots. Just remember that the tests that work well today, may not be as productive in finding new defects the next time you test.

In the second installment of this article, I’ll explore why these killer defects are so hard to eradicate and the results of failing to find and eliminate them.

Video on How to De-Motivate Your Team

2011-12-16T11:22:00.002-06:00

Hi Folks,

The featured video this week on Techwell is from the lighting keynote I did at StarEast 2011:

A little context to help understand my remarks: 1) Osama Bin Laden had just been killed and 2) there was a DECA conference at the hotel at the same time with many teenagers running around. They were well-behaved for the most part, but at the same time, we were happy to see them leave.

Enjoy!

Randy

Drilling for Gold

2011-11-28T21:14:00.003-06:00

I was watching Gold Rush last Friday night on Discovery channel. The crew felt that they needed assurance that there was some gold in the ground below them. As one of the guys remarked, "Unfortunately, there is no arrow on the map that says, 'dig for gold here'".

In fact, one veteran gold miner said, "No drill, no dig."

So, they hired a guy to bring a drilling truck out to the site to drill all the way down to bedrock. Then, they removed the dirt from the drill bit, placed it into individual buckets (one per hole), and panned the dirt for gold.

By doing this, they confirmed there was gold in the ground where they were currently digging, and no gold at all in their "plan B" location.

Does this sound familiar? They were "testing" for gold by taking samples. The results of their tests were very valuable in guiding the next steps of their work. The risk is that it costs about $1,000 per day just in fuel costs to run the earth movers.

I think this is a really great analogy of what we do as testers, especially in black-box, top-down testing. The defects are like the gold. In fact, a software defect is more valuable than one ounce of gold when you consider the cost savings of post-implementation re-work! Then, when we learn from the defects and improve our practices, the defects have even greater value.

The main difference between gold mining and testing is that we're not always sure when something is a defect, but you can't miss gold.

My approach in top-down testing is to cover the critical workflow processes (scraping the surface). When defects are found, drill down to find others. This form of sampling makes use of the principle that defects tend to cluster.

Like gold mining, in testing, there are no indicators that say, "Look here for defects." We have to sample wisely to gain insight where to focus our efforts. There's gold (or defects) in them there hills!

Small Business Saturday and Cyber Monday Specials

2011-11-21T14:50:00.001-06:00

We are excited to be a part of Small Business Saturday!

This is a special opportunity for you to support us as a small business and to get major savings on our e-learning training courses in software testing and software quality.

Every e-learning course will be discounted from Saturday, Nov. 26 through midnight on Monday, Nov. 28th. This even includes our ISTQB Foundation-level course, which includes the exam and additional text book!

There are no limits. Buy as many courses as you like!

Plus, if you pay with American Express, you can get an extra $25 back!

American Express is promoting Small Business Saturday again this year. Register your American Express when it opens November 1st, and then use your registered card to shop at a participating small business on Saturday, November 26th. You’ll receive a $25 statement credit when you spend $25 or more at participating small businesses. From this press release:

"For the second year in a row, American Express will help drive traffic through the doors of small businesses with a special incentive: a $25 statement credit offer for Cardmembers when they register their Card and spend $25 or more on Small Business Saturday at any qualifying small business that accepts the American Express Card. Cardmembers will be able to register their cards in early November."

To register your American Express card, visit http://www.facebook.com/SmallBusinessSaturday

Here's how to get these savings on our e-learning courses:

1. If the training is for your team, get your budget allocation and approvals from your management. This is a great way to make the best use of remaining 2011 training funds!

2. If the training is just for you, don't spend all your money on Friday!

3. Visit our e-commerce site at http://www.mysoftwaretesting.com and buy the courses you like. The discounted prices will be shown.

Remember that you have 12 months of access to the courses, even after you complete the course! To see other benefits and features:

http://riceconsulting.com/home/index.php/e-Learning/benefits-of-e-learning.html

http://riceconsulting.com/home/index.php/e-Learning/features.html

To see demos of the courses:

http://softwaretestingtrainingonline.com/moodle/course/category.php?id=11

Of course, I am always happy to answer any questions:

http://riceconsulting.com/home/index.php/component/option,com_chronocontact/Itemid,90/

I hope you take advantage of this rare opportunity!

Randy

Book Review - The Economics of Software Quality

2011-11-09T08:26:00.007-06:00

Publisher: Addison-Wesley
ISBN-10: 0132582201 ISBN-13: 978-0132582209
Publication Date: August 3, 2011
587 pages, hardcover

In this book, authors Capers Jones and Olivier Bonsignour quantify the factors that influence software quality and provide information for people to gain insight into how their projects might compare to others. The measurements in this book are based on thousands of software projects.

One of my frequent complaints about the software industry is that we just don't measure very many things. However, thankfully there are people like Jones and Bonsignour that do have a rich source of metrics from enough projects that we can learn from them.

Capers Jones has long been considered the source for software quality metrics. To me, Capers is the "numbers guy" of our profession. With over 40 years in the field, Jones has a wealth of information he has maintained and published over many years.

Olivier Bonsignour is responsible for Research & Development and Product Management in a continual effort to build the world’s most advanced Application Intelligence technology. Prior to joining CAST, Mr. Bonsignour was the CIO for DGA, the advanced research division of the French Ministry of Defense.

For example, the authors state that "high quality levels are invariable associated with shorter-than-average development schedules and lower-than-average development costs." This finding is based on over 13,000 projects between 1973 and today.

The authors maintain that the real economic value of high quality software is not the cost to fix defects, but rather:

the reduced likelihood of canceled projects
the reduced risk of litigation
shortened development schedules
lower development costs
reduced warranty costs
increased customer satisfaction

This book addresses:

What is software quality and how do we define its value?
How can we estimate and measure software quality?
How can software defects be prevented?
How can we find and remove defects before testing?
What are effective ways to test software and measure its effectiveness?
What is the current state of post-delivery software defects?
How do projects of various characteristics (low, average and high-quality) compare?
How can technical debt be addressed from a business value perspective?

You will find a multitude of data from projects in a variety of industries, at various levels of quality, and at various levels of practice maturity. You will see by the numbers which project approaches work and which ones don't work very well.

By reading this book, you will gain insight not only into the current state of software quality, but you will also learn about measurement and metrics of software. These are critical things for any software quality professional to learn. In fact, after reading this book, you will know more about software measurement than 95% (that's my estimate) of testers and QA professionals.

I highly recommend this book, not only as a guide for software quality efforts, but also a benchmark for your own efforts.

The Greatest Risk of All

2011-11-08T10:50:00.012-06:00

I perform quite a few risk assessments on projects and in organizations. There is one risk common in almost every situation although it is not always identified. This risk is also something I have experienced generally in life. In fact, it is a risk so familiar it’s easy to miss. People get burned by this risk all the time.

Hans Solo understood this risk when he admonished young Luke Skywalker, “Don’t get too cocky, kid.”

There are many ways this risk can be seen:

People who tune out mentally in training because they think they already know the material. However, these people hardly ever ask any questions, either.
Project planners who indicate all contingencies are considered.
Everyday situations where we buy something, then learn the details about the product that are discovered only after trying to resolve a problem.

What is this risk that is so common and so elusive?

It is the “unknown unknowns.” There is nothing new about this. British Mathematician Alfred North Whitehead, who lived from 1861 to 1947, said “Not ignorance, but ignorance of ignorance is the death of knowledge”.

In an article published recently in the journal Science Communication, Jerome Ravetz writes:

“The idea of ignorance of ignorance is quite unfamiliar. Indeed, scientific culture generally suppresses awareness of ignorance. But ignorance of ignorance was quite well-known from Plato and Socrates onward; it became unpopular in the scientific revolution with Galileo and Descartes. Since then, the triumphalist faith that science would provide the good and the true has put ignorance to one side, and led scientists to the sin of pride in their scientific conquests.” 1

We know some things pretty well. For example, we know we need to test software. We also know we can’t find all the defects in software. We know where we have missed defects in the past.

The problem is, when we tackle something new, the rules change and no one is there to tell us what the rules are and how others play to the rules or cheat the rules. The risk comes into play when we ignore that fact or minimize it.

I find much of the programming on television to be lacking, but sometimes I do get hooked on a show. One “reality” show in particular, “Gold Rush” on the Discovery channel, has grabbed my attention. This show demonstrates the unknown unknowns very well.

Looking for Gold in All the Wrong Places

“Gold Rush Alaska ” is about a team of rookie gold miners who go to Alaska to find gold. The oldest member of the team tried many years ago to find gold, but that’s about all the experience that exists on the team. It seems that at almost every step in their gold finding effort, they discover what they didn’t know. The first hurdle came when they had to drive their main earth mover through a wide river because the bridge was too old and weak to handle the weight. Then came the time when they called in a gold mining expert who showed them how their dirt washer was letting much of the gold pass right out the exit and into the river. Once again, an “unknown unknown”. There were also dozens of other things these guys learned and are still learning. I admire their tenacity and faith. I hope they succeed before they run out of money and hope.

It is often said that “Beginners make dumb mistakes.” I prefer the way that Jerry Weinberg says it instead – “Beginners make beginner mistakes.” That’s a good thing for us software people to remember.

Phases of Realization

I have observed that people tend to go through phases when faced with a new situation:

1. Cluelessness (Blissful ignorance)

At this level, people are so unaware of their situation and risks, they “don’t know that they don’t know what they don’t know.” This is total cluelessness, but also very common. It’s like deciding to build a house without really understanding all the problems and stress. It’s very exciting at first to be thinking of your dream home, but then the decisions arise, delays are encountered, things may be done incorrectly and so forth. Ask anyone who has ever had a house built and they probably have a list of ten or more major things they would do differently. One of those things might be “not to have a house built.”

2. Curiosity

This is where you have enough objectivity to at least ask, “What are we missing?” You try to analyze and plan as much as possible, but it’s still only partial understanding of the possibilities. It’s much like going on a trip, thinking through all the clothes you will need, only to learn that the weather at your destination is much different than what you expected it to be.

3. Cautious awareness

In this phase, you are aware of the possibility of that “unknown unknowns” exist. You try your best to anticipate the unknown, but you hit the wall everyone hits. That is, you can’t prepare for everything. You might also think of this as “risk acceptance”.

4. Contingency planning

Realizing that you can’t prepare for everything totally, you put in place reserves to account for the unexpected. These are your Plan B, C and D for dealing with the unknown. In fact, some of the contingencies may seem totally outrageous.

In Oklahoma, where I live, most people have some type of tornado precautions and contingencies. Not so much for earthquakes until this week. Last Saturday evening, November 5, 2011, about 11 P.M. we had a 5.7 earthquake in central Oklahoma. In the past year, we have had a sharp increase in earthquakes. It’s odd to think that I have experienced more earthquakes in Oklahoma than I did while working nine months in San Francisco a couple of years ago! Sure, we’re small time compared to other places, but I’m thinking that more Oklahoma businesses are thinking about earthquake drills. 2

5. Confidence

Now that you understand the risk of the unknowns and have some back-up plans in place, you start to feel more confident in your ability to succeed. However, it is a false level of confidence in the contingencies.

Next to the early phases of cluelessness and curiosity, this may be the most dangerous phase because it easy for us to think we know more than we actually do know. We think, “Don’t worry, we have contingency plans in place.”

This reminds me of the time many years ago when one of my clients needed to restore the main database to recover from a power outage. Then they learned that the operator had failed to mount the second tape during the backup process. For some reason, the operator had misinterpreted the message that prompted for the next tape. This had been going on for months!

6. Comprehension

There is an old adage about judgment and experience. “To avoid mistakes, you need to have good judgment. To get good judgment, you have to make some mistakes.” Comforting, isn’t it?

There are two basic ways to learn: From your own mistakes and from other people’s mistakes. The good thing about learning from other people’s mistakes is that you avoid their pain. The bad thing is that we tend to observe these mistakes and think, “Oh, that not going to happen to us.”

A good thing that happens in this phase is that people may have the foresight to get an external opinion. This would have really helped the gold miners early on.

7. Calibration

Once you have asked the key questions, developed some contingencies, and reached out to others, you are ready to make adjustments. The question is, what will those adjustments look like?

Another bit of wisdom I learned from Jerry Weinberg is the “ Hudson Bay Start.” Back in the days of the Hudson Bay Trading Company, when the Canadian frontier was being settled, part of the plan was to travel west for one day. Then the next day, they would go back to the start and pick up the things they realized they missed the first time.

This is why I like pilot projects with low risk. You get a chance to learn what you don’t know. You can make mistakes without a lot of the downside risks.

It’s important to understand in this phase that the risk of the unknowns still exist. Going forward, you have to rely on your ability to adapt to new challenges and find a way to overcome them. This is where all of your experience and wisdom are needed.

8. Compilation and completion

It’s amazing how quickly we forget things. A good practice is to actually write down the things you learn and review them from time to time. My own practice for this is to keep a journal of the important lessons I experience. I may experience a lesson several times before I really “learn” it.

As someone has said, “All lessons will be repeated until learned.” I don’t know who said that, maybe I read it on a bumper sticker. But, I know it’s true.

Albert Einstein is often quoted, “Insanity: doing the same thing over and over again and expecting different results.”

My point here is that if we don’t learn from our mistakes and omissions, then we deserve the pain from them.

I’ll admit that achieving this kind of learning at an organizational level is difficult and rare. This is not about sponsoring training programs – it is much more experience-based than training. I think this level of learning must be personal. Organizations and institutions don’t learn – people do. It takes individuals to think, reflect and change their behavior.

What Can We Do?

This is the troubling thing about the unknown unknowns. You never totally mitigate this risk. However, there are some things we can do that may or may not help:

Stay humble and recognize that unknown unknowns exists.
Be flexible and adaptive when mistakes and problems occur.
Learn from yourself and others.
Reach out to others to ask questions before you embark on something new.
Keep a record of things you learn.
Have contingency plans and resource reserves, just don't place total trust in them.

Summary

The recent master of the concept of the unknown unknowns is Donald Rumsfeld. Unfortunately, people failed to follow the message and it was the brunt of jokes.

Here is how he stated it, transformed into poetic form:

The Unknown

As we know,
There are known knowns.
There are things we know we know.
We also know
There are known unknowns.
That is to say
We know there are some things
We do not know.
But there are also unknown unknowns,
The ones we don't know
We don't know.

—Feb. 12, 2002, Department of Defense news briefing

Here is a challenge for you. Check out the blog youarenotsosmart.com written by David McRainey. He has recently published his blog as a book titled, You are Not so Smart. It’s an interesting read. The premise as stated on the blog: "The central theme here is that you are unaware of how unaware you are. There is branch of psychology and an old and growing body of research with findings that suggest you have little idea why you act or think the way you do. Despite this, you create narratives to explain your own feelings, thoughts and behaviors, and these narratives become the story of your life."

I hope this article helps us all to realize the inherent risk of the unknown unknowns. I would like to hear your comments. How have you seen this risk?

1. Jerome R. Ravetz , “The Sin of Science - Ignorance of Ignorance”, Science Communication, September 2011

2. In fact, as I am writing this article, we experienced an F4 tornado and 4.7 earthquake within just a few hours of each other on November 7, 2011.

Self-Assessment Files from Becoming an Influential Test Team Leader at StarWest 2011

2011-10-05T16:44:00.002-05:00

Hi Folks,

I promised the people in my Tuesday tutorial at StarWest 2011 the Word versions of the two self-assessments: One on the people issues in testing and the other on building core competencies in testing. Here they are!

People self-assessment
Core competency self-assessment

Feel free to modify for your own purposes. Thanks to everyone who attended my tutorial. It was a great time!

Randy

Giving the Customer a Little Extra

2011-07-09T14:39:00.008-05:00

It's 108 today in Oklahoma City, so I find myself blogging, working indoors, and playing some guitar.

I had a revelation awhile back at Five Guys Burgers. I know, for many people, revelations happen in church, during a sunset, etc. Mine happen while I'm stuffing my face.

If you have ever eaten at Five Guys, you know the throw an extra handful of fries in your bag on top of everything. There's a good chance you may get full on the fries before you ever reach the actual container of fries. This got me thinking about why they did this. Then it struck me, this is a Purple Cow. How many other fast food places do that?

In contrast, I got a burger and large fries from one of our local chains (Braum's) recently. When I looked at the fries, I could count them. There were 15 I think. Plus, they were those frozen ones, unlike Five Guys which are cut in the store (like In-n-Out).

It probably costs five cents or less to throw in the extra handful of fries. In return, Five Guys gets loyal and raving customers. I've never seen an ad - print, TV, web, anything, but I've talked to many people who brag on them.

So, what's the attraction? What makes the difference?

Isn't it nice when you get more than you expect, not less?

This is our new Five Guys in Moore. Someone apparently thought it was a drive-in.

It's sad that in our society, people are cynics about deals. I see this all the time in dealing with our customers at Rice Consulting. I quote a price and people ask, "What's the catch?". It's simple. There are no catches.

For example, I give 12 months for people to finish an e-learning course. Most of our competitors give 90 days. If someone needs more time, I always extend the time to take a course.

After a class, I'm always available to answer post-class questions for free. During an e-learning class, I always answer questions as part of the class fee. After a consulting engagement, I always answer post-engagement questions.

In the past weeks, here are some other great examples:

We bought a 2010 Ford Explorer recently from Reynolds Ford in Norman, OK. I didn't haggle on the price, bought the extra protection package, got Ford financing, the works. The whole process took a little over an hour and went as smooth as silk. Our salesmen, Brent was helpful, friendly and knowledgeable. Then, I discovered there was only one set of keys. So, I called Jayna at the dealership for an extra set since they cost about $200 these days for the chipped keys. When I picked up my car from the service area, they had an extra set of keys for me. Oh, and when we took possession of the car, it had a full tank of gas, was washed and the tires were glossed. That's extra for the customer and I'm a fan now. Reynolds Ford knows the value of a lifetime customer is more than $200.

Today, I bought two pounds of coffee at a Starbucks in Norman. The barista offered me a free cup of coffee. Even at 108 degrees out, I accepted. Let it never be said that I ever turned down a free cup of coffee. As James Garner (a Norman native) said in Support Your Local Sheriff, "I never turned down a cup of coffee in my life."

I have actually asked for a free cup of coffee when buying several bags of coffee at other Starbucks and the response was "Sorry, we don't do that anymore." That left me unimpressed. I'm much more inclined now to buy from the Norman store.

There used to be a time when giving a little extra to the customer was common practice. Now it seems that the trend is to reduce the product and charge the same. I believe that's a short-sighted approach.

It appears to me that giving added value is the result of both good policy (that's the role of management) and good performance (that's the role of the person delivering the service.)

At Five Guys, their corporate policy is to top off the fries. At Braum's, their policy is to not under any circumstances exceed the size of the container.

In other cases, like at Reynold's Ford and Starbucks, it's the performance of the right people making the right decision by the customer.

In software testing, we need to be showing our added value. One way to do that is to give a little extra to your customers. Be helpful to them. Ask what they need. If they don't know what they need, help them see the possibilities.

Some people see their customers as "those which must be endured." For sure, there are challenges. I choose to see my customers as the reason I'm in business - to serve and to stay in business.

You may not have a "business" but I suggest that you have many of the same reasons to please your customers. Without them, you wouldn't have a job.

OK, enough for now. I'll go practice Black Dog (Led Zeppelin) and Bach Cello Prelude 1. What can I say? I have a wide range of musical tastes!

Have a great weekend!

Book Review - Documenting Software Architecture, 2nd. ED

2011-07-06T21:44:00.004-05:00

Authors: Clements, Bachmann, Bass, Garlan, Ivers, Little, Merson, Nord, Stafford

Addison-Wesley, 537 pages

As a software tester, I rely heavily on system documentation. Unfortunately, documentation is often missing, obsolete, or never created in the first place.

Also, I have a great appreciation for software architects and the work they produce. As a former developer, I used to struggle with the best ways to express system architecture diagrams. After all, there are so many methods available to document systems – UML being a major one, but there are others.

When I started reading this book, I was struck by its practicality, beautiful simplicity, and integration between authors. Everything I read in this book is written in a clear and understandable way. The authors understand that different audiences will read this book, so they give graphical (of course) guidance in which chapters are most applicable to architects, stakeholders and novices.

This book covers the basics, such as module views and module styles, component and connector views, allocation views and styles.

Part two of the book goes beyond the basics and gets into issues regarding levels of detail, deciding among alternatives, documenting interfaces and documenting behavior. Part three is devoted to building the architecture documentation.

There are appendices devoted to UML, SysML and AADL to show how architectural documentation is shown in each of these.

It would be tempting to say that this book is needed for new technologies, such as SOA and the cloud, which is true, but too narrow. Actually, this book can be applied to any technology or approach – traditional, agile, iterative, or anything. That’s because the one thing people ask for and struggle with is documentation. This is especially true for architectural documentation.

Read this book and apply the things in it and you will stand out on projects - in a good way. And, of course, that’s a good thing!

Test Automation is Not Automatic

2011-07-04T20:58:00.003-05:00

Recently while teaching a workshop on Testing Dirty Systems, I uttered this “Randyism” off the top of my head, “Test automation is not automatic.” I realized immediately that I had just concisely stated the problem in making test automation a reality in many organizations.

Most testers know that test automation is not automatic. (Wouldn’t it be great?) However, management many times does not know or accept that reality.

There are some test tools, such as unit test tools, that are practically automatically applied. My remarks in this article are aimed at the capture/playback and scripting tools for test automation.

The issues are that:

1) Not every test can or should be automated

2) For those tests that can be automated, it takes time and effort to build the automation

3) For those tests that have been automated, the tests must be maintained

4) It takes time to lean how to use a tool

5) It takes effort and planning to implement a test automation framework

None of these are automatic, even with the best of tools.

Not Every Test Can or Should be Automated

Think about the things you test that are not very repeatable. Or, they may be prone to constant change. Perhaps the things you test are developed in a technology that has little or no tool support.

Then, there are tests such as user acceptance tests that need people’s evaluation to judge acceptance.

Some tests require creativity and adaptation to perform. You may have to make judgments during the test, which may be too complex to describe in a script. Test automation leverages the mundane testing to give more time and attention to the unique tests.

Your job is to identify the tests that can be automated. (They don’t come labeled!) Then, you must understand the nature of the test, such as the pre-requisites, the steps to perform it, the exceptions and where to find the expected results. None of this is automatic. It’s all test design and test implementation.

For Those Tests That Can Be Automated, It Takes Time And Effort To Build The Automation

It’s one thing to automate a function, but another to design a good test of the function. That’s why capture/playback is so appealing, yet lacking. The issues are: What are you testing in the capture session? Then, how can you extend those tests to add value?

You have to apply approaches like data-driven testing and keyword-driven testing, which take time and effort to understand and implement. These are not “out of the box” deliverables.

For Those Tests That Have Been Automated, The Tests Must Be Maintained

This has been one of the consistent issues in test automation. There are things you can do to ease the maintenance burden, but it doesn’t do away with the issue.

For example, modular and reusable test scripts are very helpful in reducing the number of tests that must be maintained. Still, you must maintain the scripts you have.

This means you must know when the application has changed, what the changes were, and create new tests for those changes. This implies the presence of configuration management and traceability.

It Takes Time To Lean How To Use A Tool

Of course, the learning curve varies by tool, but the fact remains that a tool with sophisticated features will take some time to learn. The learning curve also varies by person. Some people with deep experience in automation may be able to learn very fast, but when you consider the wider deployment, most people will fall on the lower end of the experience scale.

The time also varies by the approach used to get training. Some people try self-learning which is noble, but typically takes longer than classroom training. Mentoring from an experienced person may be the best approach.

You must assess your organization’s skills and abilities to know even if mentoring will help. The best mentor in the world can’t help the person who isn’t ready to learn. Is that another “Randyism?”

It Takes Effort, Money And Planning To Implement A Test Automation Framework

The framework can take a variety of forms, but the context here is the organizational framework which provides a way to efficiently build and control test automation. Tools are only one-third of the picture. You also need processes with trained and motivated people to make everything work together.

Out of the box, tools are just software. A process framework helps with reuse and the maintenance of test automation.

Frameworks aren’t automatic, either. They must be adapted to fit each situation, therefore they require time, effort and funding to design and implement.

Conclusion

This article is certainly not an in-depth treatment of this topic. Entire books and training courses have been written to address these and other aspects of test automation.

I hope this expands on my simple statement “Test automation is not automatic” to provoke your own thinking on this reality.

For over twenty years now, I have seen a wide variety of test tool companies promote their tools as being effort-free, script-less, or however they choose to position their products. The evidence shows these are often empty claims. Just compare the numbers of people who have been successful in using test automation tools to those who have given up using the tools. It’s about 25% successful to 75% unsuccessful in my research.

I hope this article is an encouragement to those who have tried to implement test automation as well as to those who haven’t. It’s important to go into these projects with your eyes open and expectations at a realistic level. Once you get past the idea that the tools do all the work, you can do the planning and other work needed to increase your chances of success.

Test Manager Workshop with Randall Rice and William Perry

2011-07-04T20:46:00.002-05:00

Name your software testing challenge and other test mangers and the workshop faculty will give you solutions!

As a software test manager, you face pressure from all directions. Customers expect high-quality products, project managers want to get the projects delivered on time, your management expects you to do more with less, and your team needs your leadership.

You could take a class in software test leadership or management, but much of the knowledge comes from the front of the room. What if you could determine the topics based on your own needs?

This is not a class or a conference, but a highly interactive and personal event. You will spend time personally with William Perry and Randall Rice to discuss your specific testing needs.

You need a mentor! Personal mentoring is one of the most effective ways to get guidance and build knowledge. Mentoring fills the gap that training cannot fill by giving you face-to-face, personal feedback about how to deal with tough issues.

In this unique opportunity, you can spend two days with two respected test management consultants, William E. Perry and Randall W. Rice to gain insight and solve your specific testing problems. Perry and Rice literally wrote the book on people issues in software testing. William E. Perry has written over 30 books on software testing and quality assurance. They share over 80 years of professional software quality and testing experience in world-class organizations.

Any and all topics are on the table. As an example, some of the topics suggested by test managers are:

• People issues in software development and testing
• Testing large and complex legacy systems
• Testing SOA and cloud-based applications
• Test metrics and measurements for showing your value
• How to grow as a test team leader
• How to communicate your value and your team’s value
• How to grow your team
• Test tools and test automation
• Software QA processes
• Software forensics
• Testing internal controls
• Testing information security
• Applying statistical methods to software quality
• Gathering and defining testable user requirements
• Managing expectations
• The Ins and Outs of software test certifications
• Any topic of interest to you!

However, the final topics will be determined by the attendees during the conference. That ensures your topic will be discussed!

You will leave this event with confidence and knowledge in how to best approach your situations.

Location – Orlando, FL (exact venue to be announced later)
Dates – Thursday, October 27 and Friday, October 28, 2011
Cost - $1,295 per person

Seats are limited, so reserve yours early to avoid disappointment.

Bonus Option!

Attend one day earlier to have an extra opportunity with Randy and Bill to learn how to test dirty systems.

The day prior to the workshop, William Perry and Randall Rice will be conducting a one-day interactive workshop based on their new book, Testing Dirty Systems. In this workshop you will learn:

• How to apply statistical process control (SPC) to software development and maintenance
• How to acquire system knowledge
• How to plan and perform the testing of large, complex and undocumented systems
• How to measure and report the test results in value-added ways
• How to use test results to clean a dirty system

Location – Orlando, FL (exact venue to be announced later)
Date: Wednesday, October 26, 2011
Cost - $550 per person if attended as part of the two-day Software Test Managers’ Workshop, $690 if attending only the one-day event.

Click here to register.

Click here to ask any questions.

Credibility

2011-05-20T13:09:00.003-05:00

For those of you who have been in my tutorial Becoming an Influential Test Team Leader, you know I'm big on the topic of credibility. If people don't find you credible, they don't find your information credible either. So, credibility is huge for testers and test leaders.

Today, John Maxwell's "Leadership Word of the Day" is credibility. He has a nice one-minute video and you can also sign up to get these free at:

http://johnmaxwellteam.com/credibility/

If you want to view my 45 minute conference presentation on credibility, you can find it here:

http://softwaretestingtrainingonline.com/moodle/course/view.php?id=30

(When asked for a login, you can click the button that reads "login as a guest".)

I hope you listen to one or both of these and think about what it means to be credible as a person, a leader and a tester.

Have a great weekend!

Randy

Ten Proven Ways to De-Motivate Your Team

2011-05-04T18:08:00.009-05:00

Thanks everyone for your kind words about my lightning keynote today at StarEast.

Here is the complete script:

10. Set unreasonable “stretch” goals just to see how hard people will work.

It really doesn’t matter what the goals are, or what the deadlines are, just make them really hard to achieve. If you really want to wear people down, do this at least once every 3 months. Overlapping stretch goals are especially fun.

9. Never explain your rationale for decisions.

Reasons? You don’t need no stinking reasons! “Because I said so” works just fine. In fact, it’s good mental exercise for your team to try to figure out your irrational actions.

8. Assign meaningless tasks.

The most important thing about work is that people look busy at all times. Whether it’s writing a PowerPoint presentation for you to impress your boss, or just to test until 6 in the evening, make sure everyone always looks busy.

7. No matter how good something is, criticize it.

Especially the first time you review it. Get a nice, big, red marker and go crazy. Forget about the main point of the content and focus on sentence structure or specific words you don’t like (such as “the” or “that”). Before long, people will give up and stop trying to make something right the first time.

6. Take all the credit for yourself.

After all, this team is your creation, right? This is especially important to remember at bonus time. Otherwise, you may have to suffer financially. (Don’t forget to attend all senior management meetings alone.)

5. Solve problems by building a new bureaucracy.

Remember, there are no simple solutions, only simple people. You are a much more complex person than that. You can design forms, approval processes and even spreadsheets. Of course, once you implement this required bureaucracy, you need to police it some way, so you will need a special team of spies to make sure everyone follows the “new way of doing things.”

4. Listen…like a brick wall.

The trick is to make people think you are listening. So, look them right in the eye, nod approvingly, but let your mind roam. Then, you can fulfill all your ADD fantasies. NBA scores, weekend plans, lunch plans….you name it.

3. Refuse to consider ways to do the job more effectively.

Tools? We don’t need no stinking tools, either! Besides, we can’t afford all those fancy tools. Free tools? We can’t use those – we do have rules to follow, you know. Training? We had a class 5 years ago. Can’t you people remember anything? Learn at lunch? Heck no, your team is too busy working through lunch. (See point #8 – always look busy.)

2. Treat your team like they are machines that should never break down.

I mean really. Why do these people call in sick and let you down when you least expect it. Who’s going to write your status report? And don’t even get me started about bathroom breaks!!

1. Never, ever, in any circumstance, give anyone praise or recognition.

Otherwise, people would start to feel hope and happiness, like when we saw all the teenagers leaving for home today. Remember, if you never praise anyone, you don’t have to insult them, just ignore them. Eventually they will leave and you can hire someone else to de-motivate.

StarEast 2011 - Free and Cheap Test Tools

2011-05-02T20:47:00.002-05:00

Thanks to everyone who attended my half-day tutorial today! Here are the tools that were mentioned by people in the session, plus a few more from me:

Axiom - Requirements Management tool - http://www.iconcur-software.com/solutions.html

Dia (Visio alternative, helpful for drawing process flow diagrams, etc for test planning) - http://live.gnome.org/Dia

Gliffy (a web-based diagramming tool) http://www.gliffy.com/

Shapes - Shapes is a simple, elegant Diagramming app for Mac OS X Snow Leopard. http://shapesapp.com/

FreeMind (Mindmapping, useful for test design) - http://freemind.sourceforge.net/wiki/index.php/Main_Page

Runtestrun.com - Test case management

Sikuli - Sikuli is a visual technology to automate and test graphical user interfaces (GUI) using images (screenshots) - http://sikuli.org/

IcuTest - GUI Unit Testing for WPF - http://www.nxs-7.com/icu/

Selenium Grid - Selenium Grid transparently distribute your tests on multiple machines so that you can run your tests in parallel, cutting down the time required for running in-browser test suites. http://selenium-grid.seleniumhq.org/

eggPlant - Image based test automation - http://www.testplant.com/

TestRail - Test Management - http://www.gurock.com/testrail/

InCtrl - By monitoring the changes made to your system when you install new software it enables you to troubleshoot any unexpected problems that come up. http://www.pcmag.com/article2/0,2817,25126,00.asp

Paloma Print Perfect - STREAMDiff - Comparison of PDF docs - https://www.palomaprintproducts.com/STREAMdiff/printPerfect.asp

Skitch - Screen capture, crop, resize, sketch (Mac) - http://skitch.com/

TimeSnapper - Take screenshots automatically in background as you test - http://www.timesnapper.com/

VM Ware and MS Visual Studio 2011 Test have record features.

Snagit - Screen capture - http://www.techsmith.com/

BrowserMob - Cloud-based performance testing and monitoring - http://browsermob.com/performance-testing

WebTrends - web monitoring and analytics - http://www.webtrends.com/

Presently.com - allows individuals to post short, frequent updates that are tracked or "followed" by others. Unlike Twitter, Present.ly provides a secure and private way to share updates among members of a company, without them being visible to the outside world.

Sharepoint - www.microsoft.com

Skype - www.skype.com

Fluid - lets you create a Real Mac App (or "Fluid App") out of any website or web application, effectively turning your favorite web apps into OS X desktop apps. http://fluidapp.com/

Fake - Fake is a new browser for Mac OS X that makes web automation simple. Fake allows you to drag discrete browser Actions into a graphical Workflow that can be run again and again without human interaction. The Fake Workflows you create can be saved, reopened, and shared. www.fakeapp.com

TeamViewer - You can remote control your partner's PC as if you were sitting right in front of it. http://www.teamviewer.com/en/index.aspx

Logmein.com - Remote PC access. Free for up to 2 computers.

Joinme.com - Screen sharing

IE Tester - IETester is a free WebBrowser that allows you to have the rendering and javascript engines of IE10 preview, IE9, IE8, IE7 IE 6 and IE5.5 on Windows 7, Vista and XP, as well as the installed IE in the same process. http://www.my-debugbar.com/wiki/IETester/HomePage

Thanks for all your input!

Randy

Minnesota Taxpayers to Foot a $7.25 Million Bill for a Failed Software Project

2011-03-21T10:10:00.002-05:00

For many years, when software projects failed, the vendors almost always had to pay up big time. Well...it looks like the taxpayers in Minnesota are getting stuck with a $7.25 million bill on this one. Why? Basically, the judgment was that the state also contributed to the failed project. Get this..."The payment will bring HealthMatch's total cost to more than $41 million, a department spokeswoman said. The original budget was about $13 million"

You can do a LOT of process improvement and optimization for $41! However, that's all seen as "QA stuff".

I really feel for the people of Minnesota on this one. But, I've seen this happen far too many times. The problem is that software projects are not performed in a vacuum. A vendor goes into an organization to work "with" the people, not "for" the people. So, if the organization is dysfunctional, so will be the project. The reality in this case (as I understand it) is that there were shortcomings on both sides. However, what often happens is that both sides get sucked into a death spiral. Neither can afford to leave, so they keep thrashing until the money runs out and/or someone decides to pull the plug.

Read more at Computerworld.com.

Abbott & Costello Meet Microsoft Outlook

2011-01-19T09:42:00.005-06:00

I found this on Computerworld's Shark Tank and thought about the old "Who's on First?" bit. It's also a great example of trying to understand the users' true needs and expectations of software.

"He (Big Boss) wanted me to help him understand how Microsoft Outlook calendar appointments work with changing time zones," says fish. "I updated the online docs, then walked down the hall for the explanation to the big boss."

But as soon as the big guy hears the explanation -- how Outlook will automatically reflect the time-zone shift when the clock is changed -- he tells fish, "No, that's not what I need. I need to be able to set an appointment up for a certain time slot in the other time zone for next week, but then appear in the same time slot when I look at it with my PC set to my current time zone, this week.

"If I set it for 2 p.m. there, then it needs to show up as 2 p.m. on my calendar when I look at it from here, but for next week.

"And I want everyone else invited to the meeting to see it correctly when viewing both here and in the other time zone. And, oh yeah, it needs to show the correct time slot on my smartphone calendar, and on all the attendees' smartphones, next week when we're traveling."

Fish scratches his head and then attempts to explain that an appointment set in the other time zone will always appear in a different hour time-slot when viewed from a device set in the current time zone, and that it all depends on the time and time zone set on the device.

Big boss says, "No, you're not getting what I need. I need to be able to set an appointment up for a certain time slot in the other time zone for next week, but then appear in the same time slot when I look at it with my PC set to my current time zone. I can't believe that there isn't a way built into the software to do this!"

Reports fish, "I tried to explain it one more time, but in the end the big boss looked back with glazed-over eyes and said, 'Well, since the technology can't help me, I'm just going to print out my calendar and carry it with me to the other time zone.'

"Then he proceeded to print out next week's calendar on paper."

Book Review - "Glitch - The Hidden Impact of Faulty Software"

2011-01-14T16:02:00.006-06:00

I have a great interest in the topic of software defects, their impact and how software failures happen. Books in the past such as "The Day the Phones Stopped", "Bad Software" and "Fatal Defect" have been great sources of documenting what has gone wrong due to software defects. These books serve as case studies for anyone who is involved in creating, testing or buying software products.

So, I was excited to see that a new book has been written on the topic. I'm not crazy about the title of "Glitch" because it tends to convey that a problem is minor. We see this a lot in the popular media as major system failures are described as glitches. I think the problems that Papows describe in this book are more than minor ones. Setting the title aside, I was happy to see a very recent accounting of computer failures in many domains - financial, government, medical, etc. Of course, I'm not happy for the problems, but it's good to see recent information.

This book is written at a level that non-technical people can understand and technical people can read without feeling insulted. That's a nice balance. The information is also sourced well. When it comes to reporting about failures, there are ten sides to every story. But having some objective sources is helpful.

To me, the strongest chapter was Chapter 8, "The Way Forward." In this chapter, Papows (who was the former President and CEO of Lotus Development) describes what organizations and individuals can do to help reduce the risk of computing failures. This is the only place QA and testing are mentioned in the book, which was a little disappointing to me, but I come from a biased perspective. I think people need to understand the project management reality of rushed deadlines, skimpy testing and lax attitudes toward quality to truly understand why software fails. This book gets into that to some degree, but not very deeply.

In fact, much of the book is written from a top-down perspective as opposed to an "in the trenches" perspective. I was hoping for a more detailed analysis of why some of the problems happened.

I can recommend this book to business managers to raise awareness of software risks and to software quality professionals who need good examples of past failures to make the case for better software development and testing processes. However, it may disappoint those looking for root cause analysis of software defects.

Six Seats Left for Test Automation and Test Team Leadership Training in Salt Lake City - Jan 25 - 27, 2011

2011-01-14T10:44:00.006-06:00

I just wanted to get the word out to anyone that is in the Salt Lake City area, or who may be able to travel there, that I have six seats open for two great workshops:

Practical Software Test Automation (Tues & Wed, Jan 25 & 26, 2011) - Learn hands-on techniques and approaches for building an efficient test automation architecture. We get into scripting and test tool integration. Bring your notebook computer!

Becoming an Influential Test Team Leader (Thursday, Jan 27, 2011) - This is an interactive workshop that will help you bring out the best in your test team. If you are a test team leader, or aspire to be in that role, this workshop will help you influence your team and those you interact with.

These two workshops are at special pricing for the Salt Lake City QA Focus Group, but I am extending that pricing to everyone! $750 for the Test Automation 2-day workshop (normally $1,400) and $375 for the 1-day Test Team Leadership workshop (normally $790).

The workshops will be held at The Pavilion Inn - http://www.slpavilioninn.com/

To see details and to register, just visit https://www.mysoftwaretesting.com.

Randy Rice

Tiger Woods Needs a Golf Teacher?

2011-01-13T11:39:00.006-06:00

On my last trip to Rome, while flipping the two English channels in my hotel room, I happened to catch a story on CNN about Tiger Woods and his need for a new golf teacher/coach. This fascinated me.

Now, I’m not a golfer so that explains some of my curiosity about why one the world’s greatest golfers would need a teacher. In fact, Tiger is arguably the best golfer in the world today, although in tough times right now. I don’t even watch much golf on television, so I’m really a neophyte.

Tiger’s most recent coach was Hank Haney, who coached him from 2004 to May of 2010. Haney resigned because he felt he had taken Woods as far as he could.

http://www.cnbc.com/id/37192697/Instructor_Haney_Explains_Parting_Ways_With_Tiger_Woods

But still, I kept thinking, if the world’s best (or at least in the top five) needs coaching, what does that say about the whole concept of personal improvement. How does that apply to what testers do?

After thinking about this, I started to see some reasons why even the best in what they do need a coach, teacher, or mentor.

1. Objectivity

We all need someone who will tell us the truth, no matter how ugly it may be. And, they need to be able to do that from an external and unbiased perspective. True, a golfer can have a video camera and other ways to capture their golf swing, but it takes a lot of experience and insight to see the one little flaw that can increase a score.

Tiger probably has lots of people he can ask for opinions, and most of them will tell Tiger what they think he wants to hear. I find managers like this all the time. They can’t get the real story because so many people are either afraid to give bad news, or they are people-pleasers.

2. Broad context

Hank Haney has coached thousands of golfers, and also many golf pros. He has seen more bad swings than just about anyone. He has also seen the best. He knows what makes a difference because he has seen the techniques work in practice. Haney understands the mental aspect of golf as well.

3. Accountability

If you want to improve at anything, you have to practice and take positive action. Things that are easy to do are also easy not to do. A coach or mentor can ask you if you are on track doing the things needed to improve. They can’t make you do things, but they can remind you of your goals.

4. Encouragement

Everyone needs encouragement. Encouragement is interesting because it comes from others. You can have affirmations and self-talk, but a little encouragement from someone does so much more than what you can muster up yourself.

The Results

The results of these things are motivation and improvement. You’ll notice that I didn’t include either of these in the list the coach brings. These must come from the one being coached. These are brought out by a good teacher or coach.

Likewise, desire and the choice to change must come from those wishing to improve.

Keep in mind that I’m not necessarily talking about large steps of improvement. Whether you’re at the top of the game or just starting out, one tiny change can make a world of difference.

What About Testing?

Since many of you who read this blog are software testers or involved in software quality in some way, here are some tie-ins to what testers do.

Objectivity – This is the value of independent testers. The reason testers are needed is that they have a fresh view of something. They can see defects that someone with a lot of familiarity may miss. However, some testers are fearful of giving the bad news. That’s why a good testing process with measurements and metrics is so important. The process can deliver the news, whether good or bad.

Broad context – Testers do best, I believe, when they have worked in different companies and on different types of projects. You may not have much control over this, but you can still explore and expose yourself to different things. You can read, attend conferences, and get training to broaden your horizon.

Accountability – Testers can ask probing questions like, “Has this code been reviewed?” or “Has the business user seen this requirements document?” If there are development and testing processes in place, testers can tell if those processes are being followed.

Encouragement – Testers can give more than the bad news of defects. They can also give credit for great work.

Doing It

I’ve been sitting on this article for two months. I’ve wanted to write it, but for some reason just could not get off the dime. Perhaps it was my own lack of motivation.

Then, this week I happened to be flipping through my “300 channels of nothing” on TV and came across “The Haney Project” on the Golf Channel. I had no idea what this show was about, but soon saw that it tied everything together for me. In this episode, Hank Haney was teaching Rush Limbaugh how to improve his golf game.

A couple of things that stood out to me were that 1) he told Rush to only think about one thing during his swing, not twelve, and 2) Haney was very encouraging. At the end of the program, Limbaugh remarked about how impressed he was about working with Haney. Unlike other teachers who had been negative and overbearing, Limbaugh found Haney to be supportive and positive.

When I watch TV, I like to watch programs that show how people can improve. I found this program very interesting just watching Hank Haney’s coaching style in action.

Oh, and after watching Rush play golf, I'm thinking, "Hey, I can do at least that good!" So who knows? I may actually dust off my garage sale clubs and hit the links.

The big take-away for me in all this is that we all need people around us to help us improve, to encourage us and to bring accountability. I encourage you to find someone who can be that coach to you or your team. If you need help in that effort, let me know.

I would like to hear your comments on this topic!

3 Ways to Stretch Your 2010 Training Dollars

2010-12-16T07:34:00.002-06:00

As we wind down 2010 and look forward to 2011, you may be left with some training dollars in your budget. I know I try to make the best use of my money right at the end of the year. Here are 3 ways you can do that:

1. Buy e-learning courses. I have placed several of my e-learning courses on sale until December 31, 2010. The great thing is that you have 12 months to complete the course!

2. If you live in the Utah area, or even in that region of the country, you may be interested in two workshops I will be conducting there in January. These are on the topics of test automation and test team leadership - and they are deeply discounted.

For details on either of these, visit https://www.mysoftwaretesting.com.

3. Study groups are forming for ISTQB Advanced certification. These are 6 to 7 weeks in length to support self-study. They are held online and have live web meetings with me as facilitator. You can pay in 2010 and attend in 2011. To get more details on this, visit http://riceconsulting.com/home/index.php/Home-Page/istqb-advanced-study-groups-forming.html.

I am working on a jam-packed newsletter for January, so stay tuned for that.

Finally, I wish you a Merry Christmas, Happy Hanukkah, and a prosperous and healthy New Year. Thanks for your support this past year!

Warm holiday regards,

Randy

New York City Voting Machine Problems - Implementation? We Don't Need no Stinking Implementation Plan!

2010-09-14T23:10:00.002-05:00

I've spoken out before on voting machines and the related quality issues which cause me concern. So, this article caught my eye. It seems that today in New York there were major problems (not glitches) in the rollout of new voting machines. One might think perhaps the cause was software or hardware related. However, upon reading the account, it seems that the problem was largely in the implementation.

$160 million spent and this was the result.

I like the "teachable moment" and this is one. It doesn't really matter how good the software if you don't have the hardware plugged in!

http://newyork.cbslocal.com/2010/09/14/new-yorkers-head-to-the-polls-on-primary-day/

Do You Test SaaS, Or Does SaaS Test You?

2010-09-01T22:46:00.009-05:00

Imagine you are sitting at your desk one morning and your phone rings. It's not your boss - it's your boss's boss, the CIO. He's upset because the online sales database is down and the entire sales staff is paralyzed. "The sales software is broken!" he exclaims. Then he asks, "Didn't you test this?" You take a deep breath and say, "No, that software is a service we subscribe to. We have no way to know when it changes." The CIO isn't happy, but now sees the reality of Software as a Service (SaaS).

I believe that SaaS is not a trend, but a major force that will shape the future of IT and, therefore, software testing. In fact, SaaS could dramatically change the way we think about and perform software testing from here on.

This article has a definite angle toward the risks of SaaS. I fully embrace the benefits and appreciate them. I use SaaS, as you will see later in this article. My point here is to point out the risks. Only you can weigh them against your own benefits.

What's Different?

1. You have little or no control over the software, the releases, and any remedies for problems.

In some cases, you may get the choice of when to accept an upgrade. However, even when you are on a specific version of the application, the vendor may choose to make a small change without advance notice.

In other cases, you may come in to work and notice that things on the application look a little different. Or, you get the dreaded phone call that something is broken.

But here's the real rub: Even if you isolate the problem and report it, you are still at the vendor's mercy to fix the problem.

At least with Commercial Off-the-shelf (COTS) software, you get the choice when to deploy it. So at least you get the chance to test and evaluate it beforehand in a test environment.

2. You have almost zero knowledge of structure, and often no knowledge of new and changed functionality.

Forget white-box testing, unless it is for customized interfaces (for example, APIs). Now, let's say you have noticed some functional changes. What really changed? What stayed the same? What are the rules? When are they applied? Many times, none of this is published to the customers.

Not only is SaaS a black-box, it is more like many black boxes, all in a cloud. Your SaaS application is most likely comprised of many services, each with their own logic. In fact, some of the services may be supplied by a vendor you are not even aware of.

3. There is no lifecycle process for testing.

In the past, we could test at various levels - unit, integration, system, and UAT. That's all gone with SaaS. It's all UAT. Sure, you can test low-level functionality and integration, but it's from the customer, not the developer perspective. In the lifecycle view of testing, your tests can build on each other. In the UAT or customer view, testing is a "big bang" event. So, forget "test early and often." That changes to just "Test often."

4. Testing is post-deployment.
There may be some exceptions in the case of beta testing, perhaps. But for most people, you get to see the software only after it is deployed. That's too late to prevent problems. All you can do is race to find them quickly. In other words, you are in reaction mode instead of prevention mode. Then, even if you find the problems before your customers do, they may not be fixed for some time.

5. Test automation is fragile (and futile).

You get no return on investment because ROI is achieved when tests are repeated. When a new version is released, there's a good chance your automated testware will not work. This means you are at risk of reworking your test automation at every release. That said, some people may find test automation of basic functions a way to monitor when changes have been made.

6. Test planning is also futile.

This is because there is little detailed knowledge about the application in advance of using it. There is no specification basis for testing unless you write them. Use cases might be effective for describing work processes supported by the application.

Is There Any Hope?

Yes, and it's called validation. Not validation in the sense of "all forms of testing", but validation in the sense of making sure the software supports your needs.

In Point #6 I mentioned use cases. This is a form of test planning you can perform. You can create test scenarios based on work flows you perform in your organization. However, you must not think in terms of software behavior. Instead, you must describe work processes that can be tested no matter which software is being used.

You can create use cases to describe work processes, not software processes. These can be used as a basis for testing. The flows are perfect for mapping test scenarios.

You can assess the relative risk of work flows to prioritize your testing. You can even use pairwise testing to reduce the numbers of combinations of test scenarios and test conditions.

However, only in certain situations will this help you avoid problems: 1) You are a beta tester or 2) you get to choose when to apply an upgrade.

There Will be Pain
You arrive at work on Monday morning and discover one of your SaaS applications has changed. You and your team scramble to test the high-risk scenarios (manually) and discover that many things are broken. You call the vendor only to learn that your wait time on hold is 30 minutes. You file issue reports and just get the auto-responder messages. Days go by. You keep trying, your manager tries calling, but hey, you're just one customer out of thousands. In many cases, there is no "Plan B."

This is the extreme end of the risk scale. Not all releases fail, and when they do fail, not all fail to this degree. However, the risk is real.

My Story

Just last week I experienced a problem that would have been very painful had it happened at a time I was teaching an online class. My online training services provider upgraded to a new version of the web presentation platform, but gave no notice. All they indicated was that the site would be down for maintenance on Friday evening. Turns out, they introduced a totally new and upgraded platform with several improvements. However, several key things failed: 1) I could record a session, but never get the file (I wasted an hour recording a session to find this out!), 2) I could not upload a file for presentation, 3) I could not install the new screen sharing software on Windows XP. There's not much you can do with web meeting software behaving like this.

I called tech support on Saturday. Guess what? Nobody home, even after a major release. I filed 3 problem tickets which as of this date have still not been closed out, even though the problems were fixed about 2 days later. (I did get one response.) I'm happy things are working now, but troubled about the way the release and post-release were handled. If I had been scheduled to teach an online class on Monday, I would have been stressed out for sure.

My case is fairly low-impact, but it did show me the risks of SaaS. I encourage you to keep these risks on your radar because like it or not, SaaS is in your future.

I would like to hear your experiences and ideas for testing SaaS, so leave a comment!

Software Test Automation Workshop in Oklahoma City - Sept 14 and 15, 2010

2010-08-30T11:27:00.003-05:00

I'm excited to announce we're holding the Practical Software Test Automation workshop in Oklahoma City on Tuesday, Sept. 14 and Wednesday, September 15, 2010.

The workshop will be held at the Hampton Inn, I-40 East (Tinker AFB) location at:

1833 Center Drive
Midwest City, Oklahoma, USA, 73110
1-405-732-5500

You can see the details (outline, pricing, etc.) and register at:

http://www.mysoftwaretesting.com/product_p/okcauto.htm

GSA discounts are available for this workshop. Just contact me for details.

I hope to see you there!

U.S. To Train 3,000 Offshore IT Workers

2010-08-13T19:59:00.002-05:00

I ran across this article in Information Week and I'm thinking, "What's wrong with this picture?" We have unemployed IT workers here in the USA and it seems that we are training people who will eventually compete with them for the few jobs that exist. By the way, there is an update that we will be doing the same thing in Armenia. Of course, this will all be done with borrowed money.

Some key quotes:

"Despite President Obama's pledge to retain more hi-tech jobs in the U.S., a federal agency run by a hand-picked Obama appointee has launched a $36 million program to train workers, including 3,000 specialists in IT and related functions, in South Asia. Following their training, the tech workers will be placed with outsourcing vendors in the region that provide offshore IT and business services to American companies looking to take advantage of the Asian subcontinent's low labor costs.

Under director Rajiv Shah, the United States Agency for International Development will partner with private outsourcers in Sri Lanka to teach workers there advanced IT skills like Enterprise Java (Java EE) programming, as well as skills in business process outsourcing and call center support. USAID will also help the trainees brush up on their English language proficiency."

I don't know. I've stopped asking "why?" Back when outsourcing became all the rage we were told that in a new service economy people would rise to higher level positions. Instead, these US IT workers are trying to find the best position they can in an economy where there is a race to the bottom in terms of pay.

I know there are job training programs here in the USA, but even trying to find the front door to those programs is difficult. How often do you see those advertised? And now even those programs are being cut due to lack of funds. I guess we need to pay for other training programs for people in...Sri Lanka, Armenia and who knows where else.

I would love to hear your thoughts on this.

The SaaS Performance Risk - An Example from Twitter.

2010-08-01T18:23:00.005-05:00

Recently I was interviewed about application performance risks. The final question was, "What are some performance risks you see in new technologies?" My answer was the cloud computing and Software as a Service (SaaS) risk.

Actually, there are many risks and concerns in the SaaS model. This is not to say the model does not have value, even great value. It's just that there are things you need to know before adopting SaaS.

The basic idea of SaaS is that you use software applications that are developed, hosted and maintained by a vendor. You trust that the processing will be correct, fast enough for you, secure, easy to use, available, reliable, etc. However, you have no control over those things. When the vendor has a problem, you have a problem. That can be a rude awakening to people.

To illustrate the performance risk, I saw an article recently about the overload at Twitter:

http://www.computerworld.com/s/article/9179446/Twitter_s_tech_problems_take_a_toll_on_developers?source=CTWNLE_nlt_app_2010-07-22

Twitter has a huge challenge with extreme traffic spikes. Now, on the upside, this is not a national security type of application, and most of us Twitter users have gotten used to the "Fail Whale" picture and just say, "Oh, well...I'll tweet that later."

Then, other things happen:

But there is a business impact to the companies that sell services interfaced to Twitter:

"'Twitter API issues in the previous weeks have been terrible for us," said Loic Le Meur, founder and CEO of Seesmic, which makes Twitter client applications for various desktop and mobile platforms. 'Users always blame Seesmic first since it's their primary interface to Twitter. It's extremely frustrating because there is nothing else we can do than warning users Twitter has problems. It is very damaging for us since users start to look for alternatives, which fortunately have the same problems, but damage to the brand is done,' he said via e-mail.'"

To be fair to Twitter, they've done a lot to improve reliability in recent years.

In a related story, from last week, I read on Bloomberg.com:

"Interest in the SaaS (software as a service) delivery model is growing to the point that by 2012, almost 85 percent of new vendors will be focused on SaaS services, according to new research from analyst firm IDC. Also by 2012, some two-thirds of new offerings from established vendors will be sold as SaaS, IDC said."

http://www.businessweek.com/idg/2010-07-26/idc-saas-momentum-skyrocketing.html

So, what does all this mean to you?

If you are considering SaaS as a major application delivery method, then be aware of the risks. In reality, there isn't a lot you can do during a SaaS failure. For example, you can't just call up the folks at Twitter and tell them to "get it fixed" (like perhaps you can speak with the developers in your own company).

Some of this goes back to service-level agreements (SLA), but no vendor I know of will guarantee 100% uptime. Even if they did, there are other risks, such as correctness, not typically covered in an SLA.

I wish I had a handy list of things to mitigate the risks, but every case is different.

I would be interested in hearing your experiences with SaaS and how you deal with the risks, so please leave a comment.

How to Create a Fear-Based Culture

2010-07-30T10:23:00.002-05:00

Hi folks,

Here is a great blog post from The Next Level Blog by Scott Elbin. Those who know me, know I'm big on the human factors in software development and testing. A big part of that is culture. One of Dr. Demming's 14 points is to "Drive out fear." What does that mean? Well, here's seven ways NOT to do it. See if you recognize any of them. Have a great day!

http://scotteblin.typepad.com/blog/2010/07/seven-simple-rules-to-create-a-fear-based-culture.html

Practical Software Test Automation Course Now Available in eLearning!

2010-07-21T09:46:00.002-05:00

I am excited to announce the release of my newest course, Practical Software Test Automation, in e-learning format.

This course focuses on the basics of software test automation and expands on those topics to learn some of the deeper issues of test automation. This course is not specific to any particular tool set but does include hands-on exercises using free and inexpensive test tools. The tool used for test automation exercises is Macro Scheduler.

The main objective of this course is to help you understand the landscape of software test automation and how to make test automation a reality in your organization. You will learn the top challenges of test automation and which approaches are the best ones for your situation, how to establish your own test automation organization, and how to design software with test automation in mind. You will also learn many of the lessons of test automation by performing exercises using sample test automation tools on sample applications.

I hope to see you there!

Click here to see the course outline.

Click here to take a demo. (You can login as a guest)

Click here to register for the course.

Wahington Post Series on Top Secret Sites is Shameful

2010-07-19T16:22:00.004-05:00

Hi Folks,

Normally I don't get political on this blog, and actually, I don't really think this post is political. But I do think the topic is important.

Today, the Washington Post unveiled their series on the Top Secret work done by the Federal Government.

http://projects.washingtonpost.com/top-secret-america/companies/1/

Here's my problem: There will be a lot of innocent people placed at risk simply because of who they work for. Imagine this scenario: Joe Smith, an employee (fictitious) of a Top Secret government contractor takes a trip to a quasi-friendly (or even unfriendly) country to perform work for another client. Joe winds up in a situation for some reason that involves police authorities in said country. They ask him where he works. He answers truthfully. They run that information through their systems and bingo, get a hit. (They know all the companies now because of this article) Depending on the person running the query, Joe might be flagged as an agent. He certainly has knowledge of Top Secret information. Right?

Well...maybe, maybe not. However, try convincing an authority in a foreign country of that.

In fact, people don't even have to travel abroad. Now, our enemies know exactly where the offices of these companies are. They now have all types of targets for espionage and for recruiting spies.

Some will say the articles have a noble purpose to expose government waste. Is that something we don't already know?

Some may also say, like in a Tom Clancy novel, if the Washington Post can find this information, our enemies already know it. Yes, but they've made it really easy to find - all in one place, hyperlinked, with maps and all.

It will be interesting to see if any of the same people who were outraged over the "outing" of Valerie Plame will be outraged over this. I doubt it.

I think the Washington Post has abused the liberty of freedom of the press by publishing this series, but the damage has already been done.

New Testing Skills Needed for Companies that are Rebuilding Test Teams

2010-07-14T13:51:00.004-05:00

It's still a tough economy, but some companies are starting to hire in IT again, even hiring software testers. That's an encouraging sign!

I've seen many companies struggle in the team rebuilding process, mainly due to simply getting people on the same basis of knowledge. The thing to consider is, when you start to being new people into your teams, how will they build the skills they need to be effective on your team?

Here is what often happens. After a company starts to feel the pain of tasks left undone (for testing, that means defects going straight to customers and customers leaving), they start to rebuild the teams. So, you search for the best and brightest people, and hire who you can afford. These people have a mixed bag of skills and talents, all learned from various sources, some practices effective, some ineffective. And then, some people embellish their skills on the resume, so when they are hired they don't perform as expected. But, you still stick with them at least for awhile.

Then, you have the faithful and the tough - the people who have been with the company for a long time and have never been formally trained.

If this situation is left "as is" you basically have a stew that is not very tasty.

What's the Solution?

1. Perform a Skills Assessment. This will tell you exactly where each person stands in their overall skill set.

2. Perform Training. This lays in place a foundation of common skills and terminology.

3. Perform Continued Mentoring. This reinforces the skills and may be needed for topics that the training can't reach, such as organization procedures, etc.

The sooner you can do this, the better, as long as you have most of the team in place. For the ones that join after the training, it's good to have e-learning available.

If you need help in getting your team's skills in place, contact me. I have over 60 courses in software testing and related topics. Tester certification is also a good approach for many teams.

I can create a custom training plan for your organization that will give you a head-start and boost your effectiveness as a team.

Is Software QA Dead?

2010-06-29T09:51:00.012-05:00

Is traditional Software Quality Assurance (SQA) dead? Or...does it just suffer from poor perception and bad practice? I hope it is the latter.

I started thinking about this question after hearing a presentation recently that highlighted the problems with traditional SQA. The more I think about it, the more I believe we still need true SQA (not just testing). If you don't know the difference, please read on.

QA and QC

First, we must understand that true QA is not testing and it is not a verb. So, to say “then we QA it.” is like saying “then we configuration management it.”

SQA focuses on how a process is performed and is the management of quality. SQA can encompass metrics, process definition and improvement, testing, lifecycle definition, and so forth. The SQA function may perform some tests and reviews (quality control or “QC”) but unless there is quality management, the effort can easily become haphazard and uncontrolled.

Software testing is QC. So are reviews and inspections. The key difference is that the focus is on the product to find any defects. SQA and QC must work hand-in-hand to be effective.

SQA is process assurance, that is, assurance that the process is being performed as designed. QC is product assurance, or assurance that the product meets specifications. So, both activities are needed.

New vs. Old

I've been developing and testing information systems for over 33 years now. There's always something new that people think will change the way everyone develops software. Think about all the approaches and methods that have come down the pike – from waterfall to agile. Why do people still use the waterfall? Why are some people ardent evangelists of agile?

I think much is explained by comfort zones and culture. People tend to use approaches they are comfortable with. People don't like to change.

However, people do like to be fashionable. New approaches are fashionable which gives them early adopters who become enthusiastic supporters. After all, when was the last time you saw someone excited about the waterfall approach?

Process-orientation vs. Product-orientation

Back in the 70's and 80's, one of the big issues was that much focus was on the software, not on how it was built. So, the famous quote was “If builders built buildings the same way programmers write code, the first woodpecker that came along would destroy civilization.”

In 1985, I was performing eXtreme Programming, it just wasn't called that. I worked in tandem with another programmer, developed my tests first, then coded to them, and worked from user stories. The problem was inconsistency. We were the only two working this way. There was no one in management that wanted to spread the technique. In those days, like today, waterfall was king. Why? Because the waterfall model can be explained in about 10 minutes or less.

Software project consultants looked at this state of affairs and concluded that the process has a great deal to do with creating software and that software development should be an engineering effort, not an art or a craft. Hence, the title, “software engineer” and following years of creating process models, such as the Capability Maturity Model (CMM). The CMM was very process-centric. In fact, the original version didn't have a key process area for software testing. The idea was that if the process was performed correctly, testing would not be needed.

The flaw in the process-only idea is that people are not perfect. Therefore, there will be mistakes at every step in building software, all the way from concept through system retirement. The only way to find the defects caused by these mistakes is to detect them by an effort designed to find them. A filtering approach where defects are screened out by inspections along with early and ongoing testing is a very effective approach.

The good part of the process focus is that it does typically deliver a better product with fewer defects injected throughout the project life cycle. This has been proven by organizations with high levels of process maturity who also measure defect detection percentage. (See chart.)

Indeed, processes provide a valuable framework to organize and perform all other project tasks. In short, processes can be improved, they can be shared and they can be trained.

SQA is a key mechanism by which improvements are made. In organizations it is common to find pockets of both good and bad practice. Improvement is rare, which caused people to see the need for processes to begin with. A few years back, Lewis Gray wrote a great article for Crosstalk Journal entitled, "No Hypoxic Heroes, Please", in which he makes a compelling case for software processes using the example of why mountain climbers follow processes and standards – which is to keep from making bad decisions when their minds start to become oxygen-deprived and the ability to reason is impaired. We see the same thing on software projects, especially as the deadline looms closer and closer.

Bad SQA Practice

It is possible to take any effective tool or approach and apply it in an ineffective way. Some organizations have built the SQA function into a bureaucracy which slows projects down and adds little value to the organization. In fact, defects may even increase due to people spending so much time performing paperwork. This was never the objective of SQA, “old school” or in any context.

Other organizations have turned SQA into a police force which investigates, audits and regulates software projects. The intent is noble, but the rest of the project lives in fear of the SQA team and what they can do to them.

In some organizations, SQA is a gatekeeper. To get software into production use, it must get QA approval. Once again, this is a negative view of what SQA is intended to perform. In reality, implementation should be a team-based decision based on risk.

The average lifespan for a SQA group is about two years. That's because after about two years, senior management asks, “What do these people do?” Unless the SQA team can show tangible and positive value, the decision is likely to be made to try something else to improve software quality. All too often, the SQA team's work is seen as intangible and paper-shuffling.

Back in 2000 I gave a keynote presentation at QAI's testing conference in which I made a major point that if QA and test organizations must be in alignment with business objectives and project objectives, or else they will be marginalized and most likely eliminated. In other words, the QA and test teams must find the right balance of finding and preventing problems, as compared to not stopping progress.

I believe the only people who can direct that balance are the business stakeholders. These are the people who live with the level of software quality, or at least know what they want the business's customers to experience.

What Can Really Be “Assured”?

Not much, in my experience. We can't guarantee perfection since we can't test everything. Likewise, we can't assure a process has been perfectly performed. Even if a process is perfectly performed, the process itself can be flawed.

This leads me to my final point which concerns processes as performed professionally versus those performed in a factory setting. In a factory, you want everyone doing the same thing in the same way. You do not want any variation at all.

However, the more professional the effort and the person, the more you can rely on their expertise to do the job expertly. There are many examples of this, such as great chefs knowing how to prepare great meals without a recipe, or great doctors not reading the process book as they perform surgery. However, what is not seen, are the rules, standards and protocols each of these professionals adhere to. For example, the chef cooks food to pre-established proper temperatures to prevent food poisoning. The surgeon has a team of people following an exact checklist to make sure all preparation is correct before the surgery and all surgical instruments are accounted for after the surgery.

In software development we rely on many people to get it right. All the way from concept to delivery, developers, business analysts, architects, testers, DBAs, trainers, management, customers and others must work together in professional ways. There may or may not be a formal software life cycle followed.

When things work well, no one seems to think much about software QA or testing. It's like the air conditioning - no one gives it a thought until it breaks down. However, when the product being delivered starts to slip and the customers start to leave, then management starts thinking “Maybe we need some structure in place to make sure we do the right things in the right ways.”

Smart QA

QA done well and done smartly, can be a very helpful activity. The problem is when people don't match the QA approach to the business and project context.

Instead of doing some basic root cause analysis and finding the true source of the problems, which can be addressed and prevented, some companies embark on major pushes to install a new QA program, SDLC or both.

Instead, how about some simple steps such as basic processes, checklists, guidelines and re-designed tests? Then, after seeing how those work, we can make further adjustments.

Conclusion

I don't think true QA (not testing) is dead, but I do think it suffers from bad practice and poor perception. I also believe there is a pendulum effect which swings from one extreme to another. In the case of software, the pendulum swings from “no process” to “all process”. Right now, we are at the “no process” end of the swing, with movement toward the center.

QA is a function that each organization must decide how best to adopt. Some will reject QA entirely, some will adopt it smartly and some will adopt it inconsistently or with bureaucratic approaches.

I hope you apply QA smartly. If you need help in doing that, call me!

Software Test Automation Workshop in the Dallas/Ft. Worth Area - Aug 12 and 13, 2010

2010-06-09T13:14:00.005-05:00

I'm excited to announce I'm coming to the Dallas/Ft. Worth area in August to present my newest course - Practical Software Test Automation workshop!

Here's the details and how to register:

http://riceconsulting.com/home/index.php/Public-Seminars/randy-rice-to-present-test-automation-workshop-in-dallas-ft-worth-area-aug-12-and-13-2010.html

http://www.mysoftwaretesting.com/product_p/dfwauto.htm

I hope to see you there!

Want a class like this in your city or at your company? Let me know!

Thanks,

Randy

20 Year Anniversary of Rice Consulting Services Today

2010-06-04T10:12:00.005-05:00

Today is the 20th anniversary of Rice Consulting Services! It has been an amazing ride. I'm thankful to God for keeping His business going for this long.

It was 20 years ago today...Sgt. Peppers taught his band to play....I mean we moved from Kansas City back to our home of Oklahoma City to start Rice Consulting Services. My first project was the Oklahoma City Water Trust, which was a major failure. I call it "the day I tested myself out of a job" because I asked whether or not the system had been stress tested. Turns out, it hadn't and it could not stand any load. The system never saw the light of day. You can read all about it here. It's worth your time.

Then, I got to know people like Bill Perry who gave me a great national platform with the opportunity to speak at the QAI testing conferences. We wrote together, Surviving the Top Ten Challenges of Software Testing, which opened many other doors.

Thanks to all of my friends and clients, who have supported us in the good and bad times.

Thanks to Janet, my wife and President of RCS, who is 100% behind this business and understands what small business is like. We eat what we shoot. We take risks that would make some people sleepless. There are no guarantees or bailouts for small businesses. We are small enough to fail, but by the grace of God, we keep going and serving our clients.

I feel like one the farmers who won the lottery a few years back. When asked what they were going to do with the money, one of them said, "We're gonna keep farmin' til the money runs out."

So, I raise my cup of coffee to you, my friends that read this blog. Thanks for your support. I look forward to 20 more years. We will probably still be talking about how to write test plans. And, that is...well, depressing in a way, but shows the never-ending job of skill building in testing.

Thanks, everyone!

Agile and Exploratory Testing in Kansas City - July 13 and 14, 2010

2010-06-03T14:31:00.002-05:00

I'm excited to announce I'm coming back to the Kansas City area in July to present my Agile and Exploratory Testing workshop!

Here's the details and how to register:

http://riceconsulting.com/home/index.php/Public-Seminars/randy-rice-to-present-agile-exploratory-workshop-in-kansas-city-area-july-13-and-14-2010.html

About the course: http://riceconsulting.com/home/index.php/Table/Agile-Testing/

There's a special discount for KCQAA members. I hope to see you there!

Want a class like this in your city? Let me know!

Thanks,

Randy

The Surprising Truth About What Motivates Us

2010-05-28T10:21:00.003-05:00

You have to see this video. It's 10 minutes long and I promise you will find it interesting, entertaining, and a valuable use of your time.

http://michaelhyatt.com/2010/05/the-surprising-truth-about-what-motivates-us.html

Have a great and safe weekend...and remember those who have served and died for our country!

Randy

Webinar Links - Ellusive Tester to Developer Ratio

2010-05-27T13:33:00.005-05:00

Hi everyone,

Thanks for tuning in to the webcast today.

Here are the links for the recorded session and chat transcript, along with other things I mentioned.

Recorded webinar
Chat transcript
Slides in PDF format

CTE-XL tool
Article - Ellusive Tester to Developer Ratio (The one I wrote back in 2000)

Thanks!

Randy

May 2010 Newsletter Posted

2010-05-23T22:26:00.002-05:00

Hi Folks,

Better late than never! The May issue of the Software Quality Advisor Newsletter is out:

http://riceconsulting.com/home/index.php/Newsletter-Past-Issues/may-2010-test-estimation-based-on-testware.html

You can get your copy each month to your e-mail account by signing up at:

http://riceconsulting.com/home/index.php/Newsletter/the-software-quality-advisor-newsletter-sign-up.html

Webinar - Thursday, May 27 - The Elusive Tester to Developer Ratio

2010-05-21T12:05:00.003-05:00

Join me on Thursday, May 27 as I present a webinar on The Elusive Tester to Developer Ratio. The session is free and starts at 12:00 noon Central Daylight Time. There are 50 slots available on a first come basis, so I suggest being there 10 minutes early. To register, just click here.

To join the meeting, visit https://my.dimdim.com/ricecon on May 27 at 11:50 a.m., CDT. The webinar starts at 12:00 noon.

Book Review - Reflections on Management by Watts S. Humphrey with William R. Thomas

2010-05-19T14:32:00.009-05:00

The Capability Maturity Model (CMM) and Capability Maturity Model Integrated (CMMI) have been major forces in software development for at least 20 years. Along with those, the Personal Software Process (PSP) and the Team Software Process (TSP) have also been applied to help make software projects more predictable and manageable.

This book is a collection of essays and articles written by Watts Humphrey, the man who was the influence and drive behind these models and processes. I found this book to be an interesting journey through the thinking of Humphrey as he clearly and rationally outlines the "why" behind the "what." Then, he describes "how" to do the work of managing intellectual and creative people which have to work together to deliver a technical product - on time, within budget, with the right features and with quality.

There are many gems in this very readable book (a great airplane book), such as:

Defects are Not Bugs
The Hardest Time to Make a Plan is When You Need it Most
Everyone Loses With Incompetent Planning
Every New Idea Starts as a Minority of One
Projects Get into Trouble at the Very Beginning

This book is divided into four parts:

Managing Your Projects
Managing Your Teams
Managing Your Boss
Managing Yourself

If you are a software project manager, test manager, or test team leader who has to fight the battles involved in getting a project completed within time, budget, scope and quality targets, you will find this book of immense value. Or, you might buy it as a gift for your manager who just doesn't get what's so hard about software development.

Although this book is a collection of essays, it flows very well and reads like it was written as one book. By the way, I felt the Epilogue was excellent - don't skip it.

If there are any doubts about the credibility factor of this book, the advance praise at the front of the book spans four pages and reads like a "who's who" of software development: Steve McConnell, Ed Yourdon, Ron Jeffries, Walker Royce, Capers Jones, Victor Basili, Lawrence Putnam and Bill Curtis, to name a few.

Whether you are fully immersed in the agile project world, or following the CMMI, or just trying to figure out the best way to plan, conduct and manage software projects, this is a book worth reading and taking to heart. In the advance praise, Ron Jeffries (www.XProgramming.com) writes, "I've followed Watts Humphrey's work for as long as I can remember. I recall, in my youth, thinking he was asking too much. Now that I'm suddenly about his age, I realize how many things he has gotten right. This collection from his most important writings should bring these ideas to the attention of a new audience: I urge them to listen better than I did."

Amen, Ron, amen.

Reviewed by Randy Rice

Disclosure of Material Connection: I received one or more of the products or services mentioned above for free in the hope that I would mention it on my blog. Regardless, I only recommend products or services I use personally and believe will be good for my readers. Some of the links in the post above are “affiliate links.” This means if you click on the link and purchase the item, I will receive an affiliate commission. Regardless, I only recommend products or services I use personally and believe will add value to my readers. I am disclosing this in accordance with the Federal Trade Commission’s 16 CFR, Part 255: “Guides Concerning the Use of Endorsements and Testimonials in Advertising.”

Test Estimation Based on Testware

2010-05-19T13:02:00.013-05:00

I had a conversation recently with one of my clients about test estimation based on manual test cases. As I have written previously, I feel that in many ways test estimation is inherently flawed for a variety of reasons.

However, there is a technique I have used over the years that plays on risk-based approaches. This technique can be applied to testware, such as test cases. Just remember this is not a scientific model, just an estimation technique.

What is Testware?

Testware is anything used in software testing. It can include test cases, test scripts, test data and other items.

The Problems with Test Cases

Test cases are tricky to use for estimation because:

They can represent a wide variety of strength, complexity and risk
They may be inconsistently defined across an organization
Unless you are good at measurement, you don’t know how much time or effort to estimate for a certain type of test case.
You can’t make an early estimate because you lack essential knowledge – the number of test cases, the details of the test cases and the functionality the test cases will be testing.

Dealing with Variations

“If you’ve seen one test case, you’ve seen them all.” Wrong. My experience is that test cases vary widely. However, there may be similarity between some cases, such as when test cases are logically toggled and combined.

A technique I have used to deal with test case variation is to score each test case based on complexity and risk, which are two driving factors for effort and priority.

The complexity rating is for the test case, not the item being tested. While the item’s complexity is important in assessing risk, we want to focus on the relative effort of performing the test case. You can assign a number between 1 and 10 for the complexity of a test case. It may be helpful to create criteria for this purpose. Here is an example, You can modify it for your own purposes.

1 – Very simple

2 – Simple

3 – Simple with multiple conditions (3 or less)

4 – Moderate with simple set-up

5 – Moderate with moderate set-up

6 – Moderate with moderate set-up and 3 or more conditions

7 – Moderate with complex set-up or evaluation, 3 or more conditions

8 – Complex with simple set-up, 3 or more conditions

9 – Complex with moderate set-up, 5 or more conditions

10 – Complex with complex set-up or evaluation, 7 or more conditions

This assessment doesn’t consider how the test case is described or documented, which can have an impact on how easy or hard a test case is to perform.

Assessing Risk

Risk assessment is both art and science. For estimation, you can be subjective. In fact, my experience is that risk assessment is subjective at some point or other.

This scale is based on the risk (impact) of the test case and its priority in the test. Like the complexity ranking, here are sample criteria you can adapt for your own situation:

1 – Lowest priority, lowest impact

2 – Low priority, low impact

3 – Low priority, moderate impact

4 – Moderate priority, moderate impact

5 – Moderate priority, moderate impact, may find important defects

6 – Moderate priority, high impact, has found important defects in the past

7 – High priority, moderate impact, new test

8 – High priority, high impact, may find high-value defects

9 – High priority, high impact, has found high-value defects in the past

10 – Highest priority, highest impact, must perform

Actually, the risk level could be seen from two perspectives - the risk of the item or function you are testing, or the risk of the test case itself. For example, if you fail to perform a test case that in the past has found defects, that could be seen as important enough to include every time you test. Not testing it would be a significant risk. The low risk cases would be those you could leave out and not worry about. Of course, there is a tie-in between these two views. The high-risk functions tend to have high-risk test cases. You could take either view of test case risk and be in the neighborhood for this level of test estimation.

Charting the Test Cases

To visualize how this technique works, we will look at how this could be plotted on a scatter chart. There are four quadrants:

1 – Low complexity, low risk

2 – High complexity, low risk

3 – Low complexity, high risk

4 – High complexity, high risk

Each test case will fall in one of the quadrants. One problem with the quadrant approach is that any test case in the center area of the chart could be seen as borderline. For example, in Figure 1, TC004 is in quadrant 4, but is also close to the other areas as well. So, it could actually be in quadrant 1 if the criteria are a little off.

Figure 1

For this reason, you may choose instead to divide the chart into nine sections. This “tic-tac-toe” approach gives more granularity. If a test case falls in the center of the chart, it is clearly in section 5 (Figure 2), which can have its own set of test estimation factors.

Figure 2

All You Need is a Spreadsheet

With many test cases, you would never want to go to the trouble of charting them all. All you need to know is in which section of the chart a test case resides.

Once you know the complexity and risk scores, all you need to know are the sections on the chart. For example, if the complexity is 3 or less and the risk is 3 or less, the test case falls in section 1 of the nine-section chart. These rules can be written as formulas in a spreadsheet (Figure 3).

Figure 3

Sampling

So, what if you don’t have a good history of how long certain types of test cases take to perform? You can take samples from each sector of the chart.

Take a few test cases from each section, perform the test cases and measure how long it takes to set-up, perform and evaluate each test case.

You now extend your spreadsheet to include the average effort time for each test case (Figure 4).

Figure 4

Adjusting

Your estimate is probably inaccurate. There is a tendency to believe the more involved and defined the method is, the more accurate the estimate will be. However, the reality is that any method can be flawed. In fact, I have seen very elaborate estimation tools and methods which look impressive, but were inaccurate in practice.

It’s good to have some wiggle-room in an estimate as a reserve. Think of this factor as dial you can turn as your confidence in the estimate increases.

Conclusion

Like with any estimation technique, at the end of the day, there could be any number of things that could impact the accuracy of most estimates. Estimates based on test cases can be helpful once you have enough history of measuring them.

Sampling can be helpful if you have no past measurements, or if this is a new type of project for your organization. It is still a good idea to measure the actual test case performance times so you can incorporate them in your future estimates.

I hope this technique helps you and provides a springboard for your own estimation techniques.

New Test Automation Class a Success!

2010-05-18T11:29:00.007-05:00

Thanks to everyone that participated in last week's presentation of my newest workshop, Practical Software Test Automation in Oklahoma City. The class went well, we had a great time together, and I learned some adjustments I need to make. Thanks to the Red Earth QA SIG for their sponsorship.

We also got some positive buzz from Marcus Tettmar, the maker of Macro Scheduler. Thanks, Marcus.

http://www.mjtnet.com/blog/2010/05/13/new-software-testing-course-featuring-macro-scheduler/

We use Macro Scheduler as the learning tool in the course for test automation and scripting. By the way, version 12 has just been released. I look forward to trying it!

http://www.mjtnet.com/blog/2010/05/17/macro-scheduler-12-is-shipping/

The next presentation will be next month in Rome. If you live in Italy, or have a desire to take a testing class in a great location, join me there on June 16 and 17. I will also be presenting the Innovative Software Testing Approaches workshop that week in Rome.

Innovative Software Testing Approaches
http://www.technology transfer.eu/event/984/Innovative_Software_Testing_Appro aches.html

Software Test Automation
http://www.technologytransfer.eu/event/985/Software_Test_Automation.html

If you are interesting in having this workshop in your city or company, just contact me!

The 900+ Point Drop in the Dow - A Real-Life Root Cause Analysis Challenge

2010-05-12T09:57:00.005-05:00

Just a random thought today...

The other day when the Dow dropped over 900 points, it was blamed on some trader someplace entering an order to sell "billions" instead of "millions" of P&G stock. To date, they still cannot produce the trade or the trader. Something doesn't smell right.

1. I would think that level of trade would require some sort of secondary approval.
2. Isn't there an audit trail of trades that would lead back to the trader?
3. If a billion dollar trade could do this, shouldn't there be an edit or at least warning message, "You have entered an amount in the billions. Click OK to bring down the entire global financial system."

This makes me question if this was really the case. Other possibilities could be:

1. A run on stocks that was truly panic selling and this was a way to explain it away without spooking everyone else in the country. I guess this is the "conspiracy theory" view.
2. A software defect...and maybe not a simple one. This could be one of those deeply-embedded ones. I know a little about how the Wall St. systems and processes work and believe me, this is not beyond the realm of possibility.

It may be impossible to know for certain. One would have to perform a deep dive root cause analysis, go through the change logs (if they exist), look at the exact version of code for everything going on at that time, look at a highly dynamic data stream...you get the idea. That probably won't happen. If someone does manage to isolate this as a software defect and can show it, I nominate them for the Root Cause Analysis Hall of Fame, located in Scranton, Pa. (Don't go looking for that...but there is a Tow Truck Museum in Chattanooga, TN.)

Just a random thought...

Great Customer Service in Action at Blue Bean Coffee

2010-05-04T09:37:00.002-05:00

I think as software testers (and IT professionals in general) we need to pay a lot of attention to how we serve our customers. Many companies make the outsourcing decision because IT is no longer delivering the level of service the company needs. Frankly, I have been in some IT departments that deserve to be outsourced due to complacency and lack of business focus.

However, people know great service when they see it.

This morning I was having coffee with my pastor at Blue Bean coffee here in south Oklahoma City (SW 134 and Western). All of a sudden, the lady behind the counter ran out the door with a can of whipped cream in her hand. Turned out she had forgotten to top off the customer's drink. She said, "That would have bothered me all day."

Wow. That made an impression. Really, it's not that big a deal, but in today's "lack of service" culture it stands out.

Compare that to my experience a couple of weeks ago at Starbucks, not too far away. I go in with my own mug and ask for the free cup of coffee they were promoting for Earth Day. (This happened on Earth Day!) The barista looked confused and went in the back office to ask the manager. He came back out and said, "Sorry, that was last week." "OK," I said. "I guess I'll have a tall bold." To which he replied, "We only have a little left." "Is is fresh," I asked. "We change it every 30 minutes," he said. Now, you coffee drinkers out there know that a little coffee left on the heat for even a short time gets bitter, but I took my chances. Turned out, it was bad but I has already left. I complained to Starbucks online and got a good response back and two free drink coupons.

I was left wondering, though, how hard would it have been for the barista to simply say, "That promotion was last week, but here, have a cup anyway."? Perhaps he was not empowered to do that, or maybe he just wasn't thinking about the lifetime value of a customer.

Remember how Starbucks has tried all the other "ambiance" stuff - music, etc. I applaud the efforts at creating an experience, but to me the customer, I don't go to Starbucks for music. I go there for coffee and hope for decent service. Sometime they are friendly and sometimes they aren't. For the longest time all they served was Pike Place. Then, finally, someone in corporate woke up and decided to start offering different blends again.

All I can say is that I'll use those Starbucks coupons on the road and get my local coffee at Blue Bean. It's more personal, more friendly and they have better coffee. And that's what it's all about - the coffee and the service.

Keep that in mind as you serve people in IT. If you give great service and have a great product, people will find you irreplaceable and keep coming back for more. You will be more personal and more valuable.

StarEast 2010

2010-04-29T06:36:00.005-05:00

This has been a great StarEast conference! Thanks to everyone that attended my sessions - the full-day tutorial on Monday and my two track sessions on Wednesday. In the track sessions I offered some resources:

A Deeper Dive into Dashboards

PowerPoint slides of the presentation (PDF)

Excel dashboard with macros
Excel dashboard without macros

The Xcelsius dashboard example
The full 16 minute video of how I created the dashboard

(The reason I give both versions is because the macro-enabled one will update if you are connected to the Internet.)

Here are a couple of the websites I mentioned:

www.dashboardspy.com
www.datapigtechnologies.com

The Elusive Tester to Developer Ratio

We were discussing how to leverage high ratio situations, like 1 tester to 10 developers and higher. The question was asked about how to get testers more on-board with doing unit testing. One suggestion I had was to provide with with a checklist. Here is a unit test workbench with two checklists. This is in Word format, so feel free to modify it to meet your needs.

By the way, I plan to have narrated versions of both presentations posted soon on my website.

Stay in Touch With Me!

There are several ways to stay in touch:

1) Sign up for my newsletter
2) Follow me on Twitter - @rricetester
3) Become a Facebook fan

I hold a monthly drawing for free books for my newsletter subscribers and Facebook fans. The next one is on May 1.

Thanks and have safe travels home!

Oklahoma City Test Automation Training Workshop - May 13 and 14, 2010

2010-04-25T13:26:00.003-05:00

We still have seats left for the new hands-on test automation workshop I am presenting May 13 and 14 in downtown Oklahoma City. If you live nearby, say in Dallas, Little Rock, Wichita, Lawton, Tulsa, Amarillo, heck, even Kansas City or Topeka, it is worth the drive and two night's hotel stay. At $400 per person, this is a crazy deal. Especially since this kind of training doesn't happen often in our part of the country. However, this is the first presentation and I need feedback before presenting the workshop in Rome in June.

This workshop is sponsored by the Red Earth QA SIG and hosted by Devon Energy. Thanks greatly to those organizations.

This workshop is not oriented to any specific tool, but the lessons and concepts are transferable to any tool. We will be using some free and inexpensive tools for the exercises, so bring your own computer.

Nervous about scripting? That's OK. You will be able to work at a pace that suits you and I along with others in the workshop will be there to help.

To learn more and to register, just visit http://www.mysoftwaretesting.com/product_p/okcauto.htm

If you would like for me to come to your company and present the workshop, just call me at 405-691-8075.

Thanks!

Randy

My Take on the McAfee Mess

2010-04-23T08:59:00.002-05:00

OK, first let me say I wasn't there, so like Will Rogers is quoted as saying, "All I know is what I read in the papers." In my case, the paper is Computerworld.

On April 21, McAfee released an update to their anti-virus application that disabled systems running Microsoft XP SP3. This caused a bad day at McAfee, but even worse days at all the companies and homes that were impacted.

You can read the early account here:

http://www.computerworld.com/s/article/9175928/The_McAfee_update_mess_explained?source=toc

Then, today, a more detailed explanation and apology was issued:

http://www.computerworld.com/s/article/9175940/McAfee_apologizes_for_crippling_PCs_with_bad_update?source=CTWNLE_nlt_dailyam_2010-04-23

The bottom line was a critical defect was missed and made its way to customers. Here are my observations as an interested bystander and software testing consultant:

1) The apology was cryptic for a technical audience. "We recently made a change to our QA [quality assurance] environment that resulted in a faulty DAT making its way out of our test environment and onto customer systems." However, no explanation of the change was given. Was a platform removed, or skipped? Was a test case skipped? Was there a rush to get the update out? Why was the environment changed? The blame seems to be on the change to the environment.

2) The risk was very high. This is the tester's worst nightmare and an example of what you don't want your company to go through, or your customers. This is a credibility-basher. I work with some software companies that say "We don't care about the risk. If there are problems, we'll just post a hotfix on the web site." Right.....

3) Testing can't find all the defects. However, testing is an easy role to place blame. At least they didn't blame the testers - they blamed the process and the environment, which is probably the appropriate place to focus.

4) This points out a big risk for COTS applications - applying an update without testing it. I know the updating process is automated for large companies. However, one of the things I teach in my COTS testing class is to test the updates before rolling them out to the entire company. I suspect this will be one of those lessons learned for many people. The really troubling thing is for individuals who get impacted. They have no "test" PCs.

5) The customers want to hear from the CEO over this. So, your CEO doesn't seem to care about testing? This is a good case study to show why they should care.

6) Your test is only as good as your environment. You may have great tools, great testers and great processes, but if you have gaps in your environment, you don't know for sure what you are testing.

7) This is one of those head-bangers. Apparently, this was not one of those deeply-embedded defects, but one that could have been found just in a simple update to a commonly-used platform. This is one of those defects that leaves management and customers asking, "Why didn't you guys test that? (I refer you back to observation #1) They aren't saying for sure.

8) There will be more defects escape in the future. The only question is, what will the impact be? If you really want a scare, take this scenario out to medical devices, aircraft, automobiles, utilities and other safety-critical applications. No matter how hard we try, there will still be defects because we can't test everything. That's not an easy reality to embrace because many people have grown to trust that software just works - mostly. Testers know better

OK, enough of being the armchair quarterback. This is serious and frustrating, but not the end of the world. A few weeks from now, it will all be forgotten. Actually, that's part of the problem. We experience the pain, the pain goes away, then we experience it again...and again.

Your thoughts?

Striving for Quality has Real Payoffs

2010-04-22T09:56:00.002-05:00

Here is an interesting article about the tie-in between IT, quality and business growth:

http://www.computerworld.com/s/article/9175836/Striving_for_quality_has_real_payoffs?taxonomyId=14&pageNumber=1

The common misconception is that the 20th century quality gurus, like Deming, Juran and Crosby were out there teaching about quality for the sake of virtue. The reality is that they taught quality concepts as a way to build the bottom line in a company by gaining customers and serving existing customers better. That's why I get really annoyed at people who dismiss quality improvement as an optional thing. I agree that "quality" has been so overused is fails to resonate anymore with people, but so does "safety" until you get hurt or killed. The problem is not the concept but our thinking (or lack of attention).

Read this article and learn how to make the message to management. The great thing about this article was that management made the mandate to IT. And that is exactly the direction the quality mandate should flow.

Have a quality day!

Randy

Technology is Just the Tool

2010-04-21T10:28:00.003-05:00

"The more elaborate our means of communication, the less we communicate."
— Joseph Priestley: Was an 18th-century theologian and educator

Isn't it interesting that this was written in the 18th century? It's like the idea that instead of technology making our lives easier, it has done just the opposite. Technology is just the tool. You can use any tool relentlessly and still wear yourself out. You can use any tool in an ineffective way.

One of my favorite lines from a movie is when Pappy O'Daniel says in "Oh Brother, Where Art Thou?" as he and his entourage are walking into the little radio station, "We ain't one-at-a-timin' here. We're MASS communicating!"

Sure, technology has changed the means of communication. What it has not changed is how we really connect with someone.

10 Reasons Why You Aren't Done Yet

2010-04-17T11:13:00.002-05:00

Here is a great post by Michael Hyatt on personal productivity and the work habits (and sins) that beset us. I found this article helpful and if you are like me, with too many "not dones" on my "to-dos", you will also find it helpful.

http://michaelhyatt.com/2010/03/10-reasons-why-you-aren%E2%80%99t-done-yet.html

It's About Communication

2010-04-16T09:14:00.005-05:00

I've been teaching for a long time now that I believe communication is the basis for everything we do in IT. However, communication is the one thing we ignore the most - both in practice and skill building. I dislike the term "soft skills" because "soft" implies easy or optional. The reality is that unless you communicate well with everyone in the organization, you lack the knowledge to solve problems and keep a project on track.

I ran across an interesting article this morning at computerworld.com about how one CIO does this :

CIO says communication is key to buy-in

No matter which role you are in, communication is a big part of your job, whether you practice it or not.

Here are some quick tips:

1) Be intentional

You have to keep communication in mind and be thinking all the time, "Who needs to know this?" You also have to be thinking "What do I need to know?" and "Whom should I be speaking with today?"

2) Do not confuse e-mail and meetings with communication

These can be vehicles for communication, but they have flaws. E-mail fails to convey the tone of our communication and meetings can drift into discussions of many topics. Just pick up the phone or walk down the hall if you really want to touch base with someone.

3) Practice

Yes, practice how you say things. Learn the words that carry meaning and power. Learn how to control your body language and read the body language of others. Do you have a big presentation in your future? Do you have an important conversation with someone soon? Practice!

4) Read before sending

So you've written an e-mail, status report or some other piece of written communication. Take an extra 60 seconds and read back through it. Place yourself in the role of the people who will receive it. How would be feel if you were the recipient? Are there any obvious gaps, typos, or other mistakes? Testers get nailed on this because we are highlighting others' mistakes. People aren't perfect, but try to find mistakes the best you can.

A really good book on this subject is Naomi Karten's "Communication Gaps and How to Close Them".

I hope I have communicated my thoughts on this well! Your thoughts?

Have a great day.

Randy

Being a Linchpin in a Process-Oriented Environment

2010-04-08T16:43:00.003-05:00

I had lunch with my friend Frank today and I asked him his perspective on the Linchpin idea of being an artist and how that fits or conflicts with the ideas taught by Dr. Deming. Frank and I both like Dr. Deming's ideas a lot, so I am trying to set a context that makes sense.

In Linchpin, Seth Godin writes about becoming indispensable at our jobs by becoming more unique. We become more unique by letting the artist in us shine forth in what we do. Godin admits there are times you would not want this, like for airline pilots, etc. However, that specific example caused me to think about Captain Sullenberger who skillfully guided the U.S. Airways jet to a safe landing in the Hudson River. That was artistry. My friend Frank pointed out this was not because of his training from the airline, but because he knew how to fly gliders. Interesting.

In a factory setting, you want people making the same things in the same way. Dr. Deming taught people to remove variations in the process. He taught that processes should be so well defined that anyone with the right training could perform them with identical results. We can apply that in some ways to software development. So, how does one let their artistry shine in that kind of process-oriented environment?

Frank observed that Quality Circles was a technique Dr. Deming taught to let the workers in a process have a voice in improving it. Those people who are good at seeing problems and suggesting changes to the process that save the company money are linchpins and artists. You would not want people to start changing the process whenever or however they want, but you do want to know how to improve the process at the right time, in the right way.

Certainly Dr. Deming valued the person and how they contribute to quality. However, he was not a proponent of surviving by heroic efforts. The problem of heroic efforts is that is rewards those who do sloppy work and then swoop into to fix it, while those who consistently do good work get little recognition.

Unfortunately, too many good people suggest good ideas to management and those ideas are ignored. These people often leave for companies that do value their ideas. They can do that and get more money in the process because they add value. They are linchpins. This applies to leaders as well. Leaders who listen to ideas will rise and succeed, while those who don't will continue to struggle.

Your thoughts?

Randy

Book Review - Linchpin by Seth Godin

2010-04-02T12:40:00.005-05:00

For a long time I have been telling software testers there is someone out there, either in their city or halfway around the world, that can test faster and cheaper than they can. Like it or not, software testing has become a commodity for many organizations. It doesn’t matter who is doing the testing, how they are doing it, only that testing is being done at a cost they are willing to pay.

These are the same companies who call me up to get “two days of training on testing” then have no idea what their needs are, what will happened after the class, or even who will be the instructor. They just want “a pound of training” as I call it. This type of attitude means you need a point of distinction. That’s what this book is all about – how to become indispensable by letting the artist in you emerge.

Think of the companies that lay off hundreds or thousands of people all at once. How were those people chosen? They didn’t lay off everyone, so who got to stay and why? It wasn’t the people who obediently played by the rules and didn’t make waves. The ones who stayed were difference makers – linchpins.

This book takes you on a journey. It starts with some background of how we got here. After all, for all of human history except the last 60 years or so, people lived without the idea of being taken care of by a company. Now, we find ourselves in a different ballgame with a whole new set of rules. Having a good job is no longer the definition of success. Lots of “good jobs” have gone away, never to return.

The journey continues to discuss how to become that person that the company values so highly, they will do anything to keep you – the linchpin upon which there success depends.

Of course, becoming and staying a linchpin isn’t easy. There is a lot of resistance to stay in your comfort zone. Godin goes into detail about the choice required to become a linchpin and why it’s the choice between being remarkable and being a cheap drone.

The brilliant thing about this book is that Seth goes into depth about “the resistance”, the thing that keeps us from being unique and taking the risk to let our special gifts shine. It is so easy to want to blend in and go with the flow, but Seth makes a compelling case that this is the path toward extinction.

After reading the book and thinking of my experiences in consulting in many companies, I realize that the only difference between many companies and a chicken slaughterhouse is the lack of chickens. Every day people dutifully report for duty, keep their heads down, do what they are told to do (or not to do), then go home and do it all again the next day. The work is mindless, requires no creativity and at the end of the day, people leave unfulfilled and unappreciated – and a bit bloody.

At times I felt conflicted as I read Linchpin. I identify with being the artist. I tend to procrastinate, so the idea of setting a ship date for something and then shipping no matter what has actually helped to get some things done. On the other hand, I have seen so many companies suffer from the deadline mentality I can’t totally embrace that idea.

I think there is balance in the decision to ship or not to ship. In some cases, shipping bad stuff on the deadline can be very bad. In other cases, it can be the first faltering steps toward a great eventual product. I think it all depends on how understanding people are. For things like cars, airplanes and pacemakers, it’s a bad idea to ship just based on the deadline being reached.

The other point I have difficulty with is the idea of not being attached to your worldview. I agree that we need to be able to see and understand other people’s worldviews. My point of departure comes in being so unattached that you have no stable set of beliefs. I think it’s important to know why you hold your worldview and even be willing to question it. I also think it’s fine to be passionate about your beliefs. The key is whether or not your beliefs are based in truth. I know, I know, this opens up a great philosophical discussion about “what is truth?” I just think you can be a linchpin and still be true to your beliefs.

Finally, perhaps the most troubling realization of all is the contradiction of on my closely held beliefs in the idea of systematizing of processes. I really think Dr. Deming had it right about defining a process so that anyone could perform it without error. This is a good thing for factories, hospitals and fast food places that rely on consistency of results. However, when it comes to intellectual work, we need the creativity of the artist.

This leads me to the application of this idea to software testing. On one hand, we need high levels of accuracy and repeatability for some types of testing. On the other hand, we need the creativity of the artist for discovering new defects and ways of performing testing.

This is a great book, an essential book, for anyone in today’s marketplace. Both young and not-so-young will be better prepared to deal with the world by reading Linchpin. This is an easy read and I especially like the way Seth fleshes out his ideas in a stream of coherent small sections of writing. This alone has changed how I plan to write my future books - many of which now have publication dates! Thanks, Seth, for another insightful and timely book.

I will be continuing these thoughts here on my blog in the coming days, so come on and join the discussion!

Dirty Little Secrets About Software Test Estimation

2010-04-02T12:31:00.003-05:00

I've been doing a lot of test estimation lately and it causes me to recall things that I have taught in the past about project estimation and test estimation, in particular. I call these things “Dirty Little Secrets” because they are things that many people know, but few people admit.

The context of this article is about software test estimation, but some of these secrets can be seen in other types of estimating as well. This list is not intended to be complete, but rather the things I see most often in trying to estimate software testing efforts.

Secret 1 – Your Estimate is Probably Wrong

Take comfort in this. Too many people work and work to get “just right” estimates. The problems are:

1) As a tester, your estimates often depend on other estimates, such as the time needed for development, people available, etc. These numbers are often inaccurate, so that means yours will be also.

On one project, I was asked to come up with some quick estimates on about 15 projects for a given business. To help in this effort, I was sent a set of 15 documents, each of which contained estimates for one of the projects to be estimated. My approach was to apply a factor of 25% of the total development effort. Now this factor could be considered low, since the actual percentages can range between 20 and 50%.

However, there was one thing that made my test effort factor practically irrelevant. Many of the prior estimates had “TBD” on the design and code efforts. This gap in data rendered my assumptions to the level of sheer guesswork. I had a bigger problem, though. My client still wanted estimates. What's a consultant to do?

In this case, I made some assumptions with a warning that the estimates could be way off. I estimated the effort to design the project equal to that of gathering and documenting requirements. I also made the effort to code the projects the same as the effort to design the project. Yes, this gave the same number for person-weeks for requirements, design and coding. Keep in mind I had no resource numbers at this point, only estimated weeks of effort. I documented my assumptions and also the low confidence I had in the estimate. I assured my client the estimate would change once we had numbers for design and coding. Plus, we would need resource estimates before we could estimate costs.

About a week later, I got the missing data. Not surprisingly, the newly supplied estimates for design and coding were different from the ones I assumed. So, I used the new numbers and my estimates changed. I have yet to know how accurate any of the estimates are at this point.

(By the way, as things turned out, none of the projects were ever started!)

2) The Project Scope Will Probably Change.

There's not much you can do about this. Scope change happens for a variety of reasons, some foreseen and some unforeseen.

You never know when new requirements will emerge. New opportunities can appear as well.

Last year, I had a fence installed at my home. When we first measured the dimension of the area to be fenced, I used the area that I had assumed was my property. Since there is not a house on the other side of us, I could not use the adjoining property as a basis. After reading the plat map, I learned that our property is actually about ten feet narrower than I had believed. For years, I have been watering, fertilizing and mowing about 1,000 square feet of property that did not belong to me!

This new information changed the scope and therefore, the estimate and final costs of the project.

Software projects are more complex than home improvements, and this makes the challenge even greater. Our assumptions get challenged and often proven wrong. Whether it is the scope of requirements or the number of resources needed, scope can change either up or down.

3) You Don't Know What You Don't Know.

And...you won't find that out until it's too late. This is one of the biggest project risks and impacts much more than estimates. There's not much you can do about this one either. Just keep an open mind, keep questioning decisions and information throughout the project. Realize that you don't know everything and what you do know will change.

Here are a couple of prime examples of project “surprises” :

People will fudge numbers to make themselves look better. Realize this happens and always question information, whether it be bug counts, estimates or the time spent on testing a feature.

Deliverables will be late. In fact, they will be later than you think they will. Some things will be early, but more often than not, your work will be delayed by delays experienced by others.

Secret 2 – Your Estimate Needs to be Wrong – In the Right Direction

Why do I say something so outlandish? You only know how close your estimate is when the work is done. When my focus is on the accuracy of the estimate, I am at risk of conforming the work to the estimate, not what is in the best interest of the project.

When you truly understand your estimate is just that – an estimate, you can be free to adjust it. At least, you can adjust it for your own purposes if your management is not accepting of reality.

So what is the “right” direction? It's high.

There's an interesting debate among project managers as to whether or not it is ethical to “pad” estimates for the purpose of dealing with future unknown needs. Some believe you must have pads and others believe it is unethical to give an estimate you know is higher than you think is accurate.

I fall into the camp of overestimating. However, I don't call this extra a “pad”. Instead, I call it a “reserve” that is the safety net should extra time, money and/or people be needed. Even Scotty never gave Captain Kirk the real estimate!

I don't see this as deception. I see it as contingency planning and a necessary part of any estimate. In my experience, it's better to come in “under” than “over.”

It's funny that the people I know to be the best estimators have been doing it a long time and their estimates seem very high at first. I think these people have learned the lesson of having reserves in the estimate.

Secret 3 – The Recipient of the Estimate Already Has a Number in Mind

This is why I say that estimation of anything is an inherently human activity. There's a lot of feeling involved and not much logic at times.

Understand that your customer has an expectation already in mind. If your estimate goes too far above or below the customer's early expectation, silent alarms go off in their head.

In the case of my fence project, I had a number in mind. It wasn't a low price and it wasn't too far off from the estimate I received. When I did get the estimate, at least I didn't have sticker shock. Had my expectation been a lot lower that the estimate, I would have probably looked for a cheaper option.

Once I had an experience in which I gave an estimate that was so much lower than a competitor's and I almost didn't get the work! The prospective client felt that because of the lower price I must have been leaving something out. Thankfully, they asked me to explain why my quote was so much lower and I explained it was because I had a faster and more effective way of doing the job.

Secret 4 - Your Customer Assumes Your Estimate is Too High

This is one of the biggest hindrances to good estimation. Let's say that you do your best job of estimation with no reserves included. You present these to your management or customer and they take one look and immediately reduce it by one-half because they believe you must have inflated it. After all, the last person they got an estimate from did so. Therefore, the assumption is that everyone who estimates has excess in the numbers.

What does this do? It forces people to pad their estimates because they know management will assume they have been inflated. Therefore, it is a self-fulfilling prophecy.

Secret 5 – Your Estimates are Incomplete

Most people estimate time and cost. How many people do you know that also estimate the quality of the product? Not many I would guess.

You have probably heard the old adage, “cost, schedule, quality – pick two.”

While it's true that it's very hard to deliver all three aspects, it's still possible. High quality will probably cost more and may take longer than lower quality – at least in the short term. In the long run, high quality costs less because of lower maintenance.

In software development and testing, we often fail to consider the quality levels in a given estimate. For example, how much would a test cost that found a few defects and resulted in a buggy product versus one that found many defects (or found the right ones) and resulted in a high-quality product? More often than not we estimate how long testing will take, how many people we need, and how much testing will cost. A lot fewer people estimate how many defects they find at the various levels of criticality.

Conclusion

As you go forth and estimate, keep these little secrets in mind and you will have less stress over your estimates. If you look at how people respond to your estimates, you will see many of the secrets in action. These are things you can't control. However, you can understand that the dynamics exist and be prepared to keep expectations reasonable.

Toyota and Software Problems

2010-02-24T08:38:00.002-06:00

A very interesting drama is unfolding with Toyota over quality and safety issues. If anyone knows the quality process, it is Toyota. The mistake was not following the process and being in denial about problems really being problems even in the face of reality. Others are guilty, too. Ford, GM and Chrysler have all had issues - remember the Ford Pinto?

Here's the deal. More than ever, software is an integral part of the electronics that control cars. All software has defects. When a software defect causes a car to accelerate to 100+ mph, that's not a bug. That is catastrophic system failure. A few years back, Volvos were coming to dead stop at highway speeds because of software defects.

A friend of mine last year spent months trying to convince multiple Ford dealers that her Ford Expedition was stalling in the highway. They all said the Electronic Control Unit was fine. Eventually they found the problem - the ECU was bad.

The facts are not all in, but I would not be surprised if many of the Toyota problems are software defects.

My prediction for many years is that one day a major software failure will cause such death and destruction, that congress will start to regulate any software development with safety impact, much like is currently done in the FDA and NRC. The next big area of regulation will likely be transportation. Not that this is the answer. Regulations have fallen short because people find ways to get around them. Ultimately, quality is an ethical business issue.

For many, many years, Toyota has had a halo in terms of quality. Now that halo is gone and may never be recovered.

Keep an eye on this story.

ISTQB Advanced Level Certification Public Courses

2010-02-18T14:25:00.012-06:00

If you hold the ISTQB Foundation Level certification (CTFL) and are ready to move to the next level in ISTQB software testing certification, we are offering advanced level courses in Oklahoma City staring in May, 2010.

The courses offered are ISTQB Advanced Core (2 days) and ISTQB Advanced Analyst (3 days). To sit for the ISTQB Advanced Analyst Exam on the last day of the course, both courses must be completed.

To see the complete schedule of all courses and pricing, click here.

If you would like to have advanced courses in your city in the USA and have 6 or more people that can attend, contact me and I would be happy to schedule a class close to you.

To learn more about ISTQB Advanced Level Certification and the pre-requisites, please click here.

Rice Consulting Services has teamed with Grove Consultants (http://www.grove.co.uk) to provide an outstanding training experience for advanced level certification.

I will be the trainer for these courses. I bring over 20 years software testing training experience in major organizations worldwide, as well as holding the full CTAL certification. The materials are licensed from Grove Consultants in the U.K. If you have seen any conference tutorials or presentations by Grove Consultants, you know the high quality and engaging nature of their presentations and materials.

ISTQB Advanced Level testers are an elite group in the USA right now, so now is the time to join that special minority of software testers and managers!

Foundation Exam and Book Now Included in ISTQB e-Learning

2010-02-12T11:49:00.000-06:00

In our never-ending dedication to bring you the best value in training, we are excited to provide our students one electronic exam voucher per student as part of the registration. Each student also receives a copy of the book Foundations of Software Testing.

Our students have great success in passing the exam. Every student has direct access to Randy by phone and/or e-mail to answer any questions along the way.

Combine all this with a money-back guarantee, 12 months access to materials (even after you take the exam) and you have a great deal for preparing for the CTFL exam and building your testing skills to apply on your job.

Click here to read more about the course.

Click here to take a free demo.

Click here to buy a registration for the course.

Instant Software Testing Metrics

2010-02-09T18:25:00.001-06:00

There aren't any instant software testing metrics, at least none that give much meaning. If you are looking for instant testing metrics, sorry, this article won't provide them. I confess this title is a “hook” to get your attention. However, if you are willing to consider that it takes time to grow and understand metrics, then keep reading.

It seems that so many people I consult and train want quick and easy measures from their software development or testing efforts. I wish I had that shiny silver bullet, but in my experience and observation it takes time to really get the metrics in place that works for you and your organization.

And, really, you sure don't want to make decisions based on faulty information.

Here's why good metrics take time:

You Need History

At any point in time, a measurement is like taking one frame from a movie. For it to make sense, you need to see what has happened prior to the frame. To know how things turn out, you need to see what happens after the frame.

In software, this history starts when you start taking measurements. It continues as you keep measuring and refining your measurements.

The longer the history, the more accurate your understanding of your metrics tend to be. The key word is “understanding”. Your measurements may not necessarily get more accurate over time unless you keep questioning and improving them.

Too many people want to substitute other organizations' history for their own. I discourage this practice because every situation is different. Just because one company has a certain level of success with a given level of people or type of test technique doesn't mean you will have that same level of success.

Just start measuring a few key things that are easily obtainable and that are meaningful. It's important to track measures over time with specific dates associated with them. For example, you can start measuring the number of defects found by project. While this measure alone doesn't make a metric, you can start to see trends concerning the rate of defect discovery.

You Need Context

Metrics give context to measurements. A measure is a single count or extent of something, such as the number of defects. A metric is a measure taken in context with other measures. So, the average number of defects per function tells us the defect density from the functional perspective.

It takes time to know the best way to get this context and understand what it really means to you. I was once on a project where the client wanted to know how many test cases passed during a test. Fine. That's a good thing to know, right?

Then, people started to observe that the measure of passed tests has different meaning depending on when it is measured and reported. For example, a very high percentage of passed tests in the first round of testing may indicate the tests are too weak. Just before deployment, you want a high percentage of passed tests to indicate the application is ready to release. So, you may want to have two metrics: “first time tests that pass” and “final test pass percentage.”

You Need Time for Refinement

After you measure things for awhile, you may learn you aren't measuring the right things, or you may be measuring the right things in the wrong way.

It's fine and even expected to make major re-adjustments in your measurements. Just make sure to indicate on charts and tables when the adjustments were made.

You Need Growth

Please don't start a big metrics program with dozens of measurements. These programs often fail under their own weight. They are so big and complex, people either ignore them or give up quickly. Instead, start small and grow over time.

Metrics is a discipline that takes time and attention to do right. It also takes time to gain people's trust about how the metrics will be used. Sometimes people are fearful that the measures will be used against them, such as in a performance review.

You Need Early Successes

Pilot projects or proofs of concept are great to show the value of ideas and approaches. They reduce the risk of failure and gives you a place to practice and perfect things before trying them in the larger arena where everything becomes much more visible. It's a lot better to build positive public image based on small successes that to overcome the negative image from a major public failure.

You Need Management Understanding and Buy-in

As you grow and publish your metrics, management needs to learn the great value in them for the management of projects. Management also needs to get used to the information you can provide. They may suggest or request additional metrics or changes to existing ones. That's a good thing because it shows their minds are into the effort. This only happens over time.

Summary

I hope this shows why it takes time to get reliable and meaningful metrics. There are no magic answers, tools or techniques for getting a good set of metrics in place. That's why so few organizations reach this level. It's hard work and takes time, but it's worth the effort to show the added value you bring to your company. When you are “the” person that understands the metrics, that makes you a key person not only for your team, but for your company...and that's a very good thing!

Free Virtual Conference on Application Performance Management

2010-02-09T14:39:00.002-06:00

I'll be speaking at a FREE virtual conference on application performance on Feb 24 sponsored by SearchSoftwareQuality.com. My topic will be "Finding, Diagnosing, and Fixing Performance Bugs: Choosing Tools, Refining Best Practices."

Here's the announcement from SearchSoftwareQuality.com:

We're just 2 weeks away from the complimentary virtual seminar
'Application Performance Management: Build, Bug and Lifecycle
Strategies.' Gain access to industry experts and real-world
practitioners who reveal new insights to boost app performance and
ensure it remains a key focus throughout your application development
lifecycle.

Don't miss this opportunity - register here and mark your calendar:
http://go.techtarget.com/r/10840914/7113473

===============================================================
TITLE: Application Performance Management: Build, Bug and
Lifecycle Strategies
WHEN: Wednesday, February 24, 2010
TIME: 9:30 am - 6:00 pm EST
WHERE: Your Desktop
COST: Free

REGISTER FOR THIS VIRTUAL SEMINAR TODAY:
http://go.techtarget.com/r/10840915/7113473

===============================================================

TOP 5 REASONS TO ATTEND THE APPLICATION PERFORMANCE MANAGEMENT
VIRTUAL SEMINAR:

1. GET PRACTICAL INFORMATION TO USE IMMEDIATELY
Hear relevant advice that you can put into action right away to help
your organization build the foundation for high performance. No
pie-in-the-sky theories, just practical, useful information from
today's top performance experts.

2. EXPAND YOUR NETWORK OF PEERS
Share insights and discuss relevant strategies in our unique virtual
environment with those facing the same challenges you're up against
when testing and setting the stage for monitoring and maintaining
application performance throughout its lifecycle. Stop by the
networking lounge to hear success stories and lessons learned.

3. INTERACT WITH EXPERT SPEAKERS DURING OUR PANEL DISCUSSION
Discuss performance implications and testing tools for running
application components in private and public clouds. Don't go it
alone when you can benefit from the experience of these true
performance and cloud experts.

4. PRIZES!
Sessions are an intensive, educational experience that will prove
incredibly valuable to your organization, but we've built in some
fun, interactive breaks during which we'll announce our prize
winners. Attend the day's seminar for a chance to win a Sony
Cyber-shot camera, an Amazon.com gift card, or a book on dependency
injection. You'll have multiple chances to win - by visiting a booth,
attending a session and just for coming to the event!

5. INTERACT WITH TOP VENDORS
Speak directly to leading performance vendors and test drive the
latest solutions with live product demos. They understand your pain
points, and strive to help you make the best IT investments for your
organization.

Ensure quality in your application performance management. Register
with one click here:
http://go.techtarget.com/r/10840916/7113473

Book Review - Managing the Testing Process, 3rd Ed. – Rex Black

2010-02-05T19:24:00.003-06:00

Managing the Testing Process, 3rd. Ed. by Rex Black

Published by Wiley, 2009, 638 pages

In the third edition, Rex extends a work he started in 1998 with the first edition. At that time, and still today, practical books on software test management are in a minority of the books on software testing. This book has become a commonly referenced work on software test management for a reason - it is practical.

Two things I really like about this book are that it is very readable and it has broad coverage of test management topics. As a trainer of thousands of test managers, I have learned that many test managers are thrust into the role with little knowledge or preparation. So, my perspective is that test managers need to know the mechanics of the software test profession as well as the managerial and leadership aspects of the job. This book delivers well on both counts.

By reading and applying the information in this book, you will learn the testing process from test planning and building the test architecture to building a test team, measuring test results, and conveying those results in a value-added way. Rex covers topics that are very relevant including test outsourcing and the context of projects and software lifecycles. I also very much appreciate the discussion of dealing with the people issues in testing.

Just a note on the use of spreadsheets and vendor non-specific tool examples is that once you understand the structure of organizing testware, you can apply that structure in a specific test tool. Plus, not everyone owns a commercial test management tool. I know many people who manage a lot of test items using Excel.

I can highly recommend this book to test managers and leaders, as well as people who aspire to be in those roles.

Randy Rice

Software QA and Testing Salaries

2010-02-03T07:45:00.001-06:00

A common question I get is about salaries for software testers here in the USA. Here is a place you can find that for your location - http://bit.ly/99nJtY

Articles of Interest

2010-01-20T08:45:00.002-06:00

Here are two things I just read and really liked.

1) Here's a great slide show. Those of us who are in the software quality field will recognize the "7 Deadly Sins of IT" - http://www.cioinsight.com/c/a/IT-Management/Deadly-Sins-of-IT-Operations-658137/?kc=CIOMINUTE01202010CIO1

I think it's great to see this kind of thing in CIO Insight!

2) In my Becoming an Influential Test Team Leader workshop we talk about how to influence management. Here is a great article from Michael Hyatt on making a pitch to your boss: http://www.stumbleupon.com/su/4ff3lg/michaelhyatt.com/2007/01/how-to-get-your-boss%E2%80%99s-approval-when-you-need-it.html/r:t

Enjoy!

Tester to Developer Ratio Initial Research Findings

2010-01-06T16:11:00.002-06:00

Thanks to everyone who provided data for my latest research project on tester to developer ratios. This topic has been an interest of mine for over ten years, when I did my first surveys on tester to developer ratios. The results of that survey and my thoughts at the time are in an article called, The Elusive Tester to Developer Ratio.

This short article is to document early findings and I plan to continue surveys and data gathering, so if you did not get in on the first round of surveying, I would like to hear about your ratios.

The participants of the recent survey were subscribers to my newsletter, The Software Quality Advisor, and the audience at my StarWest tutorial on Becoming an Influential Test Team Leader. There were 53 respondents in all, mostly from North America, but six were from Europe and one was from Asia.

I asked four questions:

1) How many developers are in your organization?
2) How many testers are in your organization?
3) On a scale of 1 to 6, where 1 is poor and 6 is super, how would you rate the effectiveness of your current ratio?
4) Do you have any anecdotal information about how your current ratio effectiveness?

The leanest ratio was twenty developers to one tester (effectiveness rating of “two”), while the richest ratio was fifteen developers to eighteen testers (effectiveness rating of “four”). There was one anomalous response of four developers to zero testers (The effectiveness rating on that one was “three”). The average ratio was 4.52 developers to one tester. The most common response was three developer to one tester (six responses), the next most common was 2.5 developers per tester (five responses). There were twenty-six responses with developer to tester ratios of 3:1 or lower.

Here are some of my initial observations and comments:

1) The responses varied greatly.

For those looking for an “industry norm” of developer to tester ratios, this may show that the range of workable ratios is wide. Effective testing may be achieved by better practices, tools and leveraging developer-based testing instead of having more testers.

2) Over half of the responses were at the “richer” ratios.

The average effectiveness reported by this group (3:1 or less) was four – above average. Interestingly, the average effectiveness for the higher ratios was three – average, and not a huge difference from the lower ratio group.

3) In the higher ratio group, there were some with higher than average test effectiveness of four or five.

This tells me that you have a higher ratio and still be effective at software testing. Put another way, the magic of good testing may not be in the ratio of developers to testers.

I have always questioned the idea of using developer to tester ratio as a way to staff or estimate testing efforts. Sheer body count is just not enough information to base testing effort upon.

That said, I think developer to tester ratios may be a helpful metric to understand the workload in a test organization. For example, if I were presented with a situation where the developer to tester ratio is ten to one, I would ask:

Are any test automation tools being used? If so, how effective are they?
How much responsibility do developers have in the testing process?
Is testing based on risk?
Are test optimization techniques used in test design?
What is the defect detection percentage (DDP)?
Are defect trends tracked and studied?
Have the developers and testers been trained in software testing?
Is there a defined testing process in place and being used?

These questions would help determine how the balance and effectiveness of the testing process. Before making team sizing decisions on numbers of people, it may actually be better to use the developer to tester ratio as a metric to guide the testing process.

I did this on my first job as a test manager. I had a team of three people testing the work of thirty developers. The ten to one ratio told me that we could not test all the work coming our way.

We had no tools, just our wits. So, we developed a strategy:

1) Get management to lead the way to make the message to developers that testing is part of their job
2) Train and mentor each developer to be a good tester
3) Test the high risk changes at the highest priority
4) Test anything a developer asked us to test (unless there was no documentation)
5) Do not test anything without a defined user requirement
6) Use cause/effect graphing and other test optimization techniques to get the most testing from the fewest tests
7) Build a robust and repeatable test data set for manual regression testing

The result was that 1) we kept up with the workload and 2) the error rate went from 50% of changes with defects to 2%. At this point, we still had a ten to one developer to tester ratio. This may work for you, too. If it does, please send your check made out to Rice Consulting Services at P.O. Box 892003, Oklahoma City, OK 73170. :-)

I hope this information helps you understand your own ratio a little better. If you would like to contribute your own ratio to my data, just reply to me here with the four items:

1) How many developers are in your organization?
2) How many testers are in your organization?
3) On a scale of 1 to 6, where 1 is poor and 6 is super, how would you rate the effectiveness of your current ratio?
4) Do you have any anecdotal information about how your current ratio effectiveness?

Thanks!

Jim Rohn Has Left the Building

2009-12-08T10:36:00.006-06:00

It was with sadness I learned of the passing this past weekend of a man that had a great influence on my life, Jim Rohn. Most technical people have never heard of Mr. Rohn, but he was a giant in the field of personal development. Jim was the picture of heath until a couple of years ago when he was diagnosed with pulmonary fibrosis. He was 79.

As not only a software tester, but a business owner, I learned early on that I had to learn about what it means to be successful in business. Not in a greedy way, but in a way that helps others. Mr. Rohn taught me how to do that.

I'm not a believer in "The Secret" - the idea that you attract the things that come into your life - good and bad. (I do believe we can attract certain things by what we do, but that's different.) Anyway, I did have an experience that was amazing in the realm of goals.

I started Rice Consulting back in 1990 with no idea of where it would lead or even if it would be successful. I just knew that very few consultants and trainers specialized in testing at that time.

Before long, I realized my business lacked direction and control, so I started looking for how to get that positive direction. One resource I found was a tape set called "The Art of Exceptional Living" by Jim Rohn. I listened to those tapes at least a dozen times while I would be on walks, driving, etc. In fact, I still listen to them on occasion.

I thought, "I would sure like to meet Mr. Rohn in person one day." Then, amazingly about six months later he came to Oklahoma City to present a half-day session on The Seasons of Life. I bought the VIP ticket so I could attend a pre-event reception. I remember like yesterday seeing Mr. Rohn standing at the side of the room with no one else around, so I introduced myself and asked if it was OK to ask a question. He graciously said, "Of course."

I asked Mr. Rohn a question I would not have asked anyone else because I knew he came from the same spiritual perspective as me. "Do you think it is possible to be too rich?" I asked.

He paused, pursed his lips and said, "Well...let me see...Is it possible to be too happy? Or too healthy? Or have too many good relationships? No...I don't think you can be too rich."

Now, context is important here. I had heard Jim's teaching on "enlighted wealth" where it's not the accumulation of prosperity for the sake of ourselves, so I knew he was not advocating wealth at any cost. I was just curious because of my lower middle class background and past spiritual teachings.

Since then, I have heard him say that humans are the only creatures that place limits on their own growth. So, it would be like asking, "Can a tree grow too tall?"

Jim was the person that taught me the importance of communication and how ideas are conveyed. He also taught me that it's possible to create something tangible from something intangible, like an idea.

Jim touched millions of people around the world. If you would like to see the tributes, there are many at http://tribute.jimrohn.com/. There are also some video clips there of his teaching. If you are inspired and want to go deeper, I highly recommend "The Art of Exceptional Living (abridged version)" or full version as a starting point.

Jim Rohn will be missed, but he left a legacy of teaching that will endure for a long, long time. I know he is in a better place.

Back in the USA, Flu and all

2009-11-24T09:39:00.004-06:00

Hi Folks,

It's good to be back in the USA. The courses in Rome went very well. It's always fun and the people are super, but also a challenging adjustment to be teaching from 2 a.m. to 10 a.m. body time. It's also nice to do things without converting time, money, language or electricity.

Janet went with me on the trip and we saw some sights both in London and Rome. I was able for the fifth or sixth straight year to get my annual journal at Harrods. It winds up costing me about $50, but that's a motivator to actually use it. That has become a tradition for me.

I promised Janet that the first thing I would do is buy a cup of Starbucks coffee in NYC, since we arrived at JFK, took a taxi to LaGuardia and then after spending the night at a really nice Hampton Inn (I'm serious, it was great), on to DFW and OKC. All went well until DFW, then the flu hit me and the last thing I wanted was coffee. Those who know me know coffee is a very, very important thing to me, so Janet knew I was sick. That and wearing a coat while shivering.

I went to the doctor yesterday and he confirmed flu, but not sure of "regular" or "swine". He percribed Tamiflu and I'm feeling much better today, so I'm thinking "regular".

Back to the coffee...The gentle bagage handlers managed to break my French press on the way back home. Also, I took several packets of the new Starbucks VIA instant coffee on the trip and it helped. I give the Starbucks people credit for a good instant coffee. It's not that I hate Italian coffee, it just that the intensity is pretty strong - basically expresso.

I plan to return there in June to teach Innovative Software Testing Approaches, one I've taught there before, and Practical Software Test Automation, a new class for the spring I am very excited about.

Well, back to resting some. By the way, I'll be posting soon the prelimiary research on the tester to developer ratios.

Until then, I wish all in the USA a blessed Thanksgiving!

Join Me in Rome for Software Testing Courses in November

2009-10-25T16:46:00.003-05:00

Hi everyone,

For those of you in Europe, or anyone interested in visiting Rome and attending a testing class, here's your chance. On November 16 - 20, I'll be presenting two popular courses:

Testing SOA (Nov. 16 - 17, 2009)

Advanced Software Testing (Nov. 18 - 20, 2009)

Just click on the links above for more information and to register go to www.technologytransfer.eu.

I hope to see you there!

Randy

EuroStar 2009 Software Testing Conference Ticket on eBay!

2009-10-20T23:53:00.004-05:00

Hi everyone,

Here's a great deal, especially for those of you reading this in Europe. Last year, I won a conference pass to this year's EuroStar Testing Conference to be held in Stockholm, Sweden.

I can't use the pass this year, so I have placed it up for auction on eBay. The starting price is $500 and the auction lasts for 10 days. So, if you are looking for a great deal on a great conference, here's your chance!

http://global.ebay.com/EuroStar_2009_Testing_Conference_-_Stockholm_Sweden/190346246440/item

Thanks,

Randy

Major Apple Snow Leopard Defect...Warning

2009-10-13T08:51:00.006-05:00

Hi all,

I'm not one of those Mac users who downplay the problems just to make the Mac look better than a PC! There is a major problem with Snow Leopard that everyone should be aware of. Under Snow Leopard when you log into a guest account, you lose all your data. I think this may have happened to my son. All I know is he lost everything, too. Here is a full account of the problem from Computerworld:

http://www.computerworld.com/s/article/9139250/Snow_Leopard_bug_deletes_all_user_data?source=CTWNLE_nlt_pm_2009-10-12

So...if you have upgraded to Snow Leopard, keep making those backups and don't log in to a guest account! Oh, and to Apple...fix it!!! I can see the new Mac vs. PC commercials now (of course, it would need to be a parody since Apple won't make an ad where their guy just disappears!).

StarWest 2009

2009-10-08T19:22:00.005-05:00

Hi Friends,

I'm returning home from StarWest 2009 and found some time to blog as we are delayed out of Orange Country airport by about 3 hours.

It was a good conference. I was glad to see a higher attendance than at StarEast this year. In fact, the conference had a good feel for the week attendance-wise. It makes me hopeful that the economy is coming back.

There also seemed to be quite a few international attendees and people from the central and eastern parts of the U.S.

My Monday tutorial on Becoming an Influential Test Team leader went well and the two track sessions (one on Cheap and Free Test Tools and the other on Making Your Defects Pay) seemed to go well also. It's always great to meet new people and hopefully provide some helpful information.

It was also great to connect with friends. I really enjoyed Lloyd Roden's keynote on Top Testing Challenges. He and I approach the challenges from different perspectives, mine from a survey basis and Lloyd's from an observation perspective, but I really have a lot to think about from his session. I have been working a lot with test metrics lately and Lloyd's thoughts on making metrics meaningful and accurate is one that we should all take to heart. Everyone talks about test metrics, but few understand the work it takes to define them for a particular organization. I'll blog more about that later.

I also enjoyed Julie Gardner's session on test environments. Very few people tackle that topic in a conference setting and she did a good job on striking a balance of covering the topic without getting too specific.

I heard several people say they had a hard time deciding which sessions to attend. To me, that's the sign of a good program. It seemed that people were picking up lots of good testing tips and techniques.

The vendor expo was pretty well attended by vendors and attendees. Once again, another hopeful sign of an economic recovery.

So, I give the conference high marks. Kudos to SQE for a great conference.

On my free day, Tuesday, Janet and I went to a taping of the Jay Leno Show, but got there a little late. So, instead being part of the studio audience, we got to watch the outdoor segment where Tim Allen raced a Ford electric car. That was fun. We were just a few feet away from Tim and Jay and we were on camera, too. Last year, Jay waved at us at the stop light as he was one his way home driving his Arial Atom hot-rod.

http://www.thejaylenoshow.com/video/episodes/#vid=1164214/?__source=recent-eps-module

Later,

Randy

Mac Snow Leopard and Parallels 3.0

2009-08-31T22:39:00.002-05:00

Hi Folks,

Sorry for the long delay in posting. I've been busy...really, busy.

This past weekend I installed Snow Leopard on my Mac Book and overall, have been happy with it. One problem that I want to warn people about is an incompatibility with Parallels 3.0.

Apple lists incompatibility with Parallels 2.5 and earlier, but when I upgraded and then tried to access Parallels, I got an incompatibility message. Postings at Parallels forum have been ignored on this issue, so before you upgrade to Snow Leopard, you would be wise to upgrade to Parallels 4.0 first.

The good news for me is that I basically abandoned Parallels several months ago due to instability and bugs. I have found VM Ware to be much more stable. That one works just fine under Snow Leopard.

You will likely find other incompatibilities, especially with open source software. So, take those backups.

More to come very soon...

Randy

Where's My Gate?

2009-05-27T21:38:00.005-05:00

Hi Folks,

I travel a lot. I have almost 2 million program miles just on American.

A couple of weeks ago, I experienced a new one. My wife and I were returning from Orlando and StarEast, connecting through DFW on American. (By the way, we had a great week at StarEast. It was good to see everyone.)

We were stranded in DFW on Friday evening due to the flight being delayed from Orlando so we stayed at the Hyatt in Terminal D. On Saturday morning we cleared security in Terminal D and saw that our gate to OKC was A22. So, we hopped on the SkyTrain and went to Terminal A. As we were walking to the gate, I looked at the monitors and saw to my dismay that the flight to OKC was leaving from gate D28. "Crap", I thought (and said, I think). I looked again just to make sure I wasn't looking at arrivals. Nope, it was D28.

So, we went back to Terminal D. As a sanity check, I looked again at the monitors and the flight was listed at gate A22. The information man must have seem my stunned look and asked if he could help. When I asked "Yes, which gate is the flight to Oklahoma City really leaving from?" He said, "Gate A22, just like it says there." I tried explaining that in Terminal A, the monitors said something different. By then, my wife was threatening to file for divorce.

We verified there was no plane at gate D28 heading to OKC, so BACK we went to A22. Finally, at gate A22 (my critical mistake was not actually going to gate A22 the first time), I told the gate agent that the monitor was showing the wrong gate. Her response? "Oh, those are wrong all the time."

Here's my question. Shouldn't the monitors be getting the data from the same source? Second question. If these are wrong "all the time" should someone be looking in to that? Oh well...at least we made it home. I just won't be quite as trusting in the future.

Now for something completely different....

I keep finding these great videos I intend to share and never get around to it. You really need to check these out.

The "Retroincabulator" - This must have been filmed for the Rockwell Christmas party!

http://www.youtube.com/watch?v=pb7OWlVYYRw

If you liked that, check this one out as well: http://www.youtube.com/watch?v=rLDgQg6bq7o

Watch your grammar. Otherwise, you may be visited by the grammar police!

http://www.youtube.com/watch?v=u9_kahA_wQo

Here is amazing one. Three guys playing one guitar!

http://www.youtube.com/watch?v=lrpwDvuNWro

Finally...Do you have ping pong ball skills? These guys do!

http://www.youtube.com/watch?v=LLByTnNwico

OK...now back to work! Have a great week!

Randy

Project Life

2009-04-22T23:07:00.006-05:00

Hi everyone!

I know you may have been wondering if I fell off the face of the earth, got abducted by aliens, or whatever. Well, I have ventured on to a full-time project in San Francisco. This requires that I commute from Oklahoma City each week, which is about 7+ hours each way, including connections. (That is, when I actually make it home without getting stranded someplace!)

Of course, this really constrains my time for blogging and writing. I've also not presented a podcast or webcast for over a month. I hope to get back into that soon.

This has presented some other interesting challenges and changes. For the past 6 or 7 years, I have done about 80% training and 20% consulting. Now, it's 5% training and 80% consulting and 15% travel! I enjoy consulting and getting into the trenches of testing. I'm doing a lot of mentoring, which is gratifying.

I am also experiencing death by meetings. Take today, for example. I had to cancel by participation in one last-minute called four-hour meeting so I could attend attend another more important four-hour meeting (at least they had better food!). I also had to cancel three other meetings because of the two other four-hour meetings. Then, this afternoon, I had back to back meetings from 1 until 6 PM. That's almost 10 hours straight! I look forward to the meetings tomorrow may bring.

OK...enough about that. I'm not bitter, I'm better.

I am posting three pictures I took recently. The first is a rainbow over downtown San Francisco taken from my hotel room on the 30th floor of the Westin.

The next two are also unique. I have found that one of the best places to grab a quick dinner that's really good is in the basement of Macy's across from Union Square. There's a Boudin Bakery (which has some of the best clam chowder and tomato soup I've ever had - and yes, I've been to Boston!), also there's a Wolfgang Puck Express (I love the rosemary chicken there), and other places that are good.

Outside of Macy's nearly every day, you will see people with their cameras and cell phones out taking pictures of this guy. He's got a big dog, a cat that sits on the dog, and a rat that sits on the cat. Now, it's amazing to see them just laying there on each other.

Believe it or not, the other day, I saw him walking his dog, with the cat STANDING balanced on the dog, and the rat resting on the dog. There are many homeless people on the street begging for money. I saw this guy counting his tips the other day and he had a wad of bills. I think that's awesome! He asks for a dollar tip for taking a picture. The way I see it, he's in the entertainment business. I was happy to contribute to a friendly guy with quite a little show going on.

I ought to title this picture, "Can't We All Just Get Along?"

I am also working on two books - Testing Dirty Systems and also one on UAT. I do get some time to write a little in the evenings.

I'm really looking forward to presenting a tutorial, track session and keynote at StarEast in a couple of weeks. Then, I conduct two days of training on use cases and root cause analysis in Kansas City on May 13 and 14. (We still have some seats available - go to www.kcqaa.org for details.)

I also have some other blog posts cued up, so I should have another posting soon.

Thanks for checking in!

Randy

5 IT Roles That Drive Cost Reduction...Plus One.

2009-03-23T09:26:00.003-05:00

I just got this e-mail promoting a slide show from Forrester Research about 5 IT Roles That Drive Cost Reduction. Being an inquiring mind, I was curious as to what they had. I was disappointed but not surprised to see that no relationship between quality and cost was there. For example, no mention of QA or testing, process improvement, etc.

When you consider the billions of dollars ($59.5 billion to be in the ballpark, based on a NIST study), that are spent on software defects, it would make sense that this is one gap to be closed.

This also shows me how far the business case for quality has fallen. Back in the last recession (early nineties), Deming was king and U.S. industry was clammoring for anything about quality. Unfortunately, here in the USA we fell victim to the marketing (TQM) and forgot the discipline involved. So, a few years later we had regressed to the same attitudes that had prevailed earlier.

Here's my point. If you want to cut costs AND increase profits, learn from those defects we all experience. Take those learnings and improve the processes, or at least learn how to catch the defects while they are small and less expensive to fix.

I won't list the other 5, you can read the slide show here:

http://www.cioinsight.com/c/a/Bottom-Line/5-IT-Roles-That-Drive-Cost-Reduction/?kc=CIOMINUTE03232009CIO1

Just don't forget my nomination of #6 - Software Quality

Have a great day!

Randy

New Service Offering - Live Online Training in Software Testing and QA

2009-03-19T14:51:00.003-05:00

Good Thursday everyone!

I just finished speaking at the Oklahoma City Red Earth QA SIG at lunch today. Thanks to everyone that attended. I spoke on the same topic I'll be presenting as a keynote at Star East (May 4 - 8). I hope you can make it to Orlando in May. It will be a great conference!

I've been very busy lately. First, we have a totally redesigned web site - same URL - www.riceconsulting.com. If you haven't seen it, please drop by and let me know what you think.

Part of this effort is a new service offering for me. Sure, web seminars are nothing new. However, I haven't used this medium as a standard training offering - until now.

Because the current economy has forced many companies to cut training budgets, travel budgets, etc. Since 2001, I have been working to deliver online training in effective ways.

So far, we have a great line-up of pre-recorded training courses, which many people have found a great way to balance time, money and content.

However, some people just like live training - and that's fine. The problem is that it's not always feasible to have me live at your location.

Imagine having your team of any size in any number of locations linked together, being taught live! You can ask questions, make comments and work on exercises - just like if I was at your site.

We've been doing this type of training on a limited basis for several years. Now we are offering it to you and your team at very affordable prices. You pick the course and we work together to set the outline and schedule.

The possibilities are many. You can have:

a 3-hour morning session and a 3-hour afternoon session
a 3-hour morning session or a 3-hour afternoon session, conducted for as many days as needed to complete the course. You can train half the day and still work on your projects.
a weekly session of 3-hours on the day of your choice

(the 3-hour sessions have a 15 minute break at the 1.5 hour point)

Example: Your team can take our Basic Training in Software Testing course (a 2-day class) in four 3-hour sessions! All-day Monday and Tuesday, or just in the mornings or afternoons four days out of the week.

Here's the best part - the pricing!

Because of the current economic conditions, I am going all out to help. Each 3-hour training segment is priced at $500! Course books can be printed and shipped to your site for roughly $30 per book in the USA. For international pricing, please contact me for options.

So, you could have your entire team trained live online in a 2-day class (4 sessions) for $2,000 plus books. No travel expenses, even if your team is located in multiple locations. A one-day (2 session) class would be $1,000, plus books.

There are some logistical details that have to be planned, such as getting a training room with projector, computer with Internet access, and having someone you can designate as an on-site facilitator.

If you want to learn more, contact me by e-mail or call 405-691-8075. Let's keep building those skills so you can do more with less!

Today's Webcast on Charting For Root Cause Analysis

2009-03-13T10:18:00.004-05:00

Help me test a new platform for web seminar delivery!

Today, March 13, 2009, 12:00 Noon Eastern Daylight Time I will be conducting a live web tutorial at:

http://webmeeting.dimdim.com:80/portal/JoinForm.action?confKey=rrice

The topic will be Charting Events for Root Cause Analysis. In this session you will learn how to diagram a timeline of events, identify contributing causes, and hopefully, identify the root cause of a problem.

You can download the notes here:

http://www.riceconsulting.com/public_pdf/charting_for_root_cause_analysis.pdf

This is a free session and is limited to 24 people, so please arrive a little early to get a spot. The session will last one hour or less.

Also, please understand that this is also a test. We may experience some problems, so please be patient in that regard. The audio should play over your computer speakers.

I hope you can be there!

To hear this session, click here:
http://recordings.dimdim.com/view/dimdim/da27f404-6136-102c-9991-003048944478

To download the session, click here:

http://recordings.dimdim.com/saveflv/dimdim/da27f404-6136-102c-9991-003048944478

Randy

Getting Started in Root Cause Analysis

2009-03-06T10:16:00.003-06:00

Today's podcast will be on the topic of "Getting Started in Root Cause Analysis". This is a topic that many people are interested in, and another one of the ways you can do "more with less" in software testing and QA.

To listen live toady at 12 Noon, EST, just go to:

http://www.talkshoe.com/tc/27009

You can call in or type in your questions. After the live session, you can visit the same link and listen to the recorded call.

Here are the notes for today's call:

http://www.softwaretestingtrainingonline.com/public_pdf/getting-started-in-root-cause-analysis.pdf

I hope to see you in the podcast today!

Thanks,

Randy

Five Ways to do More With Less in Software Testing

2009-02-04T22:02:00.001-06:00

It seems that many test managers are in the squeeze of tight testing budgets or no budgets at all. That means there is a lot of interest in doing more with less. The economy is bad, but we still have the need to do software testing well.

While I understand the need for efficiency in today's economy, this is not a new message. When I first started training in software testing back in 1990, there was a recession underway. There was also a quality crisis as U.S. manufacturers were battling against higher quality foreign imported items which included everything from VCRs to cars.

We learned how to recoup for the cost or quality with high-quality methods. This is what Phillip Crosby wrote about in his book "Quality is Free". In IT, however, the message of the economy of high quality has been lost over the years. In fact, in recent years I have found that senior IT and business management in many of the companies I have worked with will spend lots of money and take lots of risks with little to show for it.

So, for this window of time (hopefully a small stretch of time), the kinds of things I'll be discussing here have a home in the minds of people. However, the principles are profitable in any economy and we need to remember them.

#1 - Know What You are Doing

The number one value of training and skill building in my opinion is to learn the best and most efficient ways of doing things. After all, someone with no training could attempt just about any job but the results wouldn't be very good in most cases.

In software testing, there is a popular misconception that anyone can test. I have learned that many people have the ability to test, but there are skills that must be developed before someone can do a decent job of testing. And, some people just don't have the mindset or patience to be a good tester.

Training is how you learn what to do at the right times. Repetition is an important part of the training process. So, a three-day class once a year isn't enough to build skills.

Training can be expensive, but doesn't have to be. There are effective options:

* Self-study on the web and by reading books
* Team study such as reading and discussing the same book for a month or more
* Design and conduct your own in-house mini-course
* Have someone share a great new technique or tool at each test team meeting
* Webinars and teleconferences
* E-learning - its better than it used to be!

#2 - Reuse What You Use

This is really basic, but can be a big timesaver. If you have a good test plan or test strategy for one project, make a template and use it as the basis for other projects.

This idea can extend to detailed test design and test automation.

Perhaps the hardest part of reuse is managing the items as they are shared. This is the situation when people share and reuse test cases and test automation. There must be control over the items.

#3 - Define Tests Efficiently

There is an economy of testing. More tests are not necessarily better. In fact, each test case you have is one more you must maintain and perform. If a test case isn't adding specific value to your understanding of the software under test, you should consider why it is in the test.

Many people are surprised that once they start to analyze combinations of tests, how few tests they actually need.

While not every project is a good application of pairwise testing, it is a good technique to explore and try.

#4 - Automate Well

There is a great potential in good test automation. Some have realized this value is actual practice while others still struggle. In my experience, test automation is one of those areas where you can spend a lot of money in a hurry with little to show for it.

There are some free test tools such as FitNesse (www.fitnesse.org) that can be helpful, as well as scripting languages. Other free tools are at www.opensourcetesting.org.

It's good to learn the lessons of automation on the free and less expensive tools before moving on to the more robust and expensive tools.

#5 - Learn From Your Defects

Each defect tells a story. If you take the top 10 defects (as measured by the frequency and impact) in the previous month and perform root cause analysis to learn why the defect occurred, you will make significant strides toward improving your processes. The main investment in this effort is time. It will also take management support to fix the processes. However, these are things you can often do with very little direct cash outlay and they yield big value.

#5.5 - Bonus - Write and Use Checklists

Checklists are easy and inexpensive to create. They help improve processes and prevent mistakes. They also add consistency as to how things are done. If you don't believe me about the value of checklists, ask a pilot.

Summary

Going back in time again, one of the first books I read when starting out as a consultant in the early 90's was Jerry Weinberg's, The Secrets of Consulting. One of the laws is the "Law of Raspberry Jam", which basically says the wider you spread something, the thinner it gets.

Like jam, you can only spread people and tests so far before they lose their effectiveness and value.

We should be lean and mean in our testing because we just don't have the luxury of time (and right now, of money). So it's a good thing to learn efficient ways of testing. Just remember there are limits.

To hear my podcast on this topic, visit http://recordings.talkshoe.com/TC-27009/TS-182201.mp3

Software Testing e-learning

2009-01-31T22:02:00.003-06:00

One of the best ways to stretch your training dollars - whether you are paying for training personally or from corporate accounts - is to consider e-learning.

Not only does e-learning cost less than live public or in-house training, it is a great solution for people who don't have the time to devote 7 or 8 hours (or 2 - 5 days) for training.

Studies have shown that e-learning is just as effective in getting the information across as live training. I think it may be even more effective because you can repeat material as needed. You can't do that in a live class.

It's also a great way to make sure everyone gets the same training no matter where they live.

And...for those training managers concerned about any trainer behavior (jokes, comments, etc.), e-learning is safe. No profanity, no inappropriate remarks, etc.

I have found that the key for effective e-learning is to interact with the instructor. That's why I offer teleconference sessions for my e-learning participants. You get the chance to ask questions and interact with me. Of course, I am always reachable by e-mail and try my best to answer questions by phone.

These are just a few of the benefits of e-learning. If you want to learn more about my e-learning courses (I have 13 of them now, including an ISTQB foundation level course), just visit http://www.riceconsulting.com/training/e-learning.htm.

To experience free demos of any of my courses, just visit http://www.softwaretestingtrainingonline.com/moodle and select the demo section. You can login as a guest.

To buy a course, just visit www.mysoftwaretesting.com. We have a sale on right now!

Riceconsulting.com banned from Google

2009-01-31T19:21:00.003-06:00

Well, the saga continues. Because of the redirects on my site, then my subsequent removal of the malicious links, I have been banned from Google. I'm not upset at Google - I understand their need for maintaining integrity in the search results.

It is frustrating to go from the #1 listed site for "software testing consulting" and the #4 site for "software testing training" to not even being in the search results at all.

So, if you are looking for me on the web, I hope this post helps you to find me.

I have gone through the steps to request re-inclusion and hope to be back in there soon.

Battling the Russian Hackers

2009-01-28T11:06:00.003-06:00

I've been having an interesting time the past few days dealing with a bot attack on my web site. Don't worry - if you are a customer for my e-learning or anything else that required credit card payment. Your information is not kept on my server. Several months ago I got out of the shopping cart business and went with Volusion.com. Volusion is PCI certified and is as secure as it gets.

Also, my Moodle e-learning environment is on a new secure server as well for several months now.

So, here's what happened. I hope this helps someone else defeat these guys.

In early December I noticed that my home page at riceconsulting.com was no longer an html page, but rather a php page. Also, my htaccess file had been changed to point to this new php file. I called my web host and they didn't know how it had been changed. So, I changed my ftp password and changed things back.

Last week, this happened again and I changed it back again to index.html. The next day the home page was index.php again.

After doing some research I discovered that my old version of Moodle still on the site had vulnerabilities which allowed the attackers to place the first Mad Shell script. So, I deleted every php application on my site. I also got rid of some old cgi scripts.

Then, they messed up. That's why I think it was just script kiddies.

They added a new file, named after one of my other pages, but appended with .php. By listing the main directory in my ftp client, I found the recent change.

I looked at the page source and found an encoded script on the page, which I was able to identify as Mad Shell. Now, this is a powerful script. It allows an attacker to do anything an ftp program can do. If you want to know more about Mad Shell, visit this blog.

I deleted that page, changed my htaccess file back to normal and started watching the server logs.

Sure enough, about 10 hours later, another change!

So I started studying the server logs again. This attack was using a redirect to bounce traffic from Yahoo Slurp through my site and on to a site selling drugs that enlarge things.

I was able to identify exactly when the redirects started happening again in the log and found two deeply embedded files - both newly created as php files. So, I deleted them. They had tried to hide them deep in my folder structure, but sorting by modification date helped find the folders.

Finally, that stopped the attack.

Here are my lessons learned (you may see others - if, please comment):

1) Whenever you introduce an application to a web site you are very likely to also introduce vulnerabilities. This is especially true for php apps and even more true for open source php apps.

2) Don't leave unused apps laying around on the site. I'm the kind of person that doesn't like to throw things away. In this case, I should have!

3) Take those security updates seriously. However, in the case of open source, you may not be notified of a vulnerability. It helps to keep an eye out on the support boards for the apps you use.

4) Keep an eye on your server logs and visitor stats. I have known for some time that bots kept hitting my site, but that's just the deal with bots. Every web site owner deals with bots. However, the server logs can be very revealing.

5) Don't expect much help from the web host. While a couple of people at my web host were informative and helpful, one guy told me to "do a Google search" to learn how to secure my files. I was looking for some specific ideas and his response was one of the most unhelpful. Back in December when I was asking support for reasons why my htaccess file had changed, they didn't have a clue. I would expect a tech support person with about one week of experience to suggest that an attack might be occurring. I think I'm ready to change hosting service providers.

6) Stay vigilant and fight back. These attacks can be defended and defeated.

Now, hopefully, it's on to more productive work!

Calling All Dashboards

2009-01-14T09:05:00.003-06:00

In my December newsletter I asked for any of you that have created and/or maintained software testing or software project dashboards to let me know some of the most helpful metrics you track on those dashboards. I have received a few responses, but could use a few more to get a good sense of trends.

The reason I'm asking is because I am working on a keynote presentation for StarEast 2009 on that topic. It would be good to share the most popular dashboard items as described by you, my loyal blog readers, clients, students and friends (hey, some of you may be in all three of those categories!).

In case you are wondering what a testing dashboard is, you can view and listen to my original presentation here: Keeping it Between the Ditches - A Dashboard to Guide Your Testing. It's about 35 minutes long.

By the way, for all those who submit their ideas, I will send a copy of my StarEast presentation.

Also, I hope you can join me and my special guest Fiona Charles on Friday, January 16th at Noon Eastern time as we discuss the new book "The Gift of Time". This book is a tribute to the life of Jerry Weinberg and is a collection of essays by people like James Bach, Michael Bolton, Ester Derby, Johanna Rothman and many others.

I think you will leave this teleconference with wisdom and insights that Jerry Weinberg has imparted to us that can add a new perspective on "why" you do "what" you do in IT.

This Friday (January 16th) at noon Eastern time, you can get in on this call. You can also call in to ask your questions live!

There is no cost for this call, except for the long distance charges if you choose to call in. (You can listen free on the web.) If you can't be there live, it will be recorded for later listening.

Just go to http://www.talkshoe.com/tc/27009. The call in number is (724) 444-7444 Call ID: 26874 The start time is 12:00 EST (Friday) and the call will last no longer than an hour.

Once again, you can listen over the web and ask your questions by text message if you are in another country or just don't want to call in.

Thanks!

Randy

New Year, New Goals

2009-01-06T10:20:00.003-06:00

Happy New Year everyone!

It seems that everyone I've been speaking with over the past week or so is really glad we have 2008 behind us. I am, too, but I'm not very sure 2009 will be better (at least in terms of the economy). I hope it is better, but I like to keep my expectations in line. We have pretty big challenges as a country and world.

I try to avoid making New Year resolutions because they are so easily forgotten. Instead, I try to focus on goals. One year, I made a list "10 things I want to remember" for the coming year. That was interesting to bring to mind throughout the year the important things.

I have some pretty major goals this year:

Finish three books I have in progress
Get about six more e-learning courses produced and out on the website.
Develop some test strategies and content for cloud computing (Thanks, Mike for that suggestion).
Contact at least two people in my network each day, just to stay in touch (so don't be surprised if you get a call from me).
Complete my advanced level test certification (one part down - two to go!)
Actually publish my newsletter every month this year!
I'm also working on this major project to document all of the processes used in my office - all the way from accounting to website maintenance.

Then, there are my personal goals: books I want to read, people I want to develop deeper relationships with, a better use of my time, and then the big one: to organize my office!

I also have this car restoration project (a 1949 Plymouth) I would like to finish while my dad is still alive to see it and ride in it.

One more thing - My goal is to journal each day. I have been hit and miss, but at least have been doing it for a few years. Back in November while in London I was able to stop by Harrods and get their 2009 Diary, which I find perfect for journaling. Plus, it's expensive enough to give me the incentive to actually use it.

I learned a great tip on this from my mentor, Jim Rohn. Mr. Rohn says that it's good to have a notebook with loose leaf pages for all my projects. Each project gets a tab. Throughout the year I will make notes about how each project is going. On the journals, the one time I spoke face to face with Mr. Rohn he told me that if I stay consistent, one day I'll have an entire shelf of journals to document for my kids and grandkids my ideas, experiences, pictures and thoughts. Today, I look at my shelf and I have about ten of those books. My goal is one book per year.

I hope this prompts you to make a short list of things you would like to do, be, or experience this year. It's easy to dismiss goals, like resolutions. But they really do propel us forward and give a chance to review at the end of the year the progress we have made. For me, if it's not a goal I probably will get distracted and not do it.

Your goals might be:

To learn a new skill
To visit a new place
To make a new friend
To be better at what you do

Just remember, it's not the economy that determines our success or failure - it's your outlook, faith and philosophy. There are always people that do well in bad times as well as good.

I hope you comment on this post and share some of your goals and thoughts about the New Year!

Blended e-Learning Gets Higher Grades

2008-12-30T22:10:00.004-06:00

As many of you know, I've been working on making e-Learning an effective and attractive method for skill building in software testing. The number one thing I have learned is that there must be interaction with the instructor to make the learning effective. This interaction can take many forms - e-mail, phone, chat sessions, teleconferences, etc. That's why in my e-Learning courses I include live call-in teleconferences at talkshoe.com. My goal is to do 90% of my training this way!

One of the tools I use for creating e-Learning content is Articulate. It creates Flash presentations from PowerPoint and has a nice interface with many good features such as bookmarking, notes, attachments, etc. I ran across an interesting post on the Articulate blog recently about how one college professor compared the grades of two similar classes - one using traditional classroom teaching only and the other using blended learning (e-Learning with live reinforcement).

From the report: "A technical report from a University of Houston Department of Health and Human Performance researcher finds that students in a hybrid class that incorporated instructional technology with in-class lectures scored a letter grade higher on average than their counterparts who took the same class in a more traditional format."

You can read more here:

http://www.articulate.com/blog/use-articulate-get-better-grades/

So, why is this important to testers and managers? I think the way we build skills is as important as the skills we build. Training and skill building can be transforming and exciting, or it can be torture.

Some people look forward to a day of training (for more reasons than just getting away from work) while others dread it. When people get excited about learning, amazing things can happen. The link to the Articulate blog is interesting because it goes "behind the scenes" in the theory and practice of e-Learning.

As testers and managers, we must learn about learning. We need to understand the different learning styles, attention spans and what works in training adult learners. My observation is that it's the people that don't care about how to make training effective that are the worst trainers.

By the way, as my way of saying "thanks" for reading this post, here is a link to one of my conference presentations (The Risks of Risk-Based Testing) created using Articulate:

http://www.riceconsulting.com/articulate/risks_of_risk_based_testing/player.html

Best regards,

Randy

COTS Testing Course is Now Online

2008-12-29T18:48:00.003-06:00

I'm excited to announce the launch of my 14th e-learning course, Testing Commercial Off-the-shelf (COTS) Applications. This online course gives you the approaches you need to test commercial software. Many organizations, especially government, have been building systems comprised of commercial software. However, problems arise when people try to integrate the products to work together. Many people work under the misconception that all you have to do is buy the software, install it, maybe configure a few things and then start using it. Unfortunately, the truth is that there are a lot of things the vendors can't test. This course will lead you through a process of testing COTS products all the way from selection through maintenance.

To see a free demo of this course, just visit http://softwaretestingtrainingonline.com/moodle/course/view.php?id=45. To buy a registration, go to https://www.mysoftwaretesting.com/ProductDetails.asp?ProductCode=COTS101.

To see all the details about my e-learning courses and how they can help you build your skills on your own schedule, visit http://www.riceconsulting.com/training/e-learning.htm.

Thanks!

Merry Christmas!

2008-12-24T16:08:00.002-06:00

Hi friends - Just a note to wish you all a very Merry and Blessed Christmas. To all my Jewish friends I wish a Happy Chanukah.

At this special time of year I will be thinking of all my friends around the world and will be enjoying time with family here in Oklahoma.

Regardless of your beliefs, my wish is that the Prince of Peace will be real to all of us this year.

Warm regards,

Randy

A Good Resource for Government Computing Projects

2008-12-23T10:56:00.002-06:00

I am working on a new book project about software failures and I came across this resource from the Center for Technology in Government at the University at Albany especially aimed at government computing projects. The name of the report is Making Smart IT Choices - Understanding Value and Risk in Government IT Investments. You can get this at http://www.ctg.albany.edu/publications/guides/smartit2. In fact, you can download the entire book as a pdf file.

I like this book because it hits at the heart of IT projects - making the right choices based on risk and value. It's all common sense stuff tailored for the government sector. It proves that the answers are in plain sight. Now, the tough part is to actually act on them.

My experience is that the hard part is dealing with the stakeholders who want something different every day and the vendors who are more than happy to promise it to get more money by stringing the project out almost forever. We as taxpayers end up paying the enormous bill for this nonsense.

Doing the research for this even makes me sick, but I am glad there are those out there at least shining a light in the darkness of many IT projects.

Book Review - Clean Code by Robert C. Martin

2008-12-16T11:31:00.002-06:00

"Clean Code" by Robert C. Martin
Published by Prentice Hall
Buy at Amazon.com -
http://www.amazon.com/gp/product/0132350882?ie=UTF8&tag=randyricessof-20

(5 stars out of 5)

I knew I was going to like this book after I read the Foreword and
Introduction. I know not many people read those and as an author
myself this bothers me, but it gives insight to why the author has
written the book, which is a very important thing to understand. I
was hooked by the affirmation that details are important and that
the only meaningful metric is the number of "wtfs" per minute in a
code review. For a long time I have kept my "profanities per test
session" metric as a way to measure software usability.

Long before I was a software testing guy, I was a coder. Back in
those days it was common for rookie coders to sit next to guys with
gray hair and bad breath (or if lucky next to ladies with better
appearance and who smelled better) to learn the craft of writing
code. Yes, coding was a craft before the attempt to make it an
engineering discipline. Although there was a lot of bad code, there
were plenty of people who took pride in how they coded. This book
reminds me of what it was like to sit next to these intelligent
people as mentors.

I have often lamented that today's young coders don't have this
mentor/apprentice relationship. A reasonable facsimile would be to
read this book and practice these very sensible and helpful lessons
in coding. As a testing guy who has been through those 30 wtf/hour
code reviews, my bulletin to developers is that: Your code can be
better! This book can show you how!

Here's one example: Have meaningful names. Instead of a function
named "int d", how about "int ElapsedTimeInDays;"? There's a whole
chapter about naming things. It reminds me of the coder I knew that
liked to label COBOL paragraphs after towns and cities in Kansas.
Why? So he could write GOTOs that read "GOTO Topeka." He thought
that was cute but the rest of us found it confusing and not-so-cute.

I also appreciated the chapter on Unit Testing. As a tester, I love
it when testing is mentioned, but Martin actually has specific
guidance on what makes for good Test Driven Development and Unit
Testing.

This book is very readable, makes a lot of sense and would make
most coders better. For testers that sit in on code reviews, this
would be a great read for understanding how good code should be. I
wish every coder in college or tech school would read this book
before unleashing their creations upon the world.

Skills for Software Testers in the Future

2008-12-10T12:58:00.002-06:00

Last month I delivered a keynote address at EuroStar on "Trends that May Impact the Future of Software Testing." There were some interesting questions following my talk, one of which I would like to discuss in this post. "What do you see as essential skills for testers in the future?" Great question.

First, let me say that the list could be long and your list probably will differ from mine and that's fine. In fact, please comment with your answers to this important question.

Skill #1 -Thinking Skills

Too many testers fail to think through what they are doing and why they are doing it. They also fail to use imagination and instead rely on a repository of knowledge somewhere that is probably outdated and incorrect. Sure, we can curse the darkness of bad documentation, but we've been doing that for over 30 years already so let's get over it. Let's light a few candles to illuminate the subject of hidden understanding.

So, for example, when doing a boundary test, ask, "Is this test meaningful?", "What will it prove or disprove?", "Are there more important tests that I haven't done yet?".

As testers we have to be very careful about falling into ruts and thinking the same way all the time. Learn to look at problems from different angles.

I like to play thinking games just to stay in practice. One game that is helpful for test teams is "20 questions for software". I improvised this game in a class on exploratory testing when only I had my computer at the front of the room. Here's how it works. The leader (perhaps you) are at the front of the room with an application projected on a screen. Then, go around the room in order (a u-shaped room or circle is best), with one person starting the questioning of the application. They should ask a question such as "What happends when you...(fill in the blank)?". Then, the person next to them asks a question based on their question. This continues around the room, maybe even multiple times. There are only two rules: 1) you must wait your turn, and 2) your question must be based on someone else's question or answer before you.

You may be surprised about how much you can learn about the application by doing this. You may also be surprised (perhaps dismayed) about the quality of questions. This gives the team leader insight into where to develop the thinking skills of the test team or individuals on the team.

I also like to work Sudoku puzzles because they require deducing, eliminating possible solutions to find the right number(s), and using other thought processes that are the same as used in exploratory testing. I've never tried to work one puzzle together as a group, but that would be an interesting experiment.

Skill #2 - Communication Skills

I believe that communication is the basis for everything else we do in IT. Without good communication, we can't even define the problem to be solved, much less define the solutions to the problem or the testing of the solutions. Ironically, communication is the last thing we take training for, or practice, or apply in the workplace.

Testers especially need to be good communicators in the following ways:

Accuracy - People must be able to trust the accuracy of test results and other information from testing. Once the accuracy slips, so does the credibility.

Timeliness - Timely communication helps people to form good responses to problems, take advantage of opportunities, and generally work together better. Sitting on bad news, hoping for it to become better news, is not a good approach.

Thoroughness - Telling only part of a story is as bad as giving inaccurate information.

Courtesy - How people say things is just as important as what they say.

Helpfulness - Overall, the information a tester provides should help the rest of the project team to fix issues, make improvements and deliver a great project on or ahead of schdeule.

Meaningfulness - The information must have value to the audience, otherwise people will disregard it as "just another report" that may or may not be read.

Skill #3 - Measurement Skills

When you think about it, testing is measurement. However, we as testers often struggle with what to measure, how detailed the measures should be, what to do with the measurements, etc.

It takes skill to know what to measure without going overboard. It also takes skill to make the correct interpretation from the measurements and metrics. But how does one build these skills? I'm going to suggest a radical approach. Don't start by reading books on measurement, unless they are basic.

Like many things, including testing, we are up against Parkinson's Law that says "work expands to fill available time." My variation on this is that simple things can become complex just by writing books, having conferences, etc. As an example, function points started as a fairly simple idea but then evolved into more complex ways to count and apply them. So when you read a book on measurement your first thought may be that you need a large set of measurements to "do it right."

Instead, simply look at what needs to be improved and find two or three things to measure that could show progress in meeting those goals. You can also consider a typical project and what you need to know to stay on track. This is the dashboard concept. Have five or six indicators at the high end.

Then, find ways to measure without being intrusive. This in itself can be a big challenge. That's why simple is good.

Skill #4 - Configuration Management Skills

CM has always been an essential skill for testers. After all, what good is it if you test the wrong version or configuration of the software?

With virtual environments, it is even more important to know how to manage them. At this point, you may be thinking, "Yes, but the tool has features for that." Consider though, that a tool is not a process.

So testers need to know how to control all the items in the environment - software, hardware, data, and any other system components.

Skill #5 - Technical Skills

This is an interesting debate. How technical does a tester need to be anyway? On one hand, it is the lack of technical perspective that gives testers the same view as users. Too much technical knowledge and they may apply workarounds that users would not think of, therefore not seeing problems from a true user perspective.

The term "Technical Skills" is a fuzzy one, so I'll clarify my meaning a bit. The technical skills I refer to here are those that allow testers to use new tools and approaches effectively. I don't believe that testers need to be coders, although sometimes that skill comes in handy. But, if a tester likes to code and they can maintain objectivity, it does open some options.

I believe it is a mistake to eliminate a tester from hiring consideration just because they can't code or work with a particular tool. There are simply too many other needed skills and attributes to limit the decision to technical skills alone.

It's like being in a swimming pool and being able to go in the deep end as opposed to just staying in the shallow end. While we can all be in the pool and have fun. Some will be able to do more advanced things, like diving and snorkeling. It's not about worth or value, but about the ability to experience different things. So, if an inexperienced swimmer gets thrown into the deep end, it's "sink or swim", and that can sometimes be a dangerous thing. You may have experienced that "over my head" feeling.

Another example is owning and driving a car. Some people are happy just driving the car and keeping it filled with gas and having the fluids changed on schedule. Others want to get more involved in their vehicle's maintenance and repair by doing things such as changing the oil, replacing air filters and spark plugs, replacing fan belts, hoses and brake pads, etc. A few braver souls with older cars get into major engine repair or replacement. All of these types of car owners can drive the car well. Some can perform maintenance but due to time issues prefer to have the work done for them. All are valid owners and I would rather be on the road with good drivers who know little about auto maintenance than bad drivers who can replace engines in a day.

At the end of the day, the answer to this question of should a tester have technical skills may be a firm "it depends!" It depends on the person and their abilities, the projects they work on and the types of testing their job requires them to perform.

Skill #6 - Business Skills

Testers need to be able to speak the language of business. How can we as testers expect to be invited into business-level meetings if we don't understand the culture and language of business? This is more than just understanding business processes. We must know what's important to the business - profits, market share, strategic goals, markets, growth, legal, return on investment, and trade issues.

So, these are some of the important skills as I see them. Agree or disagree? I would like to hear your comments and ideas!

Leadership by the Book

2008-12-04T19:56:00.002-06:00

I ran across this online article in Federal Computing Week. It's an interview with Dave Wennergren, deputy chief information officer of the Defense Department. It struck a chord with me because I'm such a believer in reading as a form of personal development and skill building.

It's not how much you read, but what you read and how you apply it. You could read a book in one day and get nothing from it, or you could read a page a day and get a nugget of wisdom each day.

Unfortunately, too few IT professionals read books or articles about our profession. Testers are no exception. When I meet people in IT or not, I can normally tell within 10 minutes if they read very much.

For the price and payback, books are a great value. A $50 book can well contain thousands of dollars worth of valuable ideas and advice.

When faced with a problem, I usually ask myself, "Who can I ask?", or "What can I read?" to get a solution.

Here's a great quote from this article:

"When I was the Department of the Navy CIO, I started a leadership forum called Expanding Boundaries, which I have now continued in my DOD job as Expanding Horizons. The idea is to get the leadership team together once every couple of months, read a good book and then discuss how we could apply the book, to our work and our lives. It’s been a great way to understand the art of the possible and best practices, but even more importantly, it has helped to align the leadership team, improve trust and serve an idea creation engine as we strategize about next steps in our information transformation work."

One challenge that teams have is to communicate together well. This would be a good venue to help break that ice.

So, here's the assignment. Read this article, then let me know your thoughts.

Thanks,

Randy

The Sports Secret to Building Software Testing Skills

2008-12-01T20:27:00.002-06:00

Every once in a while, I'll run across an airline magazine article that really catches my attention. Yesterday on my way to Harrisburg on Continental, I read this article about how collegiate and professional sports stars improve their skills. I couldn't help but think of all the parallels to many of the concepts I teach in my Becoming an Influential Test Team Leader tutorial. Namely:

1) To improve a skill, you must focus on it like a laser beam. Improvement often occurs a little bit at a time (painfully slow at times), but then sometimes you have a breakthrough moment.

2) You must be willing to move out of your comfort zone to improve. Don't just work on your strengths, but also your weaknesses. Actually, I teach becoming exceptional in your strong areas and not to worry too much about the weak points. While I still believe in this philosophy, I also believe that we can all work on our weaknesses to become more well-rounded.

3) A good coach is willing to place a player in a position to fail so they will learn and improve. The player must also have the mindset that failing is an essential part of improvement.

Here's a quote I liked in the article from Steve Bzomowski, a former NBA scout and founder of Never Too Late Basketball Camps: “Pros take themselves seriously. And even though recreational players aren’t going to become pros, there’s no reason for them not to take themselves seriously. We’re always preaching, ‘Respect the game.’ So when you don’t run the court, you’re not respecting the game. Play the game right.”

So, testers, take heart. Identify your strengths and weaknesses. Practice on both, remembering it's OK to try things and fail - if you learn and improve.

Wrapping Up in Rome

2008-11-21T15:42:00.004-06:00

It's been a busy week for me here in Rome as I taught two public classes - Software Testing and Advanced Software Testing. Thanks to all the participants that made it a great experience. I look forward to a return trip in June.

In the Advanced class, we got into test efficiency techniques and how to design tests that are creative and challenging. We also discussed test metrics and how to use them to improve development and testing.

It's hard for me to remember to slow down my rate of speech when speaking through an interpreter. Italian requires many more words than the English equivalent. And forget humor. It just gets lost in translation. I really do appreciate my two interpreters and the other support during the class. You guys are the best!

Sunday was my only sightseeing day. I've been here twice before, last November and last June, so I'm getting to know my way around pretty well. My next goal is to learn more than six words in Italian!

I had not been in the Castle St. Angelo yet, so I decided to check it out. I'm glad I did. The views are spectacular. If you explore around you can go all the way to the top, which allows a full view of central Rome. It's a great value for just 5 Euros. Muphy's law was at work, though. I was taking pictures throughout the castle but when I got this wonderful view - dead battery in the camera.

Fortunately, my hotel is close by so I went back and recharged for an hour, then returned to the castle. They let me back in and I took the pictures.

Here's a travel tip: Always take a spare ATM card. In Europe, the ATMs require fully inserting your card. I like the swipe option because I don't temporarily give up my card. Sunday I was trying to get some cash from an ATM and for some reason, it kept my card. Had I not been teaching on Monday, I could have gone to the bank and retrieved it, but since the banks close at noon and at 4 P.M., I had no options except to report it lost and use my spare at a different bank closer to my hotel.

Tomorrow I head home but have an overnight layover in London. It has been a great trip, but I am ready to be home and celebrate Thanksgiving!

Travel Tips Around the Hague (World Forum and Museum Area)

2008-11-16T12:02:00.012-06:00

As a traveler, I always like to pick up and share helpful tips when possible. I like to be a traveler instead of a tourist. As a business-oriented traveler, this can be challenging. For example, instead of paying $30 for breakfast at the hotel, I like to experience local color and less expense.

So while I was staying at the Golden Tulip by the World Forum in The Hague (which I thought was a great place to stay), here are some little finds:

Right around the corner and down the block there's a little coffee shop. It's right on the #17 tram line after the museum stop. That's where locals gather in the morning and have coffee and breakfast. The owner fixed me a great breakfast of fried eggs, bacon and real coffee (brewed - not the Americano-style) for 3.20 Eur. It was great! I ate there three or four times in all. If you explore around, you will find a shortcut trail that takes you from the coffee shop to the World Forum.

Find #2

I am on a two-week trip, so I must do laundry, or have it done for me. Hotels charge an arm and a leg for this, so I scouted out this self-serve laundry just a few blocks from my hotel in the shopping district, which is only 3 - 4 blocks from the hotel. I was able to do two small loads of laundry for 12.50 Eur. While I was doing laundry, I browsed in the many stores in the shopping area. If you have trouble finding it, just ask the people in the coffee shop!

By the way, there is also another laundry just down the street.

Find #3

The shopping area just blocks from the hotel has almost anything you might need on a trip: clothing, groceries (there is a large store there by European standards), banks (ATMs), drug store, carryout pizza, restaurants, hardware, a copy shop, bookstores, amazing bakeries, chocolate and cheese stores, and more. Just watch out for the chicken wings - they didn't agree with me.

I hope these make your trip to the World Forum in The Hague a little easier and more interesting!

Thoughts About EuroStar 2008

2008-11-16T11:29:00.005-06:00

I thought EuroStar 2008 was a great conference. The tutorials, keynotes and tracks were strong in my opinion (I must also say I want to give my perspective as an attendee as well as a speaker. It's hard sometimes to step back and be objective, but I'll try.).

One of my tests of a good conference is whether I'm torn between attending track sessions - and I was. I also spent a fair amount of my time in the interactive sessions, such as the ones on the testing manifesto and testing standards. It was nice to be working on things to advance our testing profession. I'm not sure where the whole testing manifesto thing will wind up. I hope it is someplace similar to the agile manifesto - something simple, meaningful and easy to read and remember.

James Whittaker was his insightful and entertaining self (that video he shows about the future of medical software is amazing and shows we're going to need better testing). His talk was "The End of Testing as we Know it", and I felt all the keynote speakers made me think. James Lyndsay spoke on "Becoming Agile - Reshaping Testing for an Agile Team." He had great points and I'm always impressed by the simplicity and impact of his slides!

The expo was amazing. In the U.S., the exhibits tend to be rather bland. In this show there were two coffee bars, a cocktail bar (I had a smoothie), a chess match, a bicycling competition, etc.

They had a contest where if you got all the exhibitors to stamp your card, you could enter the drawing. So, I computed my odds (about 1 in 30, I figured) and went for it. The grand prizes were two free passes for next year's conference and eight Nintendo Wiis. All I wanted was a Wii. Guess what? I won a conference pass. I didn't factor that in my odds! Oh well, I don't have the luggage space or weight anyway.

The gala was also fantatstic. It was held in the Grote Kerk, which is a former cathedral. It was big night for me, winning the best tutorial award. Thanks so much to the 60 or so people in my "Becoming an Influential Test Team Leader" tutorial that made that happen. It was totally unexpected! Much more important than the award was that we experienced and learned things together that will help us be better in our leadership roles.

My prize was a beautiful 9 pound Waterford crystal vase which I will find a great place for at home. Thankfully, security in the U.K. has relaxed security rules to allow two carry-on pieces of luggage!

One other thought concerns model-based testing and how it differs in Europe than in the U.S. - At U.S. conferences, model-based testing is often presented as a manual process. In Europe, they talk about the tools that can make model-based test design happen faster and more completely than using manual methods. I just thought that was interesting.

Dot Graham will be the chair for next year's conference in Stockholm, Sweden. I wish her all the best!

I would like to thank all the kind people who helped me feel right at home. Many thanks to the Qualtech Conference team - Tracy, Siobhan, Lorraine and others. Also, thanks for the conference committee for selecting my talks and to John Fodeh for my keynote introduction and to Bob van de Burgt as conference chair. You all did a great job!

Greetings from The Hague and EuroStar 2008

2008-11-11T04:51:00.003-06:00

I've been pretty busy since StarWest, so I haven't blogged any since then. Hopefully, I'll make up for that on this trip to The Hague and Rome. I plan to blog on three topics the next couple of weeks: 1) Highlights from EuroStar, 2) My thoughts on the economy and software testing, and 3) Travel tips for The Hague and Rome - complete with pictures.

The flight over went well - as good as it gets in coach on American Airlines. In the airport in Oklahoma City I discovered I forgot my power supply for my MacBook (Darn - I didn't use my checklist!). The Apple factor introduced a complexity factor because the airport shops assume nobody owns a Mac! So, using my iPhone in Dallas, I saw there was a PC World store in Heathrow which carried Apple accessories.

I had a five-hour layover at Heathrow in the new Terminal 5. It is an amazing place. And yes, I found a power supply for my MacBook at a great price. Even with the exchange rate, I was happy to pay it!

The British Airways club there is awesome. Man, the American Airlines' Admiral Club could take some lessons from them. They have all sorts of food and drink at no charge (of course there is the elite status required), plus wi-fi included, and one of the best things after a long flight are the individual wash rooms. This all made the long layover not so bad.

I was surprised that BA had food service on my 45 minute flight to Amsterdam. We don't even get pretzels anymore on US airlines!

Luggage is often a challenge when travelling solo overseas. I took the train down to The Hague. I had a large suitcase plus a duffle bag for my computer and other carry-on essentials. I had two people (other travellers) that helped me with my bags on and off the train. I was impressed by their hospitality!

The room here at the Golden Tulip hotel by the World Forum is great. A nice bed with a U.S. sized room and weekly rate wi-fi is awesome. I only can use 3 TV channels - CNN and 2 BBC channels, but at least I keep up with news back home.

Yesterday I presented a full-day tutorial on "Becoming an Influential Test Team Leader", which seemed to go well. I haven't seen the evaluation results yet, but we had about 50 people and it was a great time together. By the end of the day, we were all good friends!

The biggest challenges are getting adjusted to the time change (I started my session at 2 a.m. CST and ended at 10 a.m. CST!) and adjusting to the cultural differences in my remarks.

This afternoon the conference starts. James Whittaker from Microsoft speaks. Later, the guys from Google (Bharat Mediratta and Antoine Picard) give their "Testing on the Toilet" talk. That has been a big hit in the U.S.

Tommorow morning I deliver my keynote address on "Trends that May Shape the Future of Software Testing". I'm making some last-minute tweaks to reflect some recent events.

Better go now, but I'll post again soon with some impressions from the conference and some pictures.

StarWest 2008 - Part 2

2008-10-04T14:06:00.005-05:00

OK, so I had good intentions about blogging during StarWest. I was too busy!

I thought the conference was good. The keynotes seemed to hit the right level of interest with people. Rob Sabourin spoke about "Testing Lessons from Springfield—Home of the Simpsons". That was a light note to start on and had some interesting application for testers. You've got to love it when someone in the audience shouts, "Eat my shorts, man!" in the middle of a keynote presentation.

Jon Bach spoke next on "Telling Your Exploratory Testing Story". Jon used the connection between journalism and testing to show how to explain to people the performance and outcome of a test.

I sat in on two of Julie Gardiner's sessions: "Five Things Every Tester Must Do" (track session) and "Branch Out Using Classification Trees" (keynote). Julie did great in both sessions. It was great to see people really get excited about the classification tree technique and tool to help optimize testing.

I wasn't able to make it to Julian Harty's session on "Six Thinking Hats for Testers" but I have heard it before and it's a great presentation.

The best I could tell, most people were having a good time and picking up good ideas. I enjoyed meeting a lot of new people, catching up with friends, and learning new things.

On one of our excursions during the week, Janet and I got to see a taping of the Tonight Show. Then, after the show we saw Jay Leno leaving driving his Ariel Atom. He smiled and waved at us. You can't beat that!