StarEast 2011 - Free and Cheap Test Tools

Thanks to everyone who attended my half-day tutorial today! Here are the tools that were mentioned by people in the session, plus a few more from me:

Axiom - Requirements Management tool - http://www.iconcur-software.com/solutions.html

Dia (Visio alternative, helpful for drawing process flow diagrams, etc for test planning) - http://live.gnome.org/Dia

Gliffy (a web-based diagramming tool) http://www.gliffy.com/

Shapes - Shapes is a simple, elegant Diagramming app for Mac OS X Snow Leopard. http://shapesapp.com/

FreeMind (Mindmapping, useful for test design) - http://freemind.sourceforge.net/wiki/index.php/Main_Page

Runtestrun.com - Test case management

Sikuli - Sikuli is a visual technology to automate and test graphical user interfaces (GUI) using images (screenshots) - http://sikuli.org/

IcuTest - GUI Unit Testing for WPF - http://www.nxs-7.com/icu/

Selenium Grid - Selenium Grid transparently distribute your tests on multiple machines so that you can run your tests in parallel, cutting down the time required for running in-browser test suites. http://selenium-grid.seleniumhq.org/

eggPlant - Image based test automation - http://www.testplant.com/

TestRail - Test Management - http://www.gurock.com/testrail/

InCtrl - By monitoring the changes made to your system when you install new software it enables you to troubleshoot any unexpected problems that come up. http://www.pcmag.com/article2/0,2817,25126,00.asp

Paloma Print Perfect - STREAMDiff - Comparison of PDF docs - https://www.palomaprintproducts.com/STREAMdiff/printPerfect.asp

Skitch - Screen capture, crop, resize, sketch (Mac) - http://skitch.com/

TimeSnapper - Take screenshots automatically in background as you test - http://www.timesnapper.com/

VM Ware and MS Visual Studio 2011 Test have record features.

Snagit - Screen capture - http://www.techsmith.com/

BrowserMob - Cloud-based performance testing and monitoring - http://browsermob.com/performance-testing

WebTrends - web monitoring and analytics - http://www.webtrends.com/

Presently.com - allows individuals to post short, frequent updates that are tracked or "followed" by others. Unlike Twitter, Present.ly provides a secure and private way to share updates among members of a company, without them being visible to the outside world.

Sharepoint - www.microsoft.com

Skype - www.skype.com

Fluid - lets you create a Real Mac App (or "Fluid App") out of any website or web application, effectively turning your favorite web apps into OS X desktop apps. http://fluidapp.com/

Fake - Fake is a new browser for Mac OS X that makes web automation simple. Fake allows you to drag discrete browser Actions into a graphical Workflow that can be run again and again without human interaction. The Fake Workflows you create can be saved, reopened, and shared. www.fakeapp.com

TeamViewer - You can remote control your partner's PC as if you were sitting right in front of it. http://www.teamviewer.com/en/index.aspx

Logmein.com - Remote PC access. Free for up to 2 computers.

Joinme.com - Screen sharing

IE Tester - IETester is a free WebBrowser that allows you to have the rendering and javascript engines of IE10 preview, IE9, IE8, IE7 IE 6 and IE5.5 on Windows 7, Vista and XP, as well as the installed IE in the same process. http://www.my-debugbar.com/wiki/IETester/HomePage

Thanks for all your input!

Randy

Minnesota Taxpayers to Foot a $7.25 Million Bill for a Failed Software Project

For many years, when software projects failed, the vendors almost always had to pay up big time. Well...it looks like the taxpayers in Minnesota are getting stuck with a $7.25 million bill on this one. Why? Basically, the judgment was that the state also contributed to the failed project. Get this..."The payment will bring HealthMatch's total cost to more than $41 million, a department spokeswoman said. The original budget was about $13 million"

You can do a LOT of process improvement and optimization for $41! However, that's all seen as "QA stuff".

I really feel for the people of Minnesota on this one. But, I've seen this happen far too many times. The problem is that software projects are not performed in a vacuum. A vendor goes into an organization to work "with" the people, not "for" the people. So, if the organization is dysfunctional, so will be the project. The reality in this case (as I understand it) is that there were shortcomings on both sides. However, what often happens is that both sides get sucked into a death spiral. Neither can afford to leave, so they keep thrashing until the money runs out and/or someone decides to pull the plug.

Read more at Computerworld.com.

Abbott & Costello Meet Microsoft Outlook

I found this on Computerworld's Shark Tank and thought about the old "Who's on First?" bit. It's also a great example of trying to understand the users' true needs and expectations of software.

"He (Big Boss) wanted me to help him understand how Microsoft Outlook calendar appointments work with changing time zones," says fish. "I updated the online docs, then walked down the hall for the explanation to the big boss."

But as soon as the big guy hears the explanation -- how Outlook will automatically reflect the time-zone shift when the clock is changed -- he tells fish, "No, that's not what I need. I need to be able to set an appointment up for a certain time slot in the other time zone for next week, but then appear in the same time slot when I look at it with my PC set to my current time zone, this week.

"If I set it for 2 p.m. there, then it needs to show up as 2 p.m. on my calendar when I look at it from here, but for next week.

"And I want everyone else invited to the meeting to see it correctly when viewing both here and in the other time zone. And, oh yeah, it needs to show the correct time slot on my smartphone calendar, and on all the attendees' smartphones, next week when we're traveling."

Fish scratches his head and then attempts to explain that an appointment set in the other time zone will always appear in a different hour time-slot when viewed from a device set in the current time zone, and that it all depends on the time and time zone set on the device.

Big boss says, "No, you're not getting what I need. I need to be able to set an appointment up for a certain time slot in the other time zone for next week, but then appear in the same time slot when I look at it with my PC set to my current time zone. I can't believe that there isn't a way built into the software to do this!"

Reports fish, "I tried to explain it one more time, but in the end the big boss looked back with glazed-over eyes and said, 'Well, since the technology can't help me, I'm just going to print out my calendar and carry it with me to the other time zone.'

"Then he proceeded to print out next week's calendar on paper."

Book Review - "Glitch - The Hidden Impact of Faulty Software"

I have a great interest in the topic of software defects, their impact and how software failures happen. Books in the past such as "The Day the Phones Stopped", "Bad Software" and "Fatal Defect" have been great sources of documenting what has gone wrong due to software defects. These books serve as case studies for anyone who is involved in creating, testing or buying software products.

So, I was excited to see that a new book has been written on the topic. I'm not crazy about the title of "Glitch" because it tends to convey that a problem is minor. We see this a lot in the popular media as major system failures are described as glitches. I think the problems that Papows describe in this book are more than minor ones. Setting the title aside, I was happy to see a very recent accounting of computer failures in many domains - financial, government, medical, etc. Of course, I'm not happy for the problems, but it's good to see recent information.

This book is written at a level that non-technical people can understand and technical people can read without feeling insulted. That's a nice balance. The information is also sourced well. When it comes to reporting about failures, there are ten sides to every story. But having some objective sources is helpful.

To me, the strongest chapter was Chapter 8, "The Way Forward." In this chapter, Papows (who was the former President and CEO of Lotus Development) describes what organizations and individuals can do to help reduce the risk of computing failures. This is the only place QA and testing are mentioned in the book, which was a little disappointing to me, but I come from a biased perspective. I think people need to understand the project management reality of rushed deadlines, skimpy testing and lax attitudes toward quality to truly understand why software fails. This book gets into that to some degree, but not very deeply.

In fact, much of the book is written from a top-down perspective as opposed to an "in the trenches" perspective. I was hoping for a more detailed analysis of why some of the problems happened.

I can recommend this book to business managers to raise awareness of software risks and to software quality professionals who need good examples of past failures to make the case for better software development and testing processes. However, it may disappoint those looking for root cause analysis of software defects.

Six Seats Left for Test Automation and Test Team Leadership Training in Salt Lake City - Jan 25 - 27, 2011

I just wanted to get the word out to anyone that is in the Salt Lake City area, or who may be able to travel there, that I have six seats open for two great workshops:

Practical Software Test Automation (Tues & Wed, Jan 25 & 26, 2011) - Learn hands-on techniques and approaches for building an efficient test automation architecture. We get into scripting and test tool integration. Bring your notebook computer!

Becoming an Influential Test Team Leader (Thursday, Jan 27, 2011) - This is an interactive workshop that will help you bring out the best in your test team. If you are a test team leader, or aspire to be in that role, this workshop will help you influence your team and those you interact with.

These two workshops are at special pricing for the Salt Lake City QA Focus Group, but I am extending that pricing to everyone! $750 for the Test Automation 2-day workshop (normally $1,400) and $375 for the 1-day Test Team Leadership workshop (normally $790).

The workshops will be held at The Pavilion Inn - http://www.slpavilioninn.com/

To see details and to register, just visit https://www.mysoftwaretesting.com.

Register soon to save your place! I hope to see you there!

Randy Rice

Tiger Woods Needs a Golf Teacher?

On my last trip to Rome, while flipping the two English channels in my hotel room, I happened to catch a story on CNN about Tiger Woods and his need for a new golf teacher/coach. This fascinated me.

Now, I’m not a golfer so that explains some of my curiosity about why one the world’s greatest golfers would need a teacher. In fact, Tiger is arguably the best golfer in the world today, although in tough times right now. I don’t even watch much golf on television, so I’m really a neophyte.

Tiger’s most recent coach was Hank Haney, who coached him from 2004 to May of 2010. Haney resigned because he felt he had taken Woods as far as he could.

http://www.cnbc.com/id/37192697/Instructor_Haney_Explains_Parting_Ways_With_Tiger_Woods

But still, I kept thinking, if the world’s best (or at least in the top five) needs coaching, what does that say about the whole concept of personal improvement. How does that apply to what testers do?

After thinking about this, I started to see some reasons why even the best in what they do need a coach, teacher, or mentor.

1. Objectivity

We all need someone who will tell us the truth, no matter how ugly it may be. And, they need to be able to do that from an external and unbiased perspective. True, a golfer can have a video camera and other ways to capture their golf swing, but it takes a lot of experience and insight to see the one little flaw that can increase a score.

Tiger probably has lots of people he can ask for opinions, and most of them will tell Tiger what they think he wants to hear. I find managers like this all the time. They can’t get the real story because so many people are either afraid to give bad news, or they are people-pleasers.

2. Broad context

Hank Haney has coached thousands of golfers, and also many golf pros. He has seen more bad swings than just about anyone. He has also seen the best. He knows what makes a difference because he has seen the techniques work in practice. Haney understands the mental aspect of golf as well.

3. Accountability

If you want to improve at anything, you have to practice and take positive action. Things that are easy to do are also easy not to do. A coach or mentor can ask you if you are on track doing the things needed to improve. They can’t make you do things, but they can remind you of your goals.

4. Encouragement

Everyone needs encouragement. Encouragement is interesting because it comes from others. You can have affirmations and self-talk, but a little encouragement from someone does so much more than what you can muster up yourself.

The Results

The results of these things are motivation and improvement. You’ll notice that I didn’t include either of these in the list the coach brings. These must come from the one being coached. These are brought out by a good teacher or coach.

Likewise, desire and the choice to change must come from those wishing to improve.

Keep in mind that I’m not necessarily talking about large steps of improvement. Whether you’re at the top of the game or just starting out, one tiny change can make a world of difference.

What About Testing?

Since many of you who read this blog are software testers or involved in software quality in some way, here are some tie-ins to what testers do.

Objectivity – This is the value of independent testers. The reason testers are needed is that they have a fresh view of something. They can see defects that someone with a lot of familiarity may miss. However, some testers are fearful of giving the bad news. That’s why a good testing process with measurements and metrics is so important. The process can deliver the news, whether good or bad.

Broad context – Testers do best, I believe, when they have worked in different companies and on different types of projects. You may not have much control over this, but you can still explore and expose yourself to different things. You can read, attend conferences, and get training to broaden your horizon.

Accountability – Testers can ask probing questions like, “Has this code been reviewed?” or “Has the business user seen this requirements document?” If there are development and testing processes in place, testers can tell if those processes are being followed.

Encouragement – Testers can give more than the bad news of defects. They can also give credit for great work.

Doing It

I’ve been sitting on this article for two months. I’ve wanted to write it, but for some reason just could not get off the dime. Perhaps it was my own lack of motivation.

Then, this week I happened to be flipping through my “300 channels of nothing” on TV and came across “The Haney Project” on the Golf Channel. I had no idea what this show was about, but soon saw that it tied everything together for me. In this episode, Hank Haney was teaching Rush Limbaugh how to improve his golf game.

A couple of things that stood out to me were that 1) he told Rush to only think about one thing during his swing, not twelve, and 2) Haney was very encouraging. At the end of the program, Limbaugh remarked about how impressed he was about working with Haney. Unlike other teachers who had been negative and overbearing, Limbaugh found Haney to be supportive and positive.

When I watch TV, I like to watch programs that show how people can improve. I found this program very interesting just watching Hank Haney’s coaching style in action.

Oh, and after watching Rush play golf, I'm thinking, "Hey, I can do at least that good!" So who knows? I may actually dust off my garage sale clubs and hit the links.

The big take-away for me in all this is that we all need people around us to help us improve, to encourage us and to bring accountability. I encourage you to find someone who can be that coach to you or your team. If you need help in that effort, let me know.

I would like to hear your comments on this topic!

3 Ways to Stretch Your 2010 Training Dollars

As we wind down 2010 and look forward to 2011, you may be left with some training dollars in your budget. I know I try to make the best use of my money right at the end of the year. Here are 3 ways you can do that:

1. Buy e-learning courses. I have placed several of my e-learning courses on sale until December 31, 2010. The great thing is that you have 12 months to complete the course!

2. If you live in the Utah area, or even in that region of the country, you may be interested in two workshops I will be conducting there in January. These are on the topics of test automation and test team leadership - and they are deeply discounted.

For details on either of these, visit https://www.mysoftwaretesting.com.

3. Study groups are forming for ISTQB Advanced certification. These are 6 to 7 weeks in length to support self-study. They are held online and have live web meetings with me as facilitator. You can pay in 2010 and attend in 2011. To get more details on this, visit http://riceconsulting.com/home/index.php/Home-Page/istqb-advanced-study-groups-forming.html.

I am working on a jam-packed newsletter for January, so stay tuned for that.

Finally, I wish you a Merry Christmas, Happy Hanukkah, and a prosperous and healthy New Year. Thanks for your support this past year!

Warm holiday regards,

Randy

New York City Voting Machine Problems - Implementation? We Don't Need no Stinking Implementation Plan!

I've spoken out before on voting machines and the related quality issues which cause me concern. So, this article caught my eye. It seems that today in New York there were major problems (not glitches) in the rollout of new voting machines. One might think perhaps the cause was software or hardware related. However, upon reading the account, it seems that the problem was largely in the implementation.

$160 million spent and this was the result.

I like the "teachable moment" and this is one. It doesn't really matter how good the software if you don't have the hardware plugged in!

http://newyork.cbslocal.com/2010/09/14/new-yorkers-head-to-the-polls-on-primary-day/

Do You Test SaaS, Or Does SaaS Test You?

Imagine you are sitting at your desk one morning and your phone rings. It's not your boss - it's your boss's boss, the CIO. He's upset because the online sales database is down and the entire sales staff is paralyzed. "The sales software is broken!" he exclaims. Then he asks, "Didn't you test this?" You take a deep breath and say, "No, that software is a service we subscribe to. We have no way to know when it changes." The CIO isn't happy, but now sees the reality of Software as a Service (SaaS).

I believe that SaaS is not a trend, but a major force that will shape the future of IT and, therefore, software testing. In fact, SaaS could dramatically change the way we think about and perform software testing from here on.

This article has a definite angle toward the risks of SaaS. I fully embrace the benefits and appreciate them. I use SaaS, as you will see later in this article. My point here is to point out the risks. Only you can weigh them against your own benefits.

What's Different?

1. You have little or no control over the software, the releases, and any remedies for problems.

In some cases, you may get the choice of when to accept an upgrade. However, even when you are on a specific version of the application, the vendor may choose to make a small change without advance notice.

In other cases, you may come in to work and notice that things on the application look a little different. Or, you get the dreaded phone call that something is broken.

But here's the real rub: Even if you isolate the problem and report it, you are still at the vendor's mercy to fix the problem.

At least with Commercial Off-the-shelf (COTS) software, you get the choice when to deploy it. So at least you get the chance to test and evaluate it beforehand in a test environment.

2. You have almost zero knowledge of structure, and often no knowledge of new and changed functionality.

Forget white-box testing, unless it is for customized interfaces (for example, APIs). Now, let's say you have noticed some functional changes. What really changed? What stayed the same? What are the rules? When are they applied? Many times, none of this is published to the customers.

Not only is SaaS a black-box, it is more like many black boxes, all in a cloud. Your SaaS application is most likely comprised of many services, each with their own logic. In fact, some of the services may be supplied by a vendor you are not even aware of.

3. There is no lifecycle process for testing.

In the past, we could test at various levels - unit, integration, system, and UAT. That's all gone with SaaS. It's all UAT. Sure, you can test low-level functionality and integration, but it's from the customer, not the developer perspective. In the lifecycle view of testing, your tests can build on each other. In the UAT or customer view, testing is a "big bang" event. So, forget "test early and often." That changes to just "Test often."

4. Testing is post-deployment.
There may be some exceptions in the case of beta testing, perhaps. But for most people, you get to see the software only after it is deployed. That's too late to prevent problems. All you can do is race to find them quickly. In other words, you are in reaction mode instead of prevention mode. Then, even if you find the problems before your customers do, they may not be fixed for some time.

5. Test automation is fragile (and futile).

You get no return on investment because ROI is achieved when tests are repeated. When a new version is released, there's a good chance your automated testware will not work. This means you are at risk of reworking your test automation at every release. That said, some people may find test automation of basic functions a way to monitor when changes have been made.

6. Test planning is also futile.

This is because there is little detailed knowledge about the application in advance of using it. There is no specification basis for testing unless you write them. Use cases might be effective for describing work processes supported by the application.

Is There Any Hope?

Yes, and it's called validation. Not validation in the sense of "all forms of testing", but validation in the sense of making sure the software supports your needs.

In Point #6 I mentioned use cases. This is a form of test planning you can perform. You can create test scenarios based on work flows you perform in your organization. However, you must not think in terms of software behavior. Instead, you must describe work processes that can be tested no matter which software is being used.

You can create use cases to describe work processes, not software processes. These can be used as a basis for testing. The flows are perfect for mapping test scenarios.

You can assess the relative risk of work flows to prioritize your testing. You can even use pairwise testing to reduce the numbers of combinations of test scenarios and test conditions.

However, only in certain situations will this help you avoid problems: 1) You are a beta tester or 2) you get to choose when to apply an upgrade.

There Will be Pain
You arrive at work on Monday morning and discover one of your SaaS applications has changed. You and your team scramble to test the high-risk scenarios (manually) and discover that many things are broken. You call the vendor only to learn that your wait time on hold is 30 minutes. You file issue reports and just get the auto-responder messages. Days go by. You keep trying, your manager tries calling, but hey, you're just one customer out of thousands. In many cases, there is no "Plan B."

This is the extreme end of the risk scale. Not all releases fail, and when they do fail, not all fail to this degree. However, the risk is real.

My Story

Just last week I experienced a problem that would have been very painful had it happened at a time I was teaching an online class. My online training services provider upgraded to a new version of the web presentation platform, but gave no notice. All they indicated was that the site would be down for maintenance on Friday evening. Turns out, they introduced a totally new and upgraded platform with several improvements. However, several key things failed: 1) I could record a session, but never get the file (I wasted an hour recording a session to find this out!), 2) I could not upload a file for presentation, 3) I could not install the new screen sharing software on Windows XP. There's not much you can do with web meeting software behaving like this.

I called tech support on Saturday. Guess what? Nobody home, even after a major release. I filed 3 problem tickets which as of this date have still not been closed out, even though the problems were fixed about 2 days later. (I did get one response.) I'm happy things are working now, but troubled about the way the release and post-release were handled. If I had been scheduled to teach an online class on Monday, I would have been stressed out for sure.

My case is fairly low-impact, but it did show me the risks of SaaS. I encourage you to keep these risks on your radar because like it or not, SaaS is in your future.

I would like to hear your experiences and ideas for testing SaaS, so leave a comment!

Software Test Automation Workshop in Oklahoma City - Sept 14 and 15, 2010

(This event has passed, but we offer an e-learning version, or will be happy to bring this training course to your company.)

I'm excited to announce we're holding the Practical Software Test Automation workshop in Oklahoma City on Tuesday, Sept. 14 and Wednesday, September 15, 2010.

The workshop will be held at the Hampton Inn, I-40 East (Tinker AFB) location at:

1833 Center Drive
Midwest City, Oklahoma, USA, 73110
1-405-732-5500

You can see the details (outline, pricing, etc.) and register at:

https://www.mysoftwaretesting.com/Practical_Software_Test_Automation_eLearning_p/pstaelearn.htm

GSA discounts are available for this workshop. Just contact me for details.

I hope to see you there!

U.S. To Train 3,000 Offshore IT Workers

I ran across this article in Information Week and I'm thinking, "What's wrong with this picture?" We have unemployed IT workers here in the USA and it seems that we are training people who will eventually compete with them for the few jobs that exist. By the way, there is an update that we will be doing the same thing in Armenia. Of course, this will all be done with borrowed money.

Some key quotes:

"Despite President Obama's pledge to retain more hi-tech jobs in the U.S., a federal agency run by a hand-picked Obama appointee has launched a $36 million program to train workers, including 3,000 specialists in IT and related functions, in South Asia. Following their training, the tech workers will be placed with outsourcing vendors in the region that provide offshore IT and business services to American companies looking to take advantage of the Asian subcontinent's low labor costs.

Under director Rajiv Shah, the United States Agency for International Development will partner with private outsourcers in Sri Lanka to teach workers there advanced IT skills like Enterprise Java (Java EE) programming, as well as skills in business process outsourcing and call center support. USAID will also help the trainees brush up on their English language proficiency."

I don't know. I've stopped asking "why?" Back when outsourcing became all the rage we were told that in a new service economy people would rise to higher level positions. Instead, these US IT workers are trying to find the best position they can in an economy where there is a race to the bottom in terms of pay.

I know there are job training programs here in the USA, but even trying to find the front door to those programs is difficult. How often do you see those advertised? And now even those programs are being cut due to lack of funds. I guess we need to pay for other training programs for people in...Sri Lanka, Armenia and who knows where else.

I would love to hear your thoughts on this.

The SaaS Performance Risk - An Example from Twitter.

Recently I was interviewed about application performance risks. The final question was, "What are some performance risks you see in new technologies?" My answer was the cloud computing and Software as a Service (SaaS) risk.

Actually, there are many risks and concerns in the SaaS model. This is not to say the model does not have value, even great value. It's just that there are things you need to know before adopting SaaS.

The basic idea of SaaS is that you use software applications that are developed, hosted and maintained by a vendor. You trust that the processing will be correct, fast enough for you, secure, easy to use, available, reliable, etc. However, you have no control over those things. When the vendor has a problem, you have a problem. That can be a rude awakening to people.

To illustrate the performance risk, I saw an article recently about the overload at Twitter:

http://www.computerworld.com/s/article/9179446/Twitter_s_tech_problems_take_a_toll_on_developers?source=CTWNLE_nlt_app_2010-07-22

Twitter has a huge challenge with extreme traffic spikes. Now, on the upside, this is not a national security type of application, and most of us Twitter users have gotten used to the "Fail Whale" picture and just say, "Oh, well...I'll tweet that later."














Then, other things happen:
















But there is a business impact to the companies that sell services interfaced to Twitter:

"'Twitter API issues in the previous weeks have been terrible for us," said Loic Le Meur, founder and CEO of Seesmic, which makes Twitter client applications for various desktop and mobile platforms. 'Users always blame Seesmic first since it's their primary interface to Twitter. It's extremely frustrating because there is nothing else we can do than warning users Twitter has problems. It is very damaging for us since users start to look for alternatives, which fortunately have the same problems, but damage to the brand is done,' he said via e-mail.'"

To be fair to Twitter, they've done a lot to improve reliability in recent years.

In a related story, from last week, I read on Bloomberg.com:

"Interest in the SaaS (software as a service) delivery model is growing to the point that by 2012, almost 85 percent of new vendors will be focused on SaaS services, according to new research from analyst firm IDC. Also by 2012, some two-thirds of new offerings from established vendors will be sold as SaaS, IDC said."

http://www.businessweek.com/idg/2010-07-26/idc-saas-momentum-skyrocketing.html

So, what does all this mean to you?

If you are considering SaaS as a major application delivery method, then be aware of the risks. In reality, there isn't a lot you can do during a SaaS failure. For example, you can't just call up the folks at Twitter and tell them to "get it fixed" (like perhaps you can speak with the developers in your own company).

Some of this goes back to service-level agreements (SLA), but no vendor I know of will guarantee 100% uptime. Even if they did, there are other risks, such as correctness, not typically covered in an SLA.

I wish I had a handy list of things to mitigate the risks, but every case is different.

I would be interested in hearing your experiences with SaaS and how you deal with the risks, so please leave a comment.

How to Create a Fear-Based Culture

Hi folks,

Here is a great blog post from The Next Level Blog by Scott Elbin. Those who know me, know I'm big on the human factors in software development and testing. A big part of that is culture. One of Dr. Demming's 14 points is to "Drive out fear." What does that mean? Well, here's seven ways NOT to do it. See if you recognize any of them. Have a great day!

http://scotteblin.typepad.com/blog/2010/07/seven-simple-rules-to-create-a-fear-based-culture.html

Practical Software Test Automation Course Now Available in eLearning!

I am excited to announce the release of my newest course, Practical Software Test Automation, in e-learning format.

This course focuses on the basics of software test automation and expands on those topics to learn some of the deeper issues of test automation. This course is not specific to any particular tool set but does include hands-on exercises using free and inexpensive test tools. The tool used for test automation exercises is Macro Scheduler.

The main objective of this course is to help you understand the landscape of software test automation and how to make test automation a reality in your organization. You will learn the top challenges of test automation and which approaches are the best ones for your situation, how to establish your own test automation organization, and how to design software with test automation in mind. You will also learn many of the lessons of test automation by performing exercises using sample test automation tools on sample applications.

I hope to see you there!

Click here to see the course outline.

Click here to take a demo. (You can login as a guest)

Click here to register for the course.

Wahington Post Series on Top Secret Sites is Shameful

Hi Folks,

Normally I don't get political on this blog, and actually, I don't really think this post is political. But I do think the topic is important.

Today, the Washington Post unveiled their series on the Top Secret work done by the Federal Government.

http://projects.washingtonpost.com/top-secret-america/companies/1/

Here's my problem: There will be a lot of innocent people placed at risk simply because of who they work for. Imagine this scenario: Joe Smith, an employee (fictitious) of a Top Secret government contractor takes a trip to a quasi-friendly (or even unfriendly) country to perform work for another client. Joe winds up in a situation for some reason that involves police authorities in said country. They ask him where he works. He answers truthfully. They run that information through their systems and bingo, get a hit. (They know all the companies now because of this article) Depending on the person running the query, Joe might be flagged as an agent. He certainly has knowledge of Top Secret information. Right?

Well...maybe, maybe not. However, try convincing an authority in a foreign country of that.

In fact, people don't even have to travel abroad. Now, our enemies know exactly where the offices of these companies are. They now have all types of targets for espionage and for recruiting spies.

Some will say the articles have a noble purpose to expose government waste. Is that something we don't already know?

Some may also say, like in a Tom Clancy novel, if the Washington Post can find this information, our enemies already know it. Yes, but they've made it really easy to find - all in one place, hyperlinked, with maps and all.

It will be interesting to see if any of the same people who were outraged over the "outing" of Valerie Plame will be outraged over this. I doubt it.

I think the Washington Post has abused the liberty of freedom of the press by publishing this series, but the damage has already been done.

New Testing Skills Needed for Companies that are Rebuilding Test Teams

It's still a tough economy, but some companies are starting to hire in IT again, even hiring software testers. That's an encouraging sign!

I've seen many companies struggle in the team rebuilding process, mainly due to simply getting people on the same basis of knowledge. The thing to consider is, when you start to being new people into your teams, how will they build the skills they need to be effective on your team?

Here is what often happens. After a company starts to feel the pain of tasks left undone (for testing, that means defects going straight to customers and customers leaving), they start to rebuild the teams. So, you search for the best and brightest people, and hire who you can afford. These people have a mixed bag of skills and talents, all learned from various sources, some practices effective, some ineffective. And then, some people embellish their skills on the resume, so when they are hired they don't perform as expected. But, you still stick with them at least for awhile.

Then, you have the faithful and the tough - the people who have been with the company for a long time and have never been formally trained.

If this situation is left "as is" you basically have a stew that is not very tasty.

What's the Solution?

1. Perform a Skills Assessment. This will tell you exactly where each person stands in their overall skill set.

2. Perform Training. This lays in place a foundation of common skills and terminology.

3. Perform Continued Mentoring. This reinforces the skills and may be needed for topics that the training can't reach, such as organization procedures, etc.

The sooner you can do this, the better, as long as you have most of the team in place. For the ones that join after the training, it's good to have e-learning available.

If you need help in getting your team's skills in place, contact me. I have over 60 courses in software testing and related topics. Tester certification is also a good approach for many teams.

I can create a custom training plan for your organization that will give you a head-start and boost your effectiveness as a team.

Is Software QA Dead?

Is traditional Software Quality Assurance (SQA) dead? Or...does it just suffer from poor perception and bad practice? I hope it is the latter.

I started thinking about this question after hearing a presentation recently that highlighted the problems with traditional SQA. The more I think about it, the more I believe we still need true SQA (not just testing). If you don't know the difference, please read on.

QA and QC

First, we must understand that true QA is not testing and it is not a verb. So, to say “then we QA it.” is like saying “then we configuration management it.”

SQA focuses on how a process is performed and is the management of quality. SQA can encompass metrics, process definition and improvement, testing, lifecycle definition, and so forth. The SQA function may perform some tests and reviews (quality control or “QC”) but unless there is quality management, the effort can easily become haphazard and uncontrolled.

Software testing is QC. So are reviews and inspections. The key difference is that the focus is on the product to find any defects. SQA and QC must work hand-in-hand to be effective.

SQA is process assurance, that is, assurance that the process is being performed as designed. QC is product assurance, or assurance that the product meets specifications. So, both activities are needed.

New vs. Old

I've been developing and testing information systems for over 33 years now. There's always something new that people think will change the way everyone develops software. Think about all the approaches and methods that have come down the pike – from waterfall to agile. Why do people still use the waterfall? Why are some people ardent evangelists of agile?

I think much is explained by comfort zones and culture. People tend to use approaches they are comfortable with. People don't like to change.

However, people do like to be fashionable. New approaches are fashionable which gives them early adopters who become enthusiastic supporters. After all, when was the last time you saw someone excited about the waterfall approach?

Process-orientation vs. Product-orientation

Back in the 70's and 80's, one of the big issues was that much focus was on the software, not on how it was built. So, the famous quote was “If builders built buildings the same way programmers write code, the first woodpecker that came along would destroy civilization.”

In 1985, I was performing eXtreme Programming, it just wasn't called that. I worked in tandem with another programmer, developed my tests first, then coded to them, and worked from user stories. The problem was inconsistency. We were the only two working this way. There was no one in management that wanted to spread the technique. In those days, like today, waterfall was king. Why? Because the waterfall model can be explained in about 10 minutes or less.

Software project consultants looked at this state of affairs and concluded that the process has a great deal to do with creating software and that software development should be an engineering effort, not an art or a craft. Hence, the title, “software engineer” and following years of creating process models, such as the Capability Maturity Model (CMM). The CMM was very process-centric. In fact, the original version didn't have a key process area for software testing. The idea was that if the process was performed correctly, testing would not be needed.

The flaw in the process-only idea is that people are not perfect. Therefore, there will be mistakes at every step in building software, all the way from concept through system retirement. The only way to find the defects caused by these mistakes is to detect them by an effort designed to find them. A filtering approach where defects are screened out by inspections along with early and ongoing testing is a very effective approach.

The good part of the process focus is that it does typically deliver a better product with fewer defects injected throughout the project life cycle. This has been proven by organizations with high levels of process maturity who also measure defect detection percentage. (See chart.)

Indeed, processes provide a valuable framework to organize and perform all other project tasks. In short, processes can be improved, they can be shared and they can be trained.

SQA is a key mechanism by which improvements are made. In organizations it is common to find pockets of both good and bad practice. Improvement is rare, which caused people to see the need for processes to begin with. A few years back, Lewis Gray wrote a great article for Crosstalk Journal entitled, "No Hypoxic Heroes, Please", in which he makes a compelling case for software processes using the example of why mountain climbers follow processes and standards – which is to keep from making bad decisions when their minds start to become oxygen-deprived and the ability to reason is impaired. We see the same thing on software projects, especially as the deadline looms closer and closer.

Bad SQA Practice

It is possible to take any effective tool or approach and apply it in an ineffective way. Some organizations have built the SQA function into a bureaucracy which slows projects down and adds little value to the organization. In fact, defects may even increase due to people spending so much time performing paperwork. This was never the objective of SQA, “old school” or in any context.

Other organizations have turned SQA into a police force which investigates, audits and regulates software projects. The intent is noble, but the rest of the project lives in fear of the SQA team and what they can do to them.

In some organizations, SQA is a gatekeeper. To get software into production use, it must get QA approval. Once again, this is a negative view of what SQA is intended to perform. In reality, implementation should be a team-based decision based on risk.

The average lifespan for a SQA group is about two years. That's because after about two years, senior management asks, “What do these people do?” Unless the SQA team can show tangible and positive value, the decision is likely to be made to try something else to improve software quality. All too often, the SQA team's work is seen as intangible and paper-shuffling.

Back in 2000 I gave a keynote presentation at QAI's testing conference in which I made a major point that if QA and test organizations must be in alignment with business objectives and project objectives, or else they will be marginalized and most likely eliminated. In other words, the QA and test teams must find the right balance of finding and preventing problems, as compared to not stopping progress.

I believe the only people who can direct that balance are the business stakeholders. These are the people who live with the level of software quality, or at least know what they want the business's customers to experience.

What Can Really Be “Assured”?

Not much, in my experience. We can't guarantee perfection since we can't test everything. Likewise, we can't assure a process has been perfectly performed. Even if a process is perfectly performed, the process itself can be flawed.

This leads me to my final point which concerns processes as performed professionally versus those performed in a factory setting. In a factory, you want everyone doing the same thing in the same way. You do not want any variation at all.

However, the more professional the effort and the person, the more you can rely on their expertise to do the job expertly. There are many examples of this, such as great chefs knowing how to prepare great meals without a recipe, or great doctors not reading the process book as they perform surgery. However, what is not seen, are the rules, standards and protocols each of these professionals adhere to. For example, the chef cooks food to pre-established proper temperatures to prevent food poisoning. The surgeon has a team of people following an exact checklist to make sure all preparation is correct before the surgery and all surgical instruments are accounted for after the surgery.

In software development we rely on many people to get it right. All the way from concept to delivery, developers, business analysts, architects, testers, DBAs, trainers, management, customers and others must work together in professional ways. There may or may not be a formal software life cycle followed.

When things work well, no one seems to think much about software QA or testing. It's like the air conditioning - no one gives it a thought until it breaks down. However, when the product being delivered starts to slip and the customers start to leave, then management starts thinking “Maybe we need some structure in place to make sure we do the right things in the right ways.”

Smart QA

QA done well and done smartly, can be a very helpful activity. The problem is when people don't match the QA approach to the business and project context.

Instead of doing some basic root cause analysis and finding the true source of the problems, which can be addressed and prevented, some companies embark on major pushes to install a new QA program, SDLC or both.

Instead, how about some simple steps such as basic processes, checklists, guidelines and re-designed tests? Then, after seeing how those work, we can make further adjustments.

Conclusion

I don't think true QA (not testing) is dead, but I do think it suffers from bad practice and poor perception. I also believe there is a pendulum effect which swings from one extreme to another. In the case of software, the pendulum swings from “no process” to “all process”. Right now, we are at the “no process” end of the swing, with movement toward the center.

QA is a function that each organization must decide how best to adopt. Some will reject QA entirely, some will adopt it smartly and some will adopt it inconsistently or with bureaucratic approaches.

I hope you apply QA smartly. If you need help in doing that, call me!

Software Test Automation Workshop in the Dallas/Ft. Worth Area - Aug 12 and 13, 2010

I'm excited to announce I'm coming to the Dallas/Ft. Worth area in August to present my newest course - Practical Software Test Automation workshop!

Here's the details and how to register:

http://riceconsulting.com/home/index.php/Public-Seminars/randy-rice-to-present-test-automation-workshop-in-dallas-ft-worth-area-aug-12-and-13-2010.html

http://www.mysoftwaretesting.com/product_p/dfwauto.htm

I hope to see you there!

Want a class like this in your city or at your company? Let me know!

Thanks,

Randy

20 Year Anniversary of Rice Consulting Services Today

Today is the 20th anniversary of Rice Consulting Services! It has been an amazing ride. I'm thankful to God for keeping His business going for this long.

It was 20 years ago today...Sgt. Peppers taught his band to play....I mean we moved from Kansas City back to our home of Oklahoma City to start Rice Consulting Services. My first project was the Oklahoma City Water Trust, which was a major failure. I call it "the day I tested myself out of a job" because I asked whether or not the system had been stress tested. Turns out, it hadn't and it could not stand any load. The system never saw the light of day. You can read all about it here. It's worth your time.

Then, I got to know people like Bill Perry who gave me a great national platform with the opportunity to speak at the QAI testing conferences. We wrote together, Surviving the Top Ten Challenges of Software Testing, which opened many other doors.

Thanks to all of my friends and clients, who have supported us in the good and bad times.

Thanks to Janet, my wife and President of RCS, who is 100% behind this business and understands what small business is like. We eat what we shoot. We take risks that would make some people sleepless. There are no guarantees or bailouts for small businesses. We are small enough to fail, but by the grace of God, we keep going and serving our clients.

I feel like one the farmers who won the lottery a few years back. When asked what they were going to do with the money, one of them said, "We're gonna keep farmin' til the money runs out."

So, I raise my cup of coffee to you, my friends that read this blog. Thanks for your support. I look forward to 20 more years. We will probably still be talking about how to write test plans. And, that is...well, depressing in a way, but shows the never-ending job of skill building in testing.

Thanks, everyone!

Agile and Exploratory Testing in Kansas City - July 13 and 14, 2010

I'm excited to announce I'm coming back to the Kansas City area in July to present my Agile and Exploratory Testing workshop!

Here's the details and how to register:

http://riceconsulting.com/home/index.php/Public-Seminars/randy-rice-to-present-agile-exploratory-workshop-in-kansas-city-area-july-13-and-14-2010.html

About the course: http://riceconsulting.com/home/index.php/Table/Agile-Testing/

There's a special discount for KCQAA members. I hope to see you there!

Want a class like this in your city? Let me know!

Thanks,

Randy

The Surprising Truth About What Motivates Us

You have to see this video. It's 10 minutes long and I promise you will find it interesting, entertaining, and a valuable use of your time.

http://michaelhyatt.com/2010/05/the-surprising-truth-about-what-motivates-us.html

Have a great and safe weekend...and remember those who have served and died for our country!

Randy

Webinar Links - Ellusive Tester to Developer Ratio

Hi everyone,

Thanks for tuning in to the webcast today.

Here are the links for the recorded session and chat transcript, along with other things I mentioned.

Recorded webinar
Chat transcript
Slides in PDF format

CTE-XL tool
Article - Ellusive Tester to Developer Ratio (The one I wrote back in 2000)

Thanks!

Randy

May 2010 Newsletter Posted

Hi Folks,

Better late than never! The May issue of the Software Quality Advisor Newsletter is out:

http://riceconsulting.com/home/index.php/Newsletter-Past-Issues/may-2010-test-estimation-based-on-testware.html

You can get your copy each month to your e-mail account by signing up at:

http://riceconsulting.com/home/index.php/Newsletter/the-software-quality-advisor-newsletter-sign-up.html

Webinar - Thursday, May 27 - The Elusive Tester to Developer Ratio

Join me on Thursday, May 27 as I present a webinar on The Elusive Tester to Developer Ratio. The session is free and starts at 12:00 noon Central Daylight Time. There are 50 slots available on a first come basis, so I suggest being there 10 minutes early. To register, just click here.

To join the meeting, visit https://my.dimdim.com/ricecon on May 27 at 11:50 a.m., CDT. The webinar starts at 12:00 noon.

Book Review - Reflections on Management by Watts S. Humphrey with William R. Thomas

The Capability Maturity Model (CMM) and Capability Maturity Model Integrated (CMMI) have been major forces in software development for at least 20 years. Along with those, the Personal Software Process (PSP) and the Team Software Process (TSP) have also been applied to help make software projects more predictable and manageable.

This book is a collection of essays and articles written by Watts Humphrey, the man who was the influence and drive behind these models and processes. I found this book to be an interesting journey through the thinking of Humphrey as he clearly and rationally outlines the "why" behind the "what." Then, he describes "how" to do the work of managing intellectual and creative people which have to work together to deliver a technical product - on time, within budget, with the right features and with quality.

There are many gems in this very readable book (a great airplane book), such as:

• Defects are Not Bugs
• The Hardest Time to Make a Plan is When You Need it Most
• Everyone Loses With Incompetent Planning
• Every New Idea Starts as a Minority of One
• Projects Get into Trouble at the Very Beginning

This book is divided into four parts:

1. Managing Your Projects
2. Managing Your Teams
3. Managing Your Boss
4. Managing Yourself

If you are a software project manager, test manager, or test team leader who has to fight the battles involved in getting a project completed within time, budget, scope and quality targets, you will find this book of immense value. Or, you might buy it as a gift for your manager who just doesn't get what's so hard about software development.

Although this book is a collection of essays, it flows very well and reads like it was written as one book. By the way, I felt the Epilogue was excellent - don't skip it.

If there are any doubts about the credibility factor of this book, the advance praise at the front of the book spans four pages and reads like a "who's who" of software development: Steve McConnell, Ed Yourdon, Ron Jeffries, Walker Royce, Capers Jones, Victor Basili, Lawrence Putnam and Bill Curtis, to name a few.

Whether you are fully immersed in the agile project world, or following the CMMI, or just trying to figure out the best way to plan, conduct and manage software projects, this is a book worth reading and taking to heart. In the advance praise, Ron Jeffries (www.XProgramming.com) writes, "I've followed Watts Humphrey's work for as long as I can remember. I recall, in my youth, thinking he was asking too much. Now that I'm suddenly about his age, I realize how many things he has gotten right. This collection from his most important writings should bring these ideas to the attention of a new audience: I urge them to listen better than I did."

Amen, Ron, amen.

Reviewed by Randy Rice

Disclosure of Material Connection: I received one or more of the products or services mentioned above for free in the hope that I would mention it on my blog. Regardless, I only recommend products or services I use personally and believe will be good for my readers. Some of the links in the post above are “affiliate links.” This means if you click on the link and purchase the item, I will receive an affiliate commission. Regardless, I only recommend products or services I use personally and believe will add value to my readers. I am disclosing this in accordance with the Federal Trade Commission’s 16 CFR, Part 255: “Guides Concerning the Use of Endorsements and Testimonials in Advertising.”

Test Estimation Based on Testware

I had a conversation recently with one of my clients about test estimation based on manual test cases. As I have written previously, I feel that in many ways test estimation is inherently flawed for a variety of reasons.

However, there is a technique I have used over the years that plays on risk-based approaches. This technique can be applied to testware, such as test cases. Just remember this is not a scientific model, just an estimation technique.

What is Testware?

Testware is anything used in software testing. It can include test cases, test scripts, test data and other items.

The Problems with Test Cases

Test cases are tricky to use for estimation because:

1. They can represent a wide variety of strength, complexity and risk

2. They may be inconsistently defined across an organization

3. Unless you are good at measurement, you don’t know how much time or effort to estimate for a certain type of test case.

4. You can’t make an early estimate because you lack essential knowledge – the number of test cases, the details of the test cases and the functionality the test cases will be testing.

Dealing with Variations

“If you’ve seen one test case, you’ve seen them all.” Wrong. My experience is that test cases vary widely. However, there may be similarity between some cases, such as when test cases are logically toggled and combined.

A technique I have used to deal with test case variation is to score each test case based on complexity and risk, which are two driving factors for effort and priority.

The complexity rating is for the test case, not the item being tested. While the item’s complexity is important in assessing risk, we want to focus on the relative effort of performing the test case. You can assign a number between 1 and 10 for the complexity of a test case. It may be helpful to create criteria for this purpose. Here is an example, You can modify it for your own purposes.

1 – Very simple

2 – Simple

3 – Simple with multiple conditions (3 or less)

4 – Moderate with simple set-up

5 – Moderate with moderate set-up

6 – Moderate with moderate set-up and 3 or more conditions

7 – Moderate with complex set-up or evaluation, 3 or more conditions

8 – Complex with simple set-up, 3 or more conditions

9 – Complex with moderate set-up, 5 or more conditions

10 – Complex with complex set-up or evaluation, 7 or more conditions

This assessment doesn’t consider how the test case is described or documented, which can have an impact on how easy or hard a test case is to perform.

Assessing Risk

Risk assessment is both art and science. For estimation, you can be subjective. In fact, my experience is that risk assessment is subjective at some point or other.

This scale is based on the risk (impact) of the test case and its priority in the test. Like the complexity ranking, here are sample criteria you can adapt for your own situation:

1 – Lowest priority, lowest impact

2 – Low priority, low impact

3 – Low priority, moderate impact

4 – Moderate priority, moderate impact

5 – Moderate priority, moderate impact, may find important defects

6 – Moderate priority, high impact, has found important defects in the past

7 – High priority, moderate impact, new test

8 – High priority, high impact, may find high-value defects

9 – High priority, high impact, has found high-value defects in the past

10 – Highest priority, highest impact, must perform

Actually, the risk level could be seen from two perspectives - the risk of the item or function you are testing, or the risk of the test case itself. For example, if you fail to perform a test case that in the past has found defects, that could be seen as important enough to include every time you test. Not testing it would be a significant risk. The low risk cases would be those you could leave out and not worry about. Of course, there is a tie-in between these two views. The high-risk functions tend to have high-risk test cases. You could take either view of test case risk and be in the neighborhood for this level of test estimation.

Charting the Test Cases

To visualize how this technique works, we will look at how this could be plotted on a scatter chart. There are four quadrants:

1 – Low complexity, low risk

2 – High complexity, low risk

3 – Low complexity, high risk

4 – High complexity, high risk

Each test case will fall in one of the quadrants. One problem with the quadrant approach is that any test case in the center area of the chart could be seen as borderline. For example, in Figure 1, TC004 is in quadrant 4, but is also close to the other areas as well. So, it could actually be in quadrant 1 if the criteria are a little off.

Figure 1

For this reason, you may choose instead to divide the chart into nine sections. This “tic-tac-toe” approach gives more granularity. If a test case falls in the center of the chart, it is clearly in section 5 (Figure 2), which can have its own set of test estimation factors.

Figure 2

All You Need is a Spreadsheet

With many test cases, you would never want to go to the trouble of charting them all. All you need to know is in which section of the chart a test case resides.

Once you know the complexity and risk scores, all you need to know are the sections on the chart. For example, if the complexity is 3 or less and the risk is 3 or less, the test case falls in section 1 of the nine-section chart. These rules can be written as formulas in a spreadsheet (Figure 3).

Figure 3

Sampling

So, what if you don’t have a good history of how long certain types of test cases take to perform? You can take samples from each sector of the chart.

Take a few test cases from each section, perform the test cases and measure how long it takes to set-up, perform and evaluate each test case.

You now extend your spreadsheet to include the average effort time for each test case (Figure 4).

Figure 4

Adjusting

Your estimate is probably inaccurate. There is a tendency to believe the more involved and defined the method is, the more accurate the estimate will be. However, the reality is that any method can be flawed. In fact, I have seen very elaborate estimation tools and methods which look impressive, but were inaccurate in practice.

It’s good to have some wiggle-room in an estimate as a reserve. Think of this factor as dial you can turn as your confidence in the estimate increases.

Conclusion

Like with any estimation technique, at the end of the day, there could be any number of things that could impact the accuracy of most estimates. Estimates based on test cases can be helpful once you have enough history of measuring them.

Sampling can be helpful if you have no past measurements, or if this is a new type of project for your organization. It is still a good idea to measure the actual test case performance times so you can incorporate them in your future estimates.

I hope this technique helps you and provides a springboard for your own estimation techniques.

New Test Automation Class a Success!

Thanks to everyone that participated in last week's presentation of my newest workshop, Practical Software Test Automation in Oklahoma City. The class went well, we had a great time together, and I learned some adjustments I need to make. Thanks to the Red Earth QA SIG for their sponsorship.

We also got some positive buzz from Marcus Tettmar, the maker of Macro Scheduler. Thanks, Marcus.

http://www.mjtnet.com/blog/2010/05/13/new-software-testing-course-featuring-macro-scheduler/

We use Macro Scheduler as the learning tool in the course for test automation and scripting. By the way, version 12 has just been released. I look forward to trying it!

http://www.mjtnet.com/blog/2010/05/17/macro-scheduler-12-is-shipping/


The next presentation will be next month in Rome. If you live in Italy, or have a desire to take a testing class in a great location, join me there on June 16 and 17. I will also be presenting the Innovative Software Testing Approaches workshop that week in Rome.








Innovative Software Testing Approaches
http://www.technologytransfer.eu/event/984/Innovative_Software_Testing_Approaches.html

Software Test Automation
http://www.technologytransfer.eu/event/985/Software_Test_Automation.html


If you are interesting in having this workshop in your city or company, just contact me!

The 900+ Point Drop in the Dow - A Real-Life Root Cause Analysis Challenge

Just a random thought today...

The other day when the Dow dropped over 900 points, it was blamed on some trader someplace entering an order to sell "billions" instead of "millions" of P&G stock. To date, they still cannot produce the trade or the trader. Something doesn't smell right.

1. I would think that level of trade would require some sort of secondary approval.
2. Isn't there an audit trail of trades that would lead back to the trader?
3. If a billion dollar trade could do this, shouldn't there be an edit or at least warning message, "You have entered an amount in the billions. Click OK to bring down the entire global financial system."

This makes me question if this was really the case. Other possibilities could be:

1. A run on stocks that was truly panic selling and this was a way to explain it away without spooking everyone else in the country. I guess this is the "conspiracy theory" view.
2. A software defect...and maybe not a simple one. This could be one of those deeply-embedded ones. I know a little about how the Wall St. systems and processes work and believe me, this is not beyond the realm of possibility.

It may be impossible to know for certain. One would have to perform a deep dive root cause analysis, go through the change logs (if they exist), look at the exact version of code for everything going on at that time, look at a highly dynamic data stream...you get the idea. That probably won't happen. If someone does manage to isolate this as a software defect and can show it, I nominate them for the Root Cause Analysis Hall of Fame, located in Scranton, Pa. (Don't go looking for that...but there is a Tow Truck Museum in Chattanooga, TN.)

Just a random thought...

Great Customer Service in Action at Blue Bean Coffee

I think as software testers (and IT professionals in general) we need to pay a lot of attention to how we serve our customers. Many companies make the outsourcing decision because IT is no longer delivering the level of service the company needs. Frankly, I have been in some IT departments that deserve to be outsourced due to complacency and lack of business focus.

However, people know great service when they see it.

This morning I was having coffee with my pastor at Blue Bean coffee here in south Oklahoma City (SW 134 and Western). All of a sudden, the lady behind the counter ran out the door with a can of whipped cream in her hand. Turned out she had forgotten to top off the customer's drink. She said, "That would have bothered me all day."

Wow. That made an impression. Really, it's not that big a deal, but in today's "lack of service" culture it stands out.

Compare that to my experience a couple of weeks ago at Starbucks, not too far away. I go in with my own mug and ask for the free cup of coffee they were promoting for Earth Day. (This happened on Earth Day!) The barista looked confused and went in the back office to ask the manager. He came back out and said, "Sorry, that was last week." "OK," I said. "I guess I'll have a tall bold." To which he replied, "We only have a little left." "Is is fresh," I asked. "We change it every 30 minutes," he said. Now, you coffee drinkers out there know that a little coffee left on the heat for even a short time gets bitter, but I took my chances. Turned out, it was bad but I has already left. I complained to Starbucks online and got a good response back and two free drink coupons.

I was left wondering, though, how hard would it have been for the barista to simply say, "That promotion was last week, but here, have a cup anyway."? Perhaps he was not empowered to do that, or maybe he just wasn't thinking about the lifetime value of a customer.

Remember how Starbucks has tried all the other "ambiance" stuff - music, etc. I applaud the efforts at creating an experience, but to me the customer, I don't go to Starbucks for music. I go there for coffee and hope for decent service. Sometime they are friendly and sometimes they aren't. For the longest time all they served was Pike Place. Then, finally, someone in corporate woke up and decided to start offering different blends again.

All I can say is that I'll use those Starbucks coupons on the road and get my local coffee at Blue Bean. It's more personal, more friendly and they have better coffee. And that's what it's all about - the coffee and the service.

Keep that in mind as you serve people in IT. If you give great service and have a great product, people will find you irreplaceable and keep coming back for more. You will be more personal and more valuable.