Thoughts on the Consulting Profession

Sometimes I come across something that makes me realize I am the "anti" version of what I am seeing or hearing.

Recently, I saw a Facebook ad for a person's consulting course that promised high income quickly with no effort on the part of the "consultant" to actually do the work. "Everything is outsourced," he goes on to say. In his videos he shows all of his expensive collections, which include both a Ferrari and a Porsche. I'm thinking "Really?"

I'm not faulting his success or his income, but I do have a problem with the promotion of the concept that one can truly call themselves a consultant or an expert in something without actually doing the work involved. His high income is based on the markup of other people's subcontracting rates because they are the ones with the actual talent. Apparently, they just don't think they are worth what they are being billed for in the marketplace.

It does sound enticing and all, but I have learned over the years that my clients want to work with me, not someone I just contract with. I would like to have the "Four Hour Workweek", but that's just not the world I live in.

Nothing wrong with subcontracting, either. I sometimes team with other highly qualified and experienced consultants to help me on engagements where the scope is large. But I'm still heavily involved on the project.

I think of people like Gerry Weinberg or Alan Weiss who are master consultants and get their hands dirty in helping solve their client's problems. I mentioned in our webinar yesterday that I was fortunate to have read Weinberg's "Secrets of Consulting" way back in 1990 when I was first starting out on my own in software testing consulting. That book is rich in practical wisdom, as are Weiss' books. (Weiss also promotes the high income potential of consulting, but it is based on the value he personally brings to his clients.)

Without tooting my own horn too loudly, I just want to state for the record that I am a software quality and testing practitioner in my consulting and training practice. That establishes credibility with my clients and students. I do not get consulting work, only to then farm it out to sub-contractors. I don't consider that as true consulting.

True consulting is strategic and high-value. My goal is to do the work, then equip my clients to carry on - not to be around forever, as is the practice of some consulting firms. However, I'm always available to support my clients personally when they need ongoing help.

Yes, I still write test plans, work with test tools, lead teams and other detailed work so I can stay sharp technically. However, that is only one dimension of the consulting game - being able to consult and advise others because you have done it before yourself (and it wasn't all done 20 years ago).

Scott Adams, the creator of the Dilbert comic strip had a heyday with poking fun at consultants. His humor had a lot of truth in it, as did the movie "Office Space."

My point?

When choosing a consultant, look for 1) experience and knowledge in your specific area of problems (or opportunities), 2) the work ethic to actually spend time on your specific concerns, and 3) integrity and trust. All three need to be in place or you will be under-served.

Rant over and thanks for reading! I would love to hear your comments.

Randy

ISTQB Advanced Technical Test Analyst e-Learning Course

I am excited to announce the availability of my newest e-Learning course - The ISTQB Advanced Technical Test Analyst certification course.

 

To register, just go to https://www.mysoftwaretesting.com/ISTQB_Advanced_Technical_Test_Analyst_e_Learning_p/advtta.htm

To take a free demo, just click on the demo button.

This e-Learning course follows on from the ISTQB Foundation Level Course and leads to the ISTQB Advanced Technical Test Analyst Certification. The course focuses specifically on technical test analyst issues such as producing test documentation in relation to technical testing, choosing and applying appropriate specification-based, structure-based, defect-based and experienced-based test design techniques, and specifying test cases to evaluate software characteristics. Candidates will be given exercises, practice exams and learning aids for the ISTQB Advanced Technical Test Analyst qualification.

Webinar Recording and Slides from Estabilishing the Software Quality Organization

Hi all,

Thanks to everyone who attended yesterday's webinar on "Establishing the Software Quality Organization" with Tom Staab and me.

Here is the recording of the session: http://youtu.be/pczgcHGvV5Q

Here are the slides in PDF format: http://www.softwaretestingtrainingonline.com/cc/public_pdf/Establishing%20The%20SQO%20webinar%2011132014-1.pdf

I hope you can attend some of our future sessions!

Thanks,

Randy

Book Review – Introduction to Agile Methods - Ashmore and Runyan

Click to Order on Amazon

First, as a disclaimer, I am a software testing consultant and also a project life cycle consultant. Although I do train teams in agile testing methods, I do not consider myself as an agile evangelist. There are many things about agile methods I like, but there are also things that trouble me about some agile methods. I work with organizations that use a wide variety of project life cycle approaches and believe that one approach is not appropriate in all situations. In this review, I want to avoid getting into the weeds of sequential life cycles versus agile, the pros and cons of agile, and instead focus on the merits of the book itself.

Since the birth of agile, I have observed that agile methods keep evolving and it is hard to find in one place a good comparison of the various agile methodologies. I have been looking for a good book that I could use in a learning context and I think this book is the one.

One of the issues that people experience in learning agile is that the early works on the topic are from the early experiences in agile. We are now almost fifteen years into the agile movement and many lessons have been learned. While this book references many of the early works, the authors present the ideas in a more recent context.

This is a very easy to read book and very thorough in it’s coverage of agile methods. I also appreciate that the authors are up-front about some of the pitfalls of certain methods in some situations.

The book is organized as a learning tool, with learning objectives, case studies, review questions and exercises, so one could easily apply this book “as-is” as the basis for training agile teams. An individual could also use this book as a self-study course on agile methods. There is also a complete glossary and index which helps in getting the terms down and using the book as a reference.

The chapter on testing provided an excellent treatment of automating agile tests, and manual testing was also discussed. In my opinion, there could have been more detail about how to test functionality from an external perspective, such as the role of functional test case design in an agile context. Too many people only test the assertions and acceptance-level tests without testing the negative conditions. The authors do, however, emphasize that software attributes such as performance, usability, and others should be tested. The “hows” of those forms of testing are not really discussed in detail. You would need a book that dives deeper into those topics.

I also question how validation is defined and described, relying more on surveys than actual real-world tests. Surveys are fine, but they don’t validate specific use of the product and features. If you want true validation, you need real users performing tests based on how they intend to use a product.

There is significant discussion on the importance of having a quality-minded culture, which is foundational for any of the methods to deliver high-quality software. This includes the ability to be open about discussing defects.

To me, the interviews were interesting, but some of the comments were a little concerning. However, I realize they are the opinions of agile practitioners and I see quite a bit of disagreement between experts at the conferences, so I really didn’t put a lot of weight in the interviews.

One nitpick I have is that on the topic of retrospectives, the foundational book, “Project Retrospectives” by Norm Kerth was not referenced. That was the book that changed the term “post-mortem” to “retrospectives” and laid the basis for the retrospective practice as we know it today.

My final thought is that the magic is not in the method. Whether using agile methods or sequential lifecycles like the waterfall, it takes more than simply following a method correctly. If there are issues such team strife, stakeholder disengagement and other organizational maladies, then agile or any other methodology will not fix that. A healthy software development culture is needed, along with a healthy enterprise culture and understanding of what it takes to build and deliver great software.

I can highly recommend this book for people who want to know the distinctions between agile methods, those that want to improve their practices and for management that may feel a disconnect in their understanding of how agile is being performed in their organization. This book is also a great standalone learning resource!

The Beginner's Mind Applied to Software Testing

Something has been bothering me for some time now as I conduct software testing training classes on a wide variety of topics - ISTQB certification, test management, test automation, and many others.

But it's not only in that venue I see the issue. I also see it in conferences where people attend, sit through presentations - some good and some not so good - and leave with comments like "I didn't learn anything new." Really?

I remember way back in the day when I chaired QAI's International Testing Conference (1995 - 2000) reading comments like those and asking "How could this be?" I knew we had people presenting techniques that were innovations at the time. One specific example was how to create test cases from use cases. At the time, it was a new and hot idea.

So this nagging feeling has been rolling around in my mind for weeks now. Then, this past week I had the need to learn more about something not related to testing at all. One article I found told of the author's similar quest. All of his previous attempts to solve the problem ended up looking similar, so he started over - again - except this time with a mindset in which he knew nothing about the subject. Then, he went on to describe the idea of the "Beginner's Mind." Then, it all clicked for me.

When I studied and practiced martial arts for about ten years, my fellow students and I learned that if we trusted in our belt color, or how many years we had been learning, we would get our butts kicked. In the martial arts, beginners and experts train in the same class and nobody complains. The experts mentor the beginners and they also perfect the minor flaws in their techniques. The real danger was when we thought we already knew what the teacher was teaching.

Now...back to testing training...

I respect that someone may have 30 years experience in software testing. However, those 30 years may be limited by working for one company or a few companies. Also, the person may only have worked in a single industry. Even if the experience is as wide as possible, you still don't have 100% knowledge of anything.

The best innovators I know in software testing are those that can take very basic ideas and combine them, or find a new twist on them and innovate a new (and better) way of doing something. But you can't do that if you think you already know it all.

In the ISTQB courses, I always tell my students, "You are going to need to 'un-learn' some things and not rely solely on your experience, as great as that might be." That's because the ISTQB has some specific ways it defines things. If you miss those nuances because you are thinking "Been there, done that," you may very well miss questions on the exam. I've seen it happen too many times. I've seen people with 30+ years of experience fail the exam even after taking a class!

So the next time you are at a conference, reading a book, attending a class, etc. and you start to get bored, adopt the beginner's mind. Look at the material, listen to the speaker and ask beginner questions, like "Why is this technique better that another one?", "Why can't you do ..... instead?", or "What would happen if I combined technique X with technique Y."

Adopt the beginner's mind and you might just find a whole new world of innovation and improvement waiting for you!

Dallas Ebola Patient Sent Home Because of Defect in Software Used by Many Hospitals

Here at Rice Consulting, we have been making the message to heathcare providers for over two years now, that the greatest risk they face is in the integration and testing of workflows to make sure electronic health records are correctly made available to everyone involved in the healthcare delivery process - all the way from patients to nurses, doctors and insurance companies. It is a complex domain and too many heathcare organizations see EHR as just a technical issue that the software vendor will address and test. However, that is not the case as demonstrated by this story.

From the Homeland Security News Wire today, October 6:

"Before Thomas Eric Duncan was placed in isolation for Ebola at Dallas’ Texas Health Presbyterian Hospitalon 28 September, he sought care for fever and abdominal pain three days earlier, but was sent home. During his initial visit to the hospital, Duncan told a nurse that he had recently traveled to West Africa — a sign that should have led hospital staff to test Duncan for Ebola. Instead, Duncan’s travel record was not shared with doctors who examined him later that day. This was the result of a flaw in the way the physician and nursing portions of our electronic health records (EHR). EHR software, used by many hospitals, contains separate workflows for doctors and nurses."

"Before Thomas Eric Duncan was placed in isolation for Ebola at Dallas’ Texas Health Presbyterian Hospital on 28 September, he sought care for fever and abdominal pain three days earlier, but was sent home. During his initial visit to the hospital, Duncan told a nurse that he had recently traveled to West Africa — a sign that should have led hospital staff to test Duncan for Ebola. Instead, Duncan’s travel record was not shared with doctors who examined him later that day.

'Protocols were followed by both the physician and the nurses. However, we have identified a flaw in the way the physician and nursing portions of our electronic health records (EHR) interacted in this specific case,” the hospital wrote in a statement explaining how it managed to release Duncan following his initial visit.

According to NextGov, EHR software used by many hospitals contains separate workflows for doctors and nurses. Patients’ travel history is visible to nurses, but such information 'would not automatically appear in the physician’s standard workflow.' As a result, a doctor treating Duncan would have no reason to suspect Duncan’s illness was related to Ebola.

Roughly 50 percent of U.S. physicians now use EHRs since the Department of Health and Human Services (HHS) began offering incentives for the adoption of digital records. In 2012, former HHS chief Kathleen Sebelius said EHRs 'will lead to more coordination of patient care, reduced medical errors, elimination of duplicate screenings and tests and greater patient engagement in their own care.' Many healthcare security professionals, however, have pointed out that some EHR systems contain loopholes and security gaps that prevent data sharing among healthcare workers.

The New York Times recently reported that several major EHR systems are built to make data sharing between competing EHR systems difficult. Additionally, a 2013 RAND Corporationstudy for the American Medical Association found that doctors felt 'current EHR technology interferes with face-to-face discussions with patients; requires physicians to spend too much time performing clerical work; and degrades the accuracy of medical records by encouraging template-generated doctors’ notes.'

Today, Dallas’s Texas Health Presbyterian Hospital has made patients’ travel history available to both doctors and nurses. It has also modified its EHR system to highlight Ebola-endemic regions in Africa. 'We have made this change to increase the visibility and documentation of the travel question in order to alert all providers. We feel that this change will improve the early identification of patients who may be at risk for communicable diseases, including Ebola,' the hospital noted."

Special on ISTQB Foundation Level e-Learning - This Week Only!

I just wanted to let you know about a special offer that is for this week only.

Before I go into that, I want you to know why I'm making this offer.

First, I know that many people have stretched training budgets and have to make every dollar count.
Second, I have over 15 e-learning courses in software testing, but the one that I thinks covers the
most information in software testing is the ISTQB Foundation Level course.

The reason I conduct and promote the ISTQB program is because it gives a well-rounded framework for building knowledge in software testing. It's great to get your team on the same page in terms
of testing terminology. It also builds credibility for testers in an organization.

ISTQB is the International Software Test Qualifications Board, a not-for-profit organization that has defined the "ISTQB® Certified Tester" program that has become the world-wide leader in the certification of competences in software testing. Over 336,000 people worldwide have been certified in this program. The ISTQB® is an organization based on volunteer work by hundreds of international testing experts, including myself.

You can learn more at www.istqb.org and about the ASTQB (American Software Testing Qualifications Board) at www.astqb.org.


I think e-Learning has the best results in preparing people to take the ISTQB Foundation Level exam because you have time to really absorb the concepts, as opposed to trying to learn everything in 3
or 4 days. Plus, you can review the material at any time. That's hard to do in a live class. I have seen people score very high on the exam after taking this course and it gets great reviews.

OK.... now for the details....

For this week only, Monday, Sept 22 through Midnight (CDT) Friday, Sept 26th I am running a special offer on ISTQB Foundation Level certification e-learning training. If you purchase the 5-team
license, you get an extra person at no extra cost - exams included!

So, if you have been thinking about getting your team certified in software testing, this is a great opportunity - an $899 value.

In addition, if you order a 5-team license or higher, I will conduct a private one-hour web meeting Q&A session with me in advance of taking the exam. Your team also gets access to the course as
long as they need it - no time limits!

All you have to do is use the code "ISTQB9922" at checkout time. You will be contacted for the names of the participants.

Payment must be by credit card or PayPal.

To see the details of the course, go to
http://riceconsulting.com/home/index.php/ISTQB-Training-for-Software-Tester-Certification/istqb-foundation-level-course-in-software-testing.html

To learn more about the e-learning program, go to
http://riceconsulting.com/home/index.php/Table/e-Learning-in-Software-Testing/.

To register, go to https://www.mysoftwaretesting.com/ISTQB_Foundation_Level_Course_in_Software_Testing_p/istqb5.htm

Any questions? Just respond to this e-mail.

Act fast, because this deal goes away at Midnight (CDT) Friday,
Sept 26th!

Thanks!

The Software Tester's Greatest Asset

I interact with thousands of testers each year. In some cases, it's in a classroom setting, in others, it may be over a cup of coffee. Sometimes, people dialog with me through this blog, my website or my Facebook page.

The thing I sense most from testers that are "stuck" in their career or just in their ability to solve problems is that they have closed minds to other ways of doing things. Perhaps they have bought into a certain philosophy of testing, or learned testing from someone who really wasn't that good at testing.

In my observation, the current testing field is fragmented into a variety of camps, such as those that like structure, or those that reject any form of structure. There are those that insist their way is the only way to perform testing. That's unfortunate - not the debate, but the ideology.

The reality is there are many ways to perform testing. It's also easy to use the wrong approach on a particular project or task. It's the old Maslow "law of the instrument" that says, "I suppose it is tempting, if the only tool you have is a hammer, to treat everything as if it were a nail."

Let me digress for a moment...

I enjoy working on cars, even though it can be a very time-consuming, dirty and frustrating experience. I've been working on my own cars for over 40 years now. I've learned little tricks along the way to remove rusted and frozen bolts. I have a lot of tools - wrenches, sockets, hammers...you name it. The most helpful tool I own is a 2-foot piece of pipe. No, I don't hit the car with it! I use it for leverage. (I can also use it for self defense, but that's another story.) It cost me five dollars, but has saved me many hours of time. Yeah, a lowly piece of pipe slipped over the end of a wrench can do wonders.

The funny thing is that I worked on cars for many years without knowing that old mechanic's trick. It makes me wonder how many other things I don't know.

Here's the challenge...

Are you open to other ways of doing things, even if you personally don't like it?

For example, if you needed to follow a testing standard, would that make you storm out of the room in a huff?

Or, if you had to do exploratory testing, would that cause you to break out in hives?

Or, if your employer mandated that the entire test team (including you) get a certification, would you quit?

I'm not suggesting you abandon your principles or beliefs about testing. I am suggesting that in the things we reject out of hand, there could be just the solution you are looking for.

The best thing a tester does is to look at things objectively, with an open mind. When we jump to conclusions too soon, we may very well find ourselves in a position where we have lost our objectivity.

As a tester, your greatest asset is an open mind. Look at the problem from various angles. Consider the pros and cons of things, realizing that even your list of pros and cons can be skewed. Then, you can work in many contexts and also enjoy the journey.

The Value of Checklists

For many years, I have included checklists on my Top 10 list of test tools (I also include "your brain"). Some people think this is ridiculous and inappropriate, but I have my reasons. I'm also not the only one who values checklists.

Atul Gawande makes a compelling case for checklists, especially in critical life-or-death situations in his book "The Checklist Manifesto." In reviewing the book on Amazon.com, Malcom Gladwell writes, "Gawande begins by making a distinction between errors of ignorance (mistakes we make because we don't know enough), and errors of ineptitude (mistakes we made because we don’t make proper use of what we know). Failure in the modern world, he writes, is really about the second of these errors, and he walks us through a series of examples from medicine showing how the routine tasks of surgeons have now become so incredibly complicated that mistakes of one kind or another are virtually inevitable: it's just too easy for an otherwise competent doctor to miss a step, or forget to ask a key question or, in the stress and pressure of the moment, to fail to plan properly for every eventuality."

Gladwell also makes another good point, "Experts need checklists--literally--written guides that walk them through the key steps in any complex procedure. In the last section of the book, Gawande shows how his research team has taken this idea, developed a safe surgery checklist, and applied it around the world, with staggering success."

In testing, we face similar challenges in testing all types of applications - from basic web sites to safety-critical systems. It is very easy to miss a critical detail in many of the things we do - from setting up a test environment to performing and evaluating a test.

I have a tried and true set of checklists that also help me to think of good tests to document and perform. It is important to note that a checklist leads to tests, but are not the same as test cases or the tests they represent.

I have been in some organizations where just a simple set of checklists would transform their test effectiveness from zero to over 80%! I even offer them my checklists, but there has to be the motivation (and humility) to use them correctly.

Humility? Yes, that's right. We miss things because we get too sure of ourselves and think we don't need something as lowly, simple and repetitive as a checklist.

Checklists cost little to produce, but have high-yield in value. By preventing just one production software defect, you save thousands of dollars in rework.

And...your checklists can grow as you learn new things to include. (This is especially true for my travel checklist!) So they are a great vehicle for process improvement.

Checklists can be great drivers for reviews as well. However, many people also skip the reviews. This is also unfortunate because reviews have been proven to be more effective than dynamic testing. Even lightweight peer reviews are very effective as pointed out in the e-book from Smartbear, Best Kept Secrets of Peer Code Reviews.

Now, there is a downside to checklists. That is, the tendency just to "check the box" without actually performing the action. So, from the QA perspective, I always spot check to get some sense of whether or not this is happening.

Just as my way of saying "thanks" for reading this, here is a link to one of my most popular checklists for common error conditions in software.

I would love to hear your comments about your experiences with checklists.

ISTQB Foundation Level Training in Software Testing (CTFL) - Chicago - Sept 16 - 18, 2014

If you are looking for ISTQB Foundation Level Software Testing Training at a reasonable price, in the Chicago area, this is your opportunity!

The class is Sept. 16 - 18, 2014, held in The Regus Offices at One Dearborn St. in downtown Chicago.

The instructor is Thomas Staab, CTFL.

We are only going to hold registrations open for one more week! We will close registrations at 5 p.m. CDT on Wed, Aug 27th.

The class is limited to 10 people. So if you want to get in on this unique opportunity - act fast!

The cost for the class is $2,000 per person, with the exam ($250 value) included in that price. The exam will be given on the final afternoon of the class.

If you bring two people, you can get $500 off your registrations. Just use code "ISTQB916" at check-out
time.

You can register here: https://www.mysoftwaretesting.com/ISTQB_Foundation_Level_Course_in_Software_Testing_p/ctflpub.htm

To see the outline: http://riceconsulting.com/home/index.php/ISTQB-Training-for-Software-Tester-Certification/istqb-foundation-level-course-in-software-testing.html 

I hope you can make it!

Randy

ISTQB Advanced Test Manager Training - Live Online Sept. 15 - 19

I am offering a special live, online version of the ISTQB Advanced Test Manager Training the week of September 15 - 19, 2014. I'm also offering a special "buy 2 get 1 free" offer. Just use coupon code "ISTQB915" at https://www.mysoftwaretesting.com/ISTQB_Advanced_Test_Manager_Course_Public_Course_p/ctaltmpub.htm

Be sure to register as a remote attendee.

I will be the instructor and the class schedule will be from 8:30 a.m. to 4:00 p.m. CDT each day. You will get a complete set of printed course notes if you register at least one week in advance of the class. The sessions will be recorded and posted daily, so if you have miss a portion you can catch up.

I hope to see you there!

The Role of a Test Architect

I get some really good questions by e-mail and this is one of them.

Q: What does a QA Architect do in a team, and what skills are needed for this job?

A; To me, (although different organizations will have different definitions) the test architect is a person who understands testing at a very in-depth level and is responsible for things such as:

• Designing test frameworks for test automation and testing in general
• Directing and coordinating the implementation of test automation and other test tools
• Designing test environments
• Providing guidance on the selection of the most effective test design techniques in a given situation
• Providing guidance on technical types of testing such as performance testing and security testing
• Designing methods for the creation of test data
• Coordinating testing with release processes

This is a "big picture" person who also understands software development and can work with developers to ensure that the test approaches align with development approaches. So, the test architect should be able to understand various project life cycles. These days, a test architect needs to understand the cloud, SOA and mobile computing.

The terms "architect" in many organizations implies a very deep level of understanding and is a highly respected position. The expectations are pretty high. The architect can provide guidance on things that the test manager may not have technical knowledge about. Yet, the test architect can focus on testing and not have to deal with administrative things that managers deal with.

Follow-up question: "What kind of company is more likely to have such a role? A large company or a smaller company? When you say "directing and coordinating" sounds like communicating across teams, like QA, dev, dev ops, DBA to get things done."

A: I would say that you would likely find the role in companies that truly understand the role and value of testing. For example, they know that QA does not equal testing. It would be an interesting project to research how many companies in a sample group would have test architect as a defined role. I would tend to think of larger companies being more likely to have a test architect, but I've seen smaller software companies with test architects and larger companies with no one in any type of test leadership or designer role.

Another indication might be those companies that have a more centralized testing function, such as testing center of excellence. I have some misgivings about the COE approach in that they often fail because people see them as little bureaucracies instead of support. Also, they tend to try to "take over the world" in a company in terms of test practice. The lifespan of a testing COE is often less than 2 years from what I have seen. It's good money for testing consultants to come in and establish the COE, but then they leave (or get asked to leave) and the energy/interest goes away.

And...the company would need to see the need for both functional and technical testing. You need a test architect to put those pieces together.

This is not "command and control" but rather design and facilitation. And you are right, the test architect role touches many other areas, tasks and people.

Follow-up Question: What kind of tools? I'm assuming you're talking more than handy shell scripts. Simulators? Rest clients like postman customizations?

A: Right, the tools are typically high-end commercial tools, but there is a trend toward open source tools and frameworks. One of my friends calls the big commercial tool approach "Big Pharma". The key is that the test architect knows how to define a framework where the tools can work together. This can include the customized homegrown tools as well. Those can be handy.

By the way, the term "framework" can have many meanings. The way I'm using the term here is a structure for test design and test automation that allows functional testers to focus on understanding the application under test and design (and implement) good tests, with the technical testers building and maintaining the technical side of automation.

We also have to expand the view to include not only test automation, but other support tools as well, such as incident management, configuration management and others. For example, there is a great opportunity to use tools to greatly reduce the time needed for test design. Tools such as ACTS (from nist.gov) and Hexawise can be helpful for combinatorial testing, test data generators are needed for test automation and performance testing, and I'm especially keen on model-based testing when the model is used to generate tests (not the manual approach). I think BenderRBT does a great job in designing tests from requirements. Grid Tools has recently introduced a tool called Agile Designer to design tests based on workflow. I'll have more information on that in an upcoming post.

What Does it Take to Become a Test Architect?

I suppose one could take many paths. However, I would not automatically assume someone in an SDET (Software Developer in Test) would be qualified. That's because more than just the technical perspective is needed. The test architect also needs to understand the business processes used in the organization. Personally, I think people at this level, or who aspire to it, would profit by studying Enterprise Architecture. My favorite is the Zachman Enterprise Architecture Framework.

I would look for:

1. Knowledge and understanding at a deep level - Not the superficial knowledge that most people never get past. This means that they know about metrics, code complexity, reliability modeling, performance modeling, security vulnerabilities - all at an advanced to expert level. This is why I encourage people who are on the certification path to go beyond foundation and on to advanced and expert levels of training. This also includes being a continuous learner and reading good books in the testing field and related disciplines. I would start with Boris Beizer's "Software Testing Techniques, 2nd Ed."

2. Meaningful experience - At the risk of just putting an arbitrary number of years as the baseline, I would think at least eight to ten years of solid test design, tool application and perhaps software design and development would be needed. You need a decent portfolio of work to show you have what it takes to work in the role.

3. Great interpersonal skills - The test architect has to negotiate at times and exert influence to get their ideas across. They have to get along with others to function as part of the larger organizational team. Of course, this also includes developers, test managers and development architects. Just because you are a guru doesn't mean you have to be a stubborn and contentious jerk.

4. Objectivity - When choosing between alternative approaches and tools, objectivity is needed.

5. Problem Solving - This requires a creative approach to solving problems and seeing challenges from different angles. It's not uncommon for solutions to be devised where no one has gone before.

I hope this helps raise the awareness of this important role.

Questions or comments? I'm happy to address them!

Randy

Live Online Classes for Software Quality

Rice Consulting has teamed with Wind Ridge International Consulting to offer the following courses live, online.


These are all short daily online events, spread over a week. Five sessions, 1.5 hours each. 

To learn more about a class, just click on the title or the "More Info" link. To see the specific dates and times, prices, and to register, click on the "Register" link below each description above. 

Building a High Performance Team
A key role of all successful managers is the building and leading a high performance team (HPT) that provides business value to the organization.  The HPT plays a central role in any software organization. Therefore, they need to have a unique combination of well-developed technical, communication, leadership, and people skills in order to be successful. This class focuses on the unique challenges the software organizations faces in building, leading, and retaining a HPT. More Info...

REGISTER HERE

Establishing the Software Quality Organization

This class describes the activities required to establish a successful software quality organization (SQO).  The SQO is comprised of three distinct quality functions, quality assurance, quality control (testing), and configuration management, that work in harmony and complement each other. More Info...

REGISTER HERE

Fundamentals of Process Improvement

There are two ways to make process improvement changes: random or planned.  You can’t start making improvements until you know your exact current state and determine where you want to go.  This class takes the students through a step-by-step proven process for making planned improvements. More Info... 

REGISTER HERE

Introduction to Configuration Management
This class introduces the attendees to the basics of CM as well as a practical process for establishing and maintaining a CM program that meets the corporate goals and information needs. More Info...  

REGISTER HERE


The instructor for these classes is Thomas Staab. Tom is a frequent advisor to numerous companies and government agencies on the topic of technology management, business/technology interface, resource and process optimization, project management, working with the multi-generational/ multi-cultural workforce, successful outsourcing, and maximizing business value and ROI.
Tom specializes in improving processes, managing projects, and maximizing results with the goal of enabling senior management, business and technology leaders to form a cohesive unit and become even more successful corporate and government leaders.
He quickly gains the full and active support from senior leaders down to the front line managers. His mission is to increased business value and produce significant return-on-investment (ROI) for clients.

Software Testing Careers - Should I Stay or Should I Go?

At the recent Better Software Conference, I really enjoyed a presentation by James Whittaker in which he gave some keys to succeeding as a tester, or any career, for that matter. The short list is:
1. Ambition
2. Passion
3. Specialization
4. Imitation of mentors
5. Re-invention
6. Clairvoyance - Keep abreast of the industry. Read!
7. Leadership
8. Find your "super powers" (Things you do really, really well)

That got me thinking about this topic again, because my passion is to help people have better lives, not just be better testers. If you are a great tester stuck in a bad company, working for a clueless boss, etc., then you will be miserable. That leads to an unhappy life everywhere outside of work. Believe me, I know.

It also caused me to rethink things I have taught in the past - especially the idea that you can thrive even in a dysfunctional environment.

Some of you reading this are fortunate to be in a great company, working for a great manager and most days are sunny. However, I talk with thousands of testers every year and I think the great majority of them are in frustrating situations.

One of my most popular "lightning talks" is "Ten Proven Ways to Demotivate Your Team." I included those slides as an opener for my keynote at StarEast 2014. Instead of being funny, I think it evoked serious feelings. I guess that's how stand-up comedy works - sometimes the material works and sometimes, it bombs.

After the StarEast session, I had several people pull me aside to ask me how to deal with their dysfunctional workplace issues. So I decided to write here about some ways to survive terrible workplaces and go on to thrive in your career.

I really thought about these points and got feedback from others in the QA and test profession. Their consensus was the same as mine.

1. The main thing is that you must see yourself as a free agent. The idea that a company will take care of you until retirement died about 25 years ago. You have to develop yourself, invest in yourself, and be prepared to work in many situations - like a sports player or a musician can go from team to team or band to band. Unfortunately, corporations have destroyed trust by outsourcing and reorganizations that have devalued people and the value they bring to the company.

2. It's one thing to have a bad manager, but another to be in a company that has a terrible culture. The problem is that even if your manager leaves or gets fired, you are still in the same cesspool as before. No matter how the team structure changes, you are still in a no-win situation. Even if you became the new manager, you will be up against the same people who make the same bad, misguided policies as before.

3. You are responsible for developing your own skills. It's sad, but too many companies don't want to invest in their people when it comes to training or skill building. They are a) tight-fisted with money and b) afraid that people will benefit from the training by leaving the company afterwards. This just shows that management knows the culture is bad, but they don't care enough to fix it. Or, they don't want to take the risk to take a few positive steps, like training.

4. You have to know "when to hold them, know when to fold them." Leaving a job has risks, just as taking a job has risks. Try your best to do your home work by talking to current employees you may have access to. It's not a sign of failure to leave a job. In fact, sometimes that's the only way to get ahead. Just be careful to not appear as a "job hopper." That can be a career killer. When explaining to a prospective employer why you are considering a new position, you want to stay on the positive side. It's risky to be critical of your present situation. You can say that you feel that your skills and talents are being underused and you feel like you can make a greater contribution someplace else. Back when I had a "real job" I took the high road and thanked the employer I was leaving. That paid off.

I think I have a nomination for the worst treatment ever. It happened recently to one of my sons who had just taken a new job and got married. The company gave him an expensive gift - worth about $300. While on his honeymoon, they fired him (He didn't find out until he opened his mail upon returning home). Then, they deducted the cost of the gift from his final paycheck.

There are no perfect answers on this issue. You have to carefully consider the options, but just keep in mind that toxic environments will eventually wear you down. As much as we like to say we can separate work life from personal life, it's practically impossible to do because we are emotional beings.

Do yourself a favor and build a career, not a job.

I hope this helps and I would love to hear your comments.

Randy

Secrets of Successful Assessments

Thanks to everyone who attended the webinar on "Secrets of Successful Assessments" that Tom Staab and I presented yesterday.

Here are the items from the webinar:

Recording of the session is at http://youtu.be/7Tk04F4NU5Y
(By the way, I have other videos there on my channel. Please subscribe to it if you want notifications of when I post new ones. We have 80 subscribers so far!)

The assessment checklist is at: http://www.riceconsulting.com/assessment_checklist.docx

The slides are at: http://www.softwaretestingtrainingonline.com/cc/public_pdf/Secrets%20of%20Successful%20Software%20Assessments%2005152014%20-%20RR.pdf


Congratulations to the winner of my book, Testing Dirty Systems, Emma Ball from Columbus, OH!


Have a great Memorial Day weekend and don't forget top honor those who have fought and died for our country!

Randy

Slides from Defect Sampling Presentation at ASTQB Conference.

Hi all,

Thanks to everyone at my presentation at the ASTQB Conference on defect sampling.

Here are my slides from today's presentation:

http://www.softwaretestingtrainingonline.com/cc/Slide%20Shows/Defect%20Sampling%20-%20ASTQB.pdf

Here is the video from Gold Rush:

http://youtu.be/0XXzdNma7O4

Enjoy!

Slides from Free and Cheap Tools Tutuorial at ASTQB Conference

Here are the slides from today's presentation at the ASTQB conference:

http://www.softwaretestingtrainingonline.com/cc/Slide%20Shows/Cheap%20and%20Free%20Test%20Tools%20-%20Rice%20-%20ASTQB.pdf

http://www.softwaretestingtrainingonline.com/cc/Slide%20Shows/Cheap%20and%20Free%20Test%20Tools%20-%20Rice%20-%20ASTQB.pptx

Webinar Video and Slides - What to Automate First?

Hi Folks,

Thanks for attending the webinar today on What to Automate First - The Low-Hanging Fruit of Test Automation. If you missed it, here is the video:

http://youtu.be/eo66ouKGyVk

Here are the slides:

http://www.slideshare.net/rrice2000/what-do-we-automate-first

I hope you enjoy the content!

Is Software Testing Once Again a "Concept Sale?"

In marketing there is a thing known as a "concept sale." That is, before you can even discuss features or price, you have to make the case for WHY the item or service is needed.

Back in the day (1990 - 1995), test automation tools were a concept sale. People had to be shown in very clear and compelling terms what test automation was and how it was helpful. After a few years, people understood the need and value, so the questions started moving toward, "Which type of test tool(s) do we need?"

Based on what I see in terms of software quality overall and hear in conversations with c-level execs, I get the feeling that software testing is going down the same path as other quality-related practices, such as six-sigma, TQM, requirements engineering, standards, etc.  Yes, I fear that software testing is quickly becoming a "concept sale."

What used to be considered an essential part of the software development life cycle, is now in danger of becoming a secondary and optional part of the life cycle. It's certainly not that way in all companies, but I see a clear trend emerging that companies are willing to take huge risks in releasing largely untested software.

It seems that management knows that testing is needed, but it is a luxury. The attitude seems to be that defects found "in the wild" can be fixed cheaper and faster than before release. Risk is secondary to release dates.

Does that mean that "testing is dead" after all? Well, it reminds me of the scene from Monty Python and the Holy Grail, where some people are trying to put a man on a cart, but he keeps saying "I'm not quite dead yet."

I have no easy answers on this. All I can say is to: 1) keep showing your value by being an information provider of things beyond defects and failures, 2) be professional and continue to work on your skills (get better at providing feedback, learn how to notice details better, become a creative thinker, etc.), and 3) keep making the message that testing is a lifecycle activity, integrated into a development process. (I do think it is significant that the concept of a software lifecycle is also a confusing and elusive topic for many companies.)

The particular method of developing software isn't the issue. The same thinking goes for architecture, requirements, design and all the other disciplines that make solid software. The magic is not in the method. The magic is to understand user needs, deliver to those needs in value-added ways, and know the solution actually works by testing it!

Don't assume that everyone understands why testing is needed.  Be prepared to show the value of your testing at a moment's notice. Dashboards are a good way to do that. In fact, I have a presentation on this posted on YouTube and Slideshare.net.

You never know when you'll have to make a concept sale for testing!

I would be interested in hearing your opinions on this.

Thanks,

Randy

Tester to Developer Webinar Slides Posted

Hi everyone,

Thanks for attending the webinar today and for all the great conversation. Here are the slides:

http://www.softwaretestingtrainingonline.com/cc/library/The%20Elusive%20Tester%20to%20Developer%20Ratio2014.pdf

The video will be posted very shortly.

Thanks!

Randy

Metrics for User Acceptance Testing

Recently I responded to a question at Quora.com about UAT metrics.

"What user acceptance testing metrics are most crucial to a business?"

Here is an expanded version of my answer, with some caveats.

The leading caveat is that you have to be very careful with metrics because they can drive the wrong behavior and decisions. It's like the unemployment rate. The government actually publishes several rates, each with different meanings and assumptions. The one we see on TV is the usually the lowest one, which doesn't factor in the people who have given up looking for work. So, the impression might be the unemployment situation is getting better, while the reality is a lot of people have left the work force or may be under-employed.

Anyway, back to testing...

If we see metrics as items on a dashboard to help us drive the car (of testing and of projects), that's fine as long as we understand that WE have to drive the car and things happen that are not shown on the dashboard.

Since UAT is often an end-project activity, all eyes are on the numbers to know if the project can be deployed on time. So there may be an effort my some stakeholders to make the numbers look as good as possible, as opposed to reflecting reality.

With that said...

One metric I find very telling is how many defects are being found per day or week. You might think of this as the defect discovery velocity. These must be analyzed in terms of severity. So, 10 new minor defects may be more acceptable than 1 critical defect. As the deadline nears, the number of new, critical, defects gains even more importance.

Another important metric is the number of resolved/unresolved defects. These must also be balanced by severity and should be reflected in the acceptance criteria. Be aware, though, that it is common (and not good) practice to reclassify critical defects as "moderate" to release the system on time. Also, keep in mind that you can "die the death of a thousand paper cuts." In other words, it's possible to have no critical issues, but many small issues that render the application useless.


Acceptance criteria coverage is another key metric to identify which criterion have and have not been tested. Of course, proceed with great care on this metric as well. Just because a criterion has been tested doesn't mean it was tested well, or even passed the test. In my Structured User Acceptance Testing course, we place a lot of focus of testing on the business processes, not just a list of acceptance criteria. That gives a much better idea of validation and whether or not the system will meet user needs in the real world.

Finally, stakeholder acceptance is the ultimate metric. How many of the original acceptance criteria have been formally accepted vs. not accepted. It may be the case where just one key issue holds up the entire project.

As far as business value is concerned, a business must see the value in UAT and the system to be released. Here is an article I wrote that address the value of software quality: The Cost of Software Quality - A Powerful Tool to Show the Value of Software Quality.

I hope this helps and I would love to hear about any metrics for UAT you have found helpful.

Thanks,

Randy

Tester to Developer Ratio Survey

I have a new survey posted for the Tester to Developer Ratio topic. If you have 5 minutes to answer it, I would appreciate it!

http://freeonlinesurveys.com/s.asp?sid=zikft6dtg1mueho417711

Free Webinar - February 20 2014 - The Elusive Tester to Developer Ratio

You are invited to attend a FREE 30-minute (or so) webinar on Thursday, February 20th, 12:00 CST.

Since 2000, I have been researching the question, "What is the recommended ratio of software testers to developers?" I have written two articles on that topic, with the original article, "The Elusive Tester to Developer Ratio" getting over 30,000 hits on my web site and being cited in many other articles and books.

 

This is an important metric, but it also raises other important questions, such as:

 

• What if your needs are different from "average"?
• Is this metric really the best way to plan the staffing of a test organization?
• What are other, perhaps better, ways to balance your workload?
• How can small test teams be successful, even in large development organizations?

 

I will also present up-to-date research.

To sign-up, just go http://www.anymeeting.com/PIID=EA52DA86864F3C (You will get automatic reminders beforehand.)

There are limited slots available, so be sure and sign-up and show-up early to reserve your place. (The last time we had a completely full session.) We will be recording the session and post it a little later on my YouTube channel.

Feel free to pass this invitation along to a friend!

I hope to see you there!

Thanks,

Randy Rice

Revising an Important Software Testing Course

You would think after writing over 60 software testing courses, I would finally get tired, get bored or something. Yet, new topics interest me, so I keep going.

Another challenge is maintaining all this courseware. Thankfully, I have some friends like Tom Staab and Tauhida Parveen for their help in this effort.

The course we are revising is Security Testing for the Enterprise and the Web. The need has never been greater for security testing, and so many organizations place too much trust in their security policies and procedures. The bad guys don't give a rip about policies and procedures. They are out to exploit software defects to give them greater leverage.

As with any software testing course, the key is application. So, we are revising this course with updated examples and exercises using recent exploits as examples.

We'll have it ready soon, so stay tuned for details.

Resources from "How to Test Without Defined Requirements" Webinar

Here are the files from today's webinar on How to Test Without Defined Requirements. 

Thanks for viewing!


Video: http://youtu.be/RW4FiM2RjxY

(I re-recorded the session due to audio problems during the webinar. I have discovered that the cause of the dropped audio was a DOS attack against the webinar's audio service provider.)

Slide Handouts
1 slide/page:

http://www.softwaretestingtrainingonline.com/cc/public_pdf/Testing%20Without%20Defined%20Requirements.pdf

2 slides/page:

http://www.softwaretestingtrainingonline.com/cc/public_pdf/Testing%20Without%20Defined%20Requirements%202-up.pdf

Other Resources Mentioned:

The article “How to Test Without Defined Requirements” at http://riceconsulting.com/home/index.php/General-Testing/testing-without-defined-requirements.html

Free graphing tools

Yed - http://www.yworks.com
Gliffy - http://www.gliffy.com
FreeMind - http://freemind.sourceforge.net/wiki/index.php/Main_Page
Xmind - http://www.xmind.net/
CTE-XL - http://www.berner-mattner.com

Checklists and Functional definition

Checklists - http://www.riceconsulting.com/public_pdf/ERROR_CONDITIONS_TESTING_CHECKLIST.docx
METS spreadsheets - http://www.riceconsulting.com/public_pdf/METS_Worksheets.xls
Or http://www.gregpaskal.com

MindMap - http://www.riceconsulting.com/public_pdf/Methods.pdf
http://www.riceconsulting.com/public_pdf/Methods.mm

Great Deal on ASTQB Conference registration

The earlybird deadline has passed. Or has it? Here is a code good for 24 hours to get 10% off your ASTQB Conference registration: astqb2014h9

Register now and join us in beautiful San Francisco this March 24-26! Build your skills, your professional connections, and your career at the ASTQB Conference:

1. Learn how the ASTQB Conference can help your software testing career.


2. Register for the ASTQB Conference now.


3. Certification isn't required to attend the conference, but we are offering ISTQB Certification exams on March 24th. If you're interested, register for the Foundation Exam or Advanced Exam.

Improve your testing. Improve your career. Learn more now.

I'll be there speaking on Free and Cheap Test tools and a track session on Defect Sampling. I hope to see you there!

Google Hummingbird and Other Stuff

As a follow-up to my last post about Google Hummingbird, it's becoming more clear what Google is really trying to do with all of their apps and services.  They want Google+ to be king over Facebook and other social sites. Problem is, the adoption just isn't there. So, they integrate Google+ will other stuff they own, like YouTube. The search results you see when you "Google" something are based on where you live, where you have been on the web, your activity on Google+ and all kinds of behavior detectors.

Here's an article that I totally agree with:

Sorry, Google+, We Still Won't Come to Your Party.

But, here's the problem. People don't like this level of privacy invasion. I know I don't.

So, the big goal for Google is to own it all by owning all the knowledge about you and your activities. This is attractive for businesses, because this level of information is normally very expensive, but anyone with a website can get it free with Google Analytics.

All I know is that the more I see of this direction, the less I like.

Here's an interesting test. Try the same search terms in Google, Bing and Yahoo and notice how the Google-affiliated sites seems to float higher. That's cool if you are in the "Google party", but what if you are just looking for helpful information?

More to come...