For many years, I have included checklists on my Top 10 list of test tools (I also include "your brain"). Some people think this is ridiculous and inappropriate, but I have my reasons. I'm also not the only one who values checklists.
Atul Gawande makes a compelling case for checklists, especially in critical life-or-death situations in his book "The Checklist Manifesto." In reviewing the book on Amazon.com, Malcom Gladwell writes, "Gawande begins by making a distinction between errors of ignorance
(mistakes we make because we don't know enough), and errors of
ineptitude (mistakes we made because we don’t make proper use of what
we know). Failure in the modern world, he writes, is really about the
second of these errors, and he walks us through a series of examples
from medicine showing how the routine tasks of surgeons have now become
so incredibly complicated that mistakes of one kind or another are
virtually inevitable: it's just too easy for an otherwise competent
doctor to miss a step, or forget to ask a key question or, in the stress
and pressure of the moment, to fail to plan properly for every
eventuality."
Gladwell also makes another good point, "Experts need checklists--literally--written guides that walk them
through the key steps in any complex procedure. In the last section of
the book, Gawande shows how his research team has taken this idea,
developed a safe surgery checklist, and applied it around the world,
with staggering success."
In testing, we face similar challenges in testing all types of applications - from basic web sites to safety-critical systems. It is very easy to miss a critical detail in many of the things we do - from setting up a test environment to performing and evaluating a test.
I have a tried and true set of checklists that also help me to think of good tests to document and perform. It is important to note that a checklist leads to tests, but are not the same as test cases or the tests they represent.
I have been in some organizations where just a simple set of checklists would transform their test effectiveness from zero to over 80%! I even offer them my checklists, but there has to be the motivation (and humility) to use them correctly.
Humility? Yes, that's right. We miss things because we get too sure of ourselves and think we don't need something as lowly, simple and repetitive as a checklist.
Checklists cost little to produce, but have high-yield in value. By preventing just one production software defect, you save thousands of dollars in rework.
And...your checklists can grow as you learn new things to include. (This is especially true for my travel checklist!) So they are a great vehicle for process improvement.
Checklists can be great drivers for reviews as well. However, many people also skip the reviews. This is also unfortunate because reviews have been proven to be more effective than dynamic testing. Even lightweight peer reviews are very effective as pointed out in the e-book from Smartbear, Best Kept Secrets of Peer Code Reviews.
Now, there is a downside to checklists. That is, the tendency just to "check the box" without actually performing the action. So, from the QA perspective, I always spot check to get some sense of whether or not this is happening.
Just as my way of saying "thanks" for reading this, here is a link to one of my most popular checklists for common error conditions in software.
I would love to hear your comments about your experiences with checklists.
Thursday, August 28, 2014
Wednesday, August 20, 2014
ISTQB Foundation Level Training in Software Testing (CTFL) - Chicago - Sept 16 - 18, 2014
If you are looking for ISTQB Foundation Level Software Testing Training at a reasonable price, in the Chicago area, this is your opportunity!
The class is Sept. 16 - 18, 2014, held in The Regus Offices at One Dearborn St. in downtown Chicago.
The instructor is Thomas Staab, CTFL.
We are only going to hold registrations open for one more week! We will close registrations at 5 p.m. CDT on Wed, Aug 27th.
The class is limited to 10 people. So if you want to get in on this unique opportunity - act fast!
The cost for the class is $2,000 per person, with the exam ($250 value) included in that price. The exam will be given on the final afternoon of the class.
If you bring two people, you can get $500 off your registrations. Just use code "ISTQB916" at check-out
time.
You can register here: https://www.mysoftwaretesting.com/ISTQB_Foundation_Level_Course_in_Software_Testing_p/ctflpub.htm
To see the outline: http://riceconsulting.com/home/index.php/ISTQB-Training-for-Software-Tester-Certification/istqb-foundation-level-course-in-software-testing.html
I hope you can make it!
Randy
Sunday, August 17, 2014
ISTQB Advanced Test Manager Training - Live Online Sept. 15 - 19
I am offering a special live, online version of the ISTQB Advanced Test Manager Training the week of September 15 - 19, 2014. I'm also offering a special "buy 2 get 1 free" offer. Just use coupon code "ISTQB915" at https://www.mysoftwaretesting.com/ISTQB_Advanced_Test_Manager_Course_Public_Course_p/ctaltmpub.htm
Be sure to register as a remote attendee.
I will be the instructor and the class schedule will be from 8:30 a.m. to 4:00 p.m. CDT each day. You will get a complete set of printed course notes if you register at least one week in advance of the class. The sessions will be recorded and posted daily, so if you have miss a portion you can catch up.
I hope to see you there!
Be sure to register as a remote attendee.
I will be the instructor and the class schedule will be from 8:30 a.m. to 4:00 p.m. CDT each day. You will get a complete set of printed course notes if you register at least one week in advance of the class. The sessions will be recorded and posted daily, so if you have miss a portion you can catch up.
I hope to see you there!
Friday, August 08, 2014
The Role of a Test Architect
I get some really good questions by e-mail and this is one of them.
Q: What does a QA Architect do in a team, and what skills are needed for this job?
A; To me, (although different organizations will have different definitions) the test architect is a person who understands testing at a very in-depth level and is responsible for things such as:
The terms "architect" in many organizations implies a very deep level of understanding and is a highly respected position. The expectations are pretty high. The architect can provide guidance on things that the test manager may not have technical knowledge about. Yet, the test architect can focus on testing and not have to deal with administrative things that managers deal with.
Follow-up question: "What kind of company is more likely to have such a role? A large company or a smaller company? When you say "directing and coordinating" sounds like communicating across teams, like QA, dev, dev ops, DBA to get things done."
A: I would say that you would likely find the role in companies that truly understand the role and value of testing. For example, they know that QA does not equal testing. It would be an interesting project to research how many companies in a sample group would have test architect as a defined role. I would tend to think of larger companies being more likely to have a test architect, but I've seen smaller software companies with test architects and larger companies with no one in any type of test leadership or designer role.
Another indication might be those companies that have a more centralized testing function, such as testing center of excellence. I have some misgivings about the COE approach in that they often fail because people see them as little bureaucracies instead of support. Also, they tend to try to "take over the world" in a company in terms of test practice. The lifespan of a testing COE is often less than 2 years from what I have seen. It's good money for testing consultants to come in and establish the COE, but then they leave (or get asked to leave) and the energy/interest goes away.
And...the company would need to see the need for both functional and technical testing. You need a test architect to put those pieces together.
This is not "command and control" but rather design and facilitation. And you are right, the test architect role touches many other areas, tasks and people.
A: Right, the tools are typically high-end commercial tools, but there is a trend toward open source tools and frameworks. One of my friends calls the big commercial tool approach "Big Pharma". The key is that the test architect knows how to define a framework where the tools can work together. This can include the customized homegrown tools as well. Those can be handy.
By the way, the term "framework" can have many meanings. The way I'm using the term here is a structure for test design and test automation that allows functional testers to focus on understanding the application under test and design (and implement) good tests, with the technical testers building and maintaining the technical side of automation.
We also have to expand the view to include not only test automation, but other support tools as well, such as incident management, configuration management and others. For example, there is a great opportunity to use tools to greatly reduce the time needed for test design. Tools such as ACTS (from nist.gov) and Hexawise can be helpful for combinatorial testing, test data generators are needed for test automation and performance testing, and I'm especially keen on model-based testing when the model is used to generate tests (not the manual approach). I think BenderRBT does a great job in designing tests from requirements. Grid Tools has recently introduced a tool called Agile Designer to design tests based on workflow. I'll have more information on that in an upcoming post.
What Does it Take to Become a Test Architect?
I suppose one could take many paths. However, I would not automatically assume someone in an SDET (Software Developer in Test) would be qualified. That's because more than just the technical perspective is needed. The test architect also needs to understand the business processes used in the organization. Personally, I think people at this level, or who aspire to it, would profit by studying Enterprise Architecture. My favorite is the Zachman Enterprise Architecture Framework.
I would look for:
1. Knowledge and understanding at a deep level - Not the superficial knowledge that most people never get past. This means that they know about metrics, code complexity, reliability modeling, performance modeling, security vulnerabilities - all at an advanced to expert level. This is why I encourage people who are on the certification path to go beyond foundation and on to advanced and expert levels of training. This also includes being a continuous learner and reading good books in the testing field and related disciplines. I would start with Boris Beizer's "Software Testing Techniques, 2nd Ed."
2. Meaningful experience - At the risk of just putting an arbitrary number of years as the baseline, I would think at least eight to ten years of solid test design, tool application and perhaps software design and development would be needed. You need a decent portfolio of work to show you have what it takes to work in the role.
3. Great interpersonal skills - The test architect has to negotiate at times and exert influence to get their ideas across. They have to get along with others to function as part of the larger organizational team. Of course, this also includes developers, test managers and development architects. Just because you are a guru doesn't mean you have to be a stubborn and contentious jerk.
4. Objectivity - When choosing between alternative approaches and tools, objectivity is needed.
5. Problem Solving - This requires a creative approach to solving problems and seeing challenges from different angles. It's not uncommon for solutions to be devised where no one has gone before.
I hope this helps raise the awareness of this important role.
Questions or comments? I'm happy to address them!
Randy
Q: What does a QA Architect do in a team, and what skills are needed for this job?
A; To me, (although different organizations will have different definitions) the test architect is a person who understands testing at a very in-depth level and is responsible for things such as:
- Designing test frameworks for test automation and testing in general
- Directing and coordinating the implementation of test automation and other test tools
- Designing test environments
- Providing guidance on the selection of the most effective test design techniques in a given situation
- Providing guidance on technical types of testing such as performance testing and security testing
- Designing methods for the creation of test data
- Coordinating testing with release processes
The terms "architect" in many organizations implies a very deep level of understanding and is a highly respected position. The expectations are pretty high. The architect can provide guidance on things that the test manager may not have technical knowledge about. Yet, the test architect can focus on testing and not have to deal with administrative things that managers deal with.
Follow-up question: "What kind of company is more likely to have such a role? A large company or a smaller company? When you say "directing and coordinating" sounds like communicating across teams, like QA, dev, dev ops, DBA to get things done."
A: I would say that you would likely find the role in companies that truly understand the role and value of testing. For example, they know that QA does not equal testing. It would be an interesting project to research how many companies in a sample group would have test architect as a defined role. I would tend to think of larger companies being more likely to have a test architect, but I've seen smaller software companies with test architects and larger companies with no one in any type of test leadership or designer role.
Another indication might be those companies that have a more centralized testing function, such as testing center of excellence. I have some misgivings about the COE approach in that they often fail because people see them as little bureaucracies instead of support. Also, they tend to try to "take over the world" in a company in terms of test practice. The lifespan of a testing COE is often less than 2 years from what I have seen. It's good money for testing consultants to come in and establish the COE, but then they leave (or get asked to leave) and the energy/interest goes away.
And...the company would need to see the need for both functional and technical testing. You need a test architect to put those pieces together.
This is not "command and control" but rather design and facilitation. And you are right, the test architect role touches many other areas, tasks and people.
Follow-up Question: What kind of tools? I'm assuming you're talking
more than handy shell scripts. Simulators? Rest clients like postman
customizations?
A: Right, the tools are typically high-end commercial tools, but there is a trend toward open source tools and frameworks. One of my friends calls the big commercial tool approach "Big Pharma". The key is that the test architect knows how to define a framework where the tools can work together. This can include the customized homegrown tools as well. Those can be handy.
By the way, the term "framework" can have many meanings. The way I'm using the term here is a structure for test design and test automation that allows functional testers to focus on understanding the application under test and design (and implement) good tests, with the technical testers building and maintaining the technical side of automation.
We also have to expand the view to include not only test automation, but other support tools as well, such as incident management, configuration management and others. For example, there is a great opportunity to use tools to greatly reduce the time needed for test design. Tools such as ACTS (from nist.gov) and Hexawise can be helpful for combinatorial testing, test data generators are needed for test automation and performance testing, and I'm especially keen on model-based testing when the model is used to generate tests (not the manual approach). I think BenderRBT does a great job in designing tests from requirements. Grid Tools has recently introduced a tool called Agile Designer to design tests based on workflow. I'll have more information on that in an upcoming post.
What Does it Take to Become a Test Architect?
I suppose one could take many paths. However, I would not automatically assume someone in an SDET (Software Developer in Test) would be qualified. That's because more than just the technical perspective is needed. The test architect also needs to understand the business processes used in the organization. Personally, I think people at this level, or who aspire to it, would profit by studying Enterprise Architecture. My favorite is the Zachman Enterprise Architecture Framework.
I would look for:
1. Knowledge and understanding at a deep level - Not the superficial knowledge that most people never get past. This means that they know about metrics, code complexity, reliability modeling, performance modeling, security vulnerabilities - all at an advanced to expert level. This is why I encourage people who are on the certification path to go beyond foundation and on to advanced and expert levels of training. This also includes being a continuous learner and reading good books in the testing field and related disciplines. I would start with Boris Beizer's "Software Testing Techniques, 2nd Ed."
2. Meaningful experience - At the risk of just putting an arbitrary number of years as the baseline, I would think at least eight to ten years of solid test design, tool application and perhaps software design and development would be needed. You need a decent portfolio of work to show you have what it takes to work in the role.
3. Great interpersonal skills - The test architect has to negotiate at times and exert influence to get their ideas across. They have to get along with others to function as part of the larger organizational team. Of course, this also includes developers, test managers and development architects. Just because you are a guru doesn't mean you have to be a stubborn and contentious jerk.
4. Objectivity - When choosing between alternative approaches and tools, objectivity is needed.
5. Problem Solving - This requires a creative approach to solving problems and seeing challenges from different angles. It's not uncommon for solutions to be devised where no one has gone before.
I hope this helps raise the awareness of this important role.
Questions or comments? I'm happy to address them!
Randy
Monday, August 04, 2014
Live Online Classes for Software Quality
Rice Consulting has teamed with Wind Ridge International Consulting to offer the following courses live, online.
These are all short daily online events, spread over a week. Five sessions, 1.5 hours each.
To learn more about a class, just click on the title or the "More Info" link. To see the specific dates and times, prices, and to register, click on the "Register" link below each description above.
Building a High Performance Team
A key role of all successful managers is the building and leading a high performance team (HPT) that provides business value to the organization. The HPT plays a central role in any software organization. Therefore, they need to have a unique combination of well-developed technical, communication, leadership, and people skills in order to be successful. This class focuses on the unique challenges the software organizations faces in building, leading, and retaining a HPT. More Info...
REGISTER HERE
Establishing the Software Quality Organization
This class describes the activities required to establish a successful software quality organization (SQO). The SQO is comprised of three distinct quality functions, quality assurance, quality control (testing), and configuration management, that work in harmony and complement each other. More Info...
REGISTER HERE
Fundamentals of Process Improvement
There are two ways to make process improvement changes: random or planned. You can’t start making improvements until you know your exact current state and determine where you want to go. This class takes the students through a step-by-step proven process for making planned improvements. More Info...
REGISTER HERE
Introduction to Configuration Management
This class introduces the attendees to the basics of CM as well as a practical process for establishing and maintaining a CM program that meets the corporate goals and information needs. More Info...
REGISTER HERE
The instructor for these classes is Thomas Staab. Tom is a frequent advisor to numerous companies and government agencies on the topic of technology management, business/technology interface, resource and process optimization, project management, working with the multi-generational/ multi-cultural workforce, successful outsourcing, and maximizing business value and ROI.
Tom specializes in improving processes, managing projects, and maximizing results with the goal of enabling senior management, business and technology leaders to form a cohesive unit and become even more successful corporate and government leaders.
He quickly gains the full and active support from senior leaders down to the front line managers. His mission is to increased business value and produce significant return-on-investment (ROI) for clients.
These are all short daily online events, spread over a week. Five sessions, 1.5 hours each.
To learn more about a class, just click on the title or the "More Info" link. To see the specific dates and times, prices, and to register, click on the "Register" link below each description above.
Building a High Performance Team
A key role of all successful managers is the building and leading a high performance team (HPT) that provides business value to the organization. The HPT plays a central role in any software organization. Therefore, they need to have a unique combination of well-developed technical, communication, leadership, and people skills in order to be successful. This class focuses on the unique challenges the software organizations faces in building, leading, and retaining a HPT. More Info...
REGISTER HERE
Establishing the Software Quality Organization
This class describes the activities required to establish a successful software quality organization (SQO). The SQO is comprised of three distinct quality functions, quality assurance, quality control (testing), and configuration management, that work in harmony and complement each other. More Info...
REGISTER HERE
Fundamentals of Process Improvement
There are two ways to make process improvement changes: random or planned. You can’t start making improvements until you know your exact current state and determine where you want to go. This class takes the students through a step-by-step proven process for making planned improvements. More Info...
REGISTER HERE
Introduction to Configuration Management
This class introduces the attendees to the basics of CM as well as a practical process for establishing and maintaining a CM program that meets the corporate goals and information needs. More Info...
REGISTER HERE
The instructor for these classes is Thomas Staab. Tom is a frequent advisor to numerous companies and government agencies on the topic of technology management, business/technology interface, resource and process optimization, project management, working with the multi-generational/ multi-cultural workforce, successful outsourcing, and maximizing business value and ROI.
Tom specializes in improving processes, managing projects, and maximizing results with the goal of enabling senior management, business and technology leaders to form a cohesive unit and become even more successful corporate and government leaders.
He quickly gains the full and active support from senior leaders down to the front line managers. His mission is to increased business value and produce significant return-on-investment (ROI) for clients.
Subscribe to:
Posts (Atom)