Integration and Interoperability Testing of Software

My last post about the American Airlines baggage system failure mentioned interoperability testing and it got me thinking about this important type of testing that is often minimized or overlooked in systems testing.

First, I just need to make a distinction realizing that software testing terminology is often applied in a variety of ways depending on how you were exposed to it. For example, a test case can mean many things to different people. Although standard definitions exist, they are often unknown to professional testers.

So, here are my working definitions:

Integration testing is the verification or validation that data and control can be passed correctly between two or more related systems or system components. In other words, "Can each component or system talk to each other correctly?"

Interoperability testing is the verification or validation that data and control, once passed correctly between two or more related systems or system components, can be applied correctly to achieve the expected results or behavior. In other words, "Can the passed data or control be used correctly to get the right outcome."

The idea is that it's one thing to have integration between items, but another for that integration to be applied correctly. By the way, the definitions I used are the same or similar to that used in defense applications where integration and interoperability testing is paramount.

I consider integration testing a phase or level of testing performed once the related items or systems are available. I consider interoperability testing a type of testing, which can be performed at any phase of testing, from unit to user acceptance testing.

Here's a fictitious (but realistic) example of the two tests in action based on some of my previous rants (er.., comments). Let's say that I am passing some data to another system for sending a passenger's bag to the right destination - either the conveyor belt to be picked up, or on to their next flight. The integration test would tell me if the data made it correctly to the sorting system. The interoperability test would tell me if the sorting system processed that data correctly, made the right determination, and then sent the bags to the correct destination. It is possible that a logic error could send bags that were meant to be on a connecting flight to the conveyor belt for pickup. (NOTE: I must emphasize this is just an example and not what happened with the American problems. It may have been something similar, but the information is not available yet to know what happened.)

The lesson we can all learn is that this type of end-to-end interoperability testing is hugely important but not necessarily easy. Many times it requires validation in the actual operational environment to get a true assessment of correctness.

By the way, I offer a course in Integration and Interoperability Testing.

One more realted note - I found this link to one of my favorite articles from Scientific American on software defects. It's called "Software's Chronic Crisis". Although it's from 1994, don't let the date keep you from reading this very informative article. I'm really glad I found it!

Do you have any experiences with performing (or not performing) interoperability testing? If so, I would love to read your comments.