Lessons Learned in Test Automation Through Sudoku

For many years, I have recommended Sudoku as a mind training game for testers. I think Sudoku requires some of the same thinking skills that testers need, such as the ability to eliminate invalid possibilities, deduce correct answers and so forth.

Much like an application that may lack documentation, Sudoku only gives you a partial solution and you have to fill in the rest. Unlike software, however, guessing actually prevents you from solving the puzzle - mainly because you don't see the impact of an incorrect guess until it is too late to change it.

My friend, Frank Rowland, developed an Excel spreadsheet some time ago, that he used to help solve Sudoku puzzles by adding macros to identify cells that had only one possible correct value. At the time, I thought that was pretty cool. Of course, you still had to enter one value at a time manually. But I thought it was a good example of using a degree of automation to solve a problem.

Fast forward to last week. I was having lunch with Frank and he whips out his notebook PC and shows me the newest version of the spreadsheet. After listening to a math lecture from The Great Courses, he learned some new approaches for solving Sudoku.

Armed with this new information, Frank was successful in practically automating the solution of a Sudoku puzzle. I say "practically" because at times, some human intervention is required.

Now, I think the spreadsheet is very cool and I think that the approach used to solve the puzzle can also be applied to test automation. The twist is that the automation is not pre-determined as far as the numeric values are concerned. The numbers are derived totally dynamically.

Contrast this with traditional test automation. In the traditional approach to test automation (even keyword-driven), you would be able to place numbers in the cells, but only in a repeatable way - not a dynamic way.

In Franks's approach, the actions are determined based on the previous actions and outcomes. For example, when a block of nine cells are filled, that drives the possible values in related cells. The macros in this case know how to deduce the other possibilities and also can eliminate the invalid possibilities. In this case of "Man vs. Machine", the machine wins big time.

I don't have all the answers and I know that work has been done by others to create this type of dynamic test automation. I just want to present the example and would love to hear your experiences in similar efforts.

I think the traditional view of test automation is limited and fragile. It has helped in establishing repeatable tests for some applications, but the problem is that most applications are too dynamic. This causes the automation to fail. At that point, people often get frustrated and give up.

I've been working with test automation since 1989, so I have seen a lot of ideas come down the pike. I really like the possibilities of test automation with AI.

I hope to get a video posted soon to show more about how this works. Once again, I would also love to hear your feedback.