The Seen and Unseen

I found the kind of bug this morning that I really enjoy. Well, it would be more accurate to say I missed the kind of bug I really enjoy, but found it eventually.

Yesterday we updated some of the images in our product. These images reside on our server, and are dynamically loaded when a user opens the options page in their browser. When the new images were put on the server, I made sure they were right in both the released version (since those users will see them too) and in the upcoming version that I am currently testing. Good job testing by me, right?

Wrong.

Where I did not test them was on a Mac. This was a simple change and I did not think it was necessary, especially given all the other stuff that I was testing so we could finish this release on time. My mistake. I got a midnight e-mail from my boss that there was a small white space on the bottom of two of the buttons on the Mac:

OK, not a huge deal, but something we wanted to fix right away since all our current users (who use Macs) could see this. This is where the story gets good. New graphics were created and uploaded to the server this morning. Both my boss and I checked them on Windows and Mac, and the white space was definitely gone. I gave a big thumbs-up to the change. Good job testing by me, right?

Wrong.

About ten minutes later, in the course of testing something else, I realized that in fixing the whitespace problem, somehow the names of the images got switched for the first two buttons. In other words, the first button looked like the second button should, and vice versa, and so neither one did what it said it would. This was a much worse problem, and not just because it effected Windows users too.

What are the lesson to learn from this? I'll save the testing on Mac vs PC lessons for another post (or ten). I'm also going to save the lesson about how fixing minor bugs often creates major bugs. Today's lesson is about how a software test professional such as myself (not to mention my boss) could miss the blatant fact that the buttons were totally wrong when we were looking right at them. It was because of selective attention - we were focused on whether or not there was a little white space under the buttons. In software testing, focus is a fickle mistress. You can just as easily have too much as you can not enough. That is why I like exploratory testing. By not looking for just one specific thing, you tend to see a lot more. But that is also the topic for another post.