In everyday life, computers fail all the time in all kinds of wonderful and exciting ways. Thank goodness for this, because it keeps us testers employed. Like I always say, within these failures lies joy, at least to the destructive mind of the test professional.
That is, unless you work somewhere like NASA. When lives are on the line, I'd think that testers would feel relief at finding some nasty bugs, along with some fear about "What if we had missed this one?" or worse, "What if there is something else we missed?" That is why I have such respect for NASA testers.
That is also why I have to scoff at the person who sabotaged a computer that was about to travel on the space shuttle Endeavor to be installed in the space station. He did this by cutting a bunch of wires. What was this guy thinking? Did he really think this not-so subtle "bug" would get past the testers of NASA? We've all missed a bug or two in our days, but I mean, how poor a tester would you have to be to let this one get by you?
The only way this could happen would be if the testers went out drinking with the astronauts before running their tests. Two things about that. For one, if I was strapped into a space craft attached to giant tanks of highly explosive rocket fuel, all built by the lowest bidder, I might want to have a drink or three as well. Second, sometimes I find that having a drink before testing actually helps (six-pack test factor).
So, lets not worry too much about the astronauts knocking down a few the night before the launch. And lets really not worry at all about disgruntled contractors sabotaging the space shuttle, because we can be confident that the testers of NASA will find out. What NASA should really worry about is that their contractors are hiring people so stupid that this was the best idea they could come up with for sabotage.
Thursday, July 26, 2007
Monday, July 23, 2007
What's Behind Your Product?
There is a lesson in photography that teaches you when composing a shot, to not just pay attention to your subject, but also check the rest of the frame to make sure there is nothing lurking in the background that should not be. This will help you keep trees from growing out of people's heads, and other distractions that would make your photography less compelling.
One of the main products that I currently work on is BlueOrganizer, which is a Firefox add-on for smart browsing. Given that we run in Firefox, we are very much dependent on having a solid Firefox behind us as a platform. Last week Firefox had a minor update, 2.0.0.5. Well, it was minor for them, but it totally broke some important functionality in our product. Luckily we happened to realize this quickly, and were able to get ahack patch out to work-around the problem.
This reminded me of that photography lesson. It does not matter if your product runs in a browser, native to an OS, or on a device. As testers we tend to focus on testing our product, and ignore the foundation it runs on. We focus on our own stuff and don't think about the potential problems behind it, often until it is too late.
For us it was lucky we found the problem on the same day Firefox started to roll-out their update. It would have been better had we found it the day or even the week before. This would have saved our users some frustration and saved us from yet another fire-drill. What I should have done, and will now do in the future is to keep track of what the folks at Mozilla are up to, and work out some way to try their new releases before they are released.
What does your product rely on, and how can you keep it on the radar?
One of the main products that I currently work on is BlueOrganizer, which is a Firefox add-on for smart browsing. Given that we run in Firefox, we are very much dependent on having a solid Firefox behind us as a platform. Last week Firefox had a minor update, 2.0.0.5. Well, it was minor for them, but it totally broke some important functionality in our product. Luckily we happened to realize this quickly, and were able to get a
This reminded me of that photography lesson. It does not matter if your product runs in a browser, native to an OS, or on a device. As testers we tend to focus on testing our product, and ignore the foundation it runs on. We focus on our own stuff and don't think about the potential problems behind it, often until it is too late.
For us it was lucky we found the problem on the same day Firefox started to roll-out their update. It would have been better had we found it the day or even the week before. This would have saved our users some frustration and saved us from yet another fire-drill. What I should have done, and will now do in the future is to keep track of what the folks at Mozilla are up to, and work out some way to try their new releases before they are released.
What does your product rely on, and how can you keep it on the radar?
Tuesday, July 17, 2007
Harry Potter - Software Tester
OK, so maybe we can make a good guess about that last question and say that Harry Potter will probably not become a software tester. But what if he did? How would his life experiences prepare him for this noble profession.
Dealing With Bullies - During all the time he lived with the Dursleys, Harry had a lot of experience with being bullied. He eventually learned to stand up for himself. This is a key trait of an effective software tester. When the deadline looms and the pressure mounts, a software tester needs to stand up to anyone who tries to ship before the software is ready.
Exploration - Whether it is sneaking through secret passages to Hogsmeade, or playing with mysterious devices in Dumbledor's office, Harry has always been one to let his curiosity lead him to try new things. If he used this part of his nature to do exploratory testing, he would likely find a good deal of bugs that lie off the beaten path.
Training - Sure, Harry has lots of magical training, but I don't recall him ever taking a class on quality engineering, test automation, or dealing with developers (the last one might have been covered in Defense Against the Dark Arts). So from a training stand-point, Harry might not be the best software tester. But, didn't most of us come into testing with little or no formal training or education? If you have the knack, it is possible to learn this job on the job, so Harry might be OK. Of course, he might be able to use some spell that could give him an advantage over Muggle testers. One well placed "Repairo!" could fix any piece of shoddy software.
Hedwig - Harry has shown that he is a good communicator, which is crucial for a software tester since we deal in information. He might need to adjust his ways a bit and use e-mail or IM instead of his owl, but we have every reason to believe he will get the message through.
So, assuming Harry lives and decides to become a software tester, I'd say he has a good chance at being quite successful.
Monday, July 16, 2007
Don't Let The Bad Bugs Bite
Software testers are in the unique position of enjoying bugs, and are actively hoping to find them. That is why this blog is called Blue Screen Joy - to celebrate the rush you get when you find that truly nasty bug. I'm talking about those wonderful Severity 1, crash your system, lose your data, smoke coming out of the box kind of bugs.
Those are good bugs. I love those bugs . At least I do when I find them in something I am testing. Not so much when they happen to me outside of work.
For example, last week, when I smelled that smell that electronics make when they are burning, it did not make me excited. Unsuccessful at finding the source, I wrote the smell off as a fluke. I'll tend to do the same thing when I am testing and see a really good bug that I can't reproduce. That is what I consider a bad bug. I don't love those.
Just like with software testing, writing off non-reproducible bugs as flukes can come back to bite you. Last night, the upstairs of my house was a tropic 87 degrees Fahrenheit, despite the fact that the air-conditioner was running. The downstairs was only slightly cooler. I'm no Bob Vila, but after some investigation I discovered:
- The furnace filter was long past due for being changed
- A good portion of the air-conditioning system was frozen
Not only had I thought I had found the bug, but I also thought I had an easy (and free) solution. All I needed to do was change the filter and let the air conditioner thaw, and everything would be fine, right? Wrong. Remember that burning smell that I could not track down? The friendly neighborhood ARS man found out that the source of this smell was some high voltage wiring that led to a capacitor in the blower assembly, which is what makes air conditioning possible.
With the wires charred and the capacitor destroyed not only would my house get really hot, but I would also be unable to travel through time. Good news is that for a sum of money that made me question my career choice, the ARS man was able to fix everything and make me cool again.
The lesson here is that whether it is a bad bug you can't reproduce or a nasty smell you can't track down, you need to be careful when you write something off as a fluke. Instead, look at other seemingly unrelated problems you are seeing and try to imagine any possible connection. Failing that, come up with a Plan B. Cold beer works nicely in both situations.
Those are good bugs. I love those bugs . At least I do when I find them in something I am testing. Not so much when they happen to me outside of work.
For example, last week, when I smelled that smell that electronics make when they are burning, it did not make me excited. Unsuccessful at finding the source, I wrote the smell off as a fluke. I'll tend to do the same thing when I am testing and see a really good bug that I can't reproduce. That is what I consider a bad bug. I don't love those.
Just like with software testing, writing off non-reproducible bugs as flukes can come back to bite you. Last night, the upstairs of my house was a tropic 87 degrees Fahrenheit, despite the fact that the air-conditioner was running. The downstairs was only slightly cooler. I'm no Bob Vila, but after some investigation I discovered:
- The furnace filter was long past due for being changed
- A good portion of the air-conditioning system was frozen
Not only had I thought I had found the bug, but I also thought I had an easy (and free) solution. All I needed to do was change the filter and let the air conditioner thaw, and everything would be fine, right? Wrong. Remember that burning smell that I could not track down? The friendly neighborhood ARS man found out that the source of this smell was some high voltage wiring that led to a capacitor in the blower assembly, which is what makes air conditioning possible.
With the wires charred and the capacitor destroyed not only would my house get really hot, but I would also be unable to travel through time. Good news is that for a sum of money that made me question my career choice, the ARS man was able to fix everything and make me cool again.
The lesson here is that whether it is a bad bug you can't reproduce or a nasty smell you can't track down, you need to be careful when you write something off as a fluke. Instead, look at other seemingly unrelated problems you are seeing and try to imagine any possible connection. Failing that, come up with a Plan B. Cold beer works nicely in both situations.
Friday, July 13, 2007
QA vs. QE
Depending on where you go, software testers might be involved in Quality Assurance (QA) or Quality Engineering (QE). Each of these terms have been defined countless times, often in contradictory ways. They also tend to be used interchangeably to mean the same things.
But are they the same? If we look at the words that make up the terms, they would seem to indicate two different activities.
Quality Assurance to me implies the job of assuring that the software has some level of quality. This works great when the software is good - you can just say, "Yup, the software is good. I assure you." But what about when the software is not so good. How can you assure quality when there is none? What do you say then?
"This software stinks"?
"This software is buggier than a stagnant pond in July"?
True as these statements may be, it is not assuring anyone that it is quality software, which is what the term Quality Assurance is all about. That is why I don't like the term. To steal from the overused truism of why you should not assume, you should not assure, because it makes an ass out of you and R.E.
Quality Engineering on the other had seems to be more about a process for improving quality. Engineering is about designing and building things, so Quality Engineering should be about designing and building quality. The input could be of any level of quality, but after it goes through the QE process it should be of higher quality than when it came in. That means that in QE you not only find bugs in the product, but also in the process.
That all sounds great, especially if you are giving an talk at a software testing conference, but what does it really mean to people in the trenches. Whatever QE process your company uses, it could probably be improved. Unfortunately, we're usually too busy finding bugs, writing defect reports, and verifying fixes to have much time left for improving the process. You could move the world if you had a place to stand, right? But, how do you get to that place? My answer to that question is the same as my answer to "How do you eat an elephant?" One bite at a time. Every thing you can do to improve the process, no matter how small, should mean less bugs finding their way into test, which means more time for the Quality Engineer to make more improvements to the process.
In the end, I don't think it really pays to get all worked up about if you are doing QA, QE, or Testing. These are all just words. What matters is that you know what you are trying to accomplish and how to get there. Let people who spend more time with PowerPoint slides than bugs worry about the terminology.
But are they the same? If we look at the words that make up the terms, they would seem to indicate two different activities.
Quality Assurance to me implies the job of assuring that the software has some level of quality. This works great when the software is good - you can just say, "Yup, the software is good. I assure you." But what about when the software is not so good. How can you assure quality when there is none? What do you say then?
"This software stinks"?
"This software is buggier than a stagnant pond in July"?
True as these statements may be, it is not assuring anyone that it is quality software, which is what the term Quality Assurance is all about. That is why I don't like the term. To steal from the overused truism of why you should not assume, you should not assure, because it makes an ass out of you and R.E.
Quality Engineering on the other had seems to be more about a process for improving quality. Engineering is about designing and building things, so Quality Engineering should be about designing and building quality. The input could be of any level of quality, but after it goes through the QE process it should be of higher quality than when it came in. That means that in QE you not only find bugs in the product, but also in the process.
That all sounds great, especially if you are giving an talk at a software testing conference, but what does it really mean to people in the trenches. Whatever QE process your company uses, it could probably be improved. Unfortunately, we're usually too busy finding bugs, writing defect reports, and verifying fixes to have much time left for improving the process. You could move the world if you had a place to stand, right? But, how do you get to that place? My answer to that question is the same as my answer to "How do you eat an elephant?" One bite at a time. Every thing you can do to improve the process, no matter how small, should mean less bugs finding their way into test, which means more time for the Quality Engineer to make more improvements to the process.
In the end, I don't think it really pays to get all worked up about if you are doing QA, QE, or Testing. These are all just words. What matters is that you know what you are trying to accomplish and how to get there. Let people who spend more time with PowerPoint slides than bugs worry about the terminology.
Wednesday, July 11, 2007
Blue Screen Joy is On
The first step to making something better is to point out what is wrong with it.
With that in mind, I welcome you to Blue Screen Joy - a blog about software testing, quality engineering, the pursuit of bugs, and the ongoing quest for the holy grail - the Blue Screen of Death.
In this blog I will be sharing what I've learned about software testing during the last 10+ years as a professional software quality engineer. During that time I've done all kinds of testing in companies ranging from 100,000+ employees, to start-ups that were at times as small as 2 people. Speaking of people, I've also been known to test the patience of some folks along the way, especially developers. It is all for the cause, though.
I'll also be offering reviews and rants about software and tech products for which I am just a user. Nothing drives me as crazy as when I pay money for something and it does not work well - especially when the flaws could have and should have easily been discovered during test.
So, I hope you enjoy the blog and find it useful. Now, lets go kick the tires and light the fires!
Subscribe to:
Posts (Atom)
