I would love to hand over an application to the client that has no bugs in it.  In reality, every time our company completes a project, we’re confident that in fact there are no bugs… sort of.

We test and we fix bugs, but we also realize testing is not about perfection.  It’s about getting a return on your investment.

When we think an app is ready to deploy and we hand it off to you, we’ve tested it to a level that makes sense for the particular project.

This is determined by:

1) Who is the audience?

2)  What exactly are we building?

3)  What are the consequences of a bug?

For example on the first point, if we’re in a controlled environment where we know everyone’s PC will be configured the same, we can safely test for only the one browser that is everyone is using (and if it’s IE6 hopefully we can convince them to upgrade everyone’s browser before we begin).  Conversely if we’re releasing an app where the majority of users will be from large corporations, government agencies, and universities, we’ll probably have to spend a lot of time testing on older browsers (yuck).

Either way, what we’re testing for is communicated to the client so they know if bugs are found from something we didn’t agree to test against, it’s not a big deal.

On point 2 and 3, these kind of go together.  If we’re building an app that ships millions of dollars out of a warehouse, or purchases and sells hundreds of thousands of dollars worth of stock on a daily basis, you can bet the testing is ramped up considerably.  And you can also bet the client is willing to pay for it (and if not you should probably run away from the project as fast as you can).    Again, we go over the test strategy with the client and we’re confident that if we perform all the agreed upon tests there won’t be any bugs for the tested scenarios.    We’re not saying the app will be bug free, but we are saying that any bugs which could cost the client loads of cash are going to be caught and fixed before we ever go live.

On the flip side of things, we have to think of things that we should purposely not test.

Let’s say you have a web app that has a three step wizard process.  First, there are tons of variables to consider before the user even loads the app.  What operating system are they on, what browser are they using, do they have javascript enabled, are they accessing the site through a mobile phone?

Next, while the process is linear, they may complete step one, hit the back button, submit step 1 again, go all the way to step 3, go back to step 1, leave the computer for 30 minutes, then complete the process.    This craziness could produce a bug.  Who cares. Chances are the majority of the time the user is not going to do this.  And if they do, and it happens once, we still don’t care because the chances of that same user doing the same exact thing a second time are pretty slim.

What I’m getting at is for us to test that scenario there are probably also several other thousand crazy scenarios we’d have to test before we even got to that.  We could do that but at that point we’ve probably bankrupted the client.

Now as a programmer, you might be thinking to yourself “I hate to test so you don’t have to worry about me testing for that scenario!”

That’s probably true but the point I’m making is to help you deal with clients.   After you release your software, chances are that if you have enough users someone will do something crazy that will cause a bug and the client is going to have an angry user on their hands.  You can try to tell them that testing for every single possible  scenario would not have been cost effective but now you’re just the defensive consultant who won’t own up to a problem.

A better way is to communicate with your client about testing up font.   Tell them about the risks and consequences of bugs and know what is most critical and what is not.  Tell them you can test for everything but more testing = more $$ and that when you release software chances are good there’s probably going to be some bugs.    Decide together what level of testing is appropriate for the project and in the end your software will be bug free… sort of.