Filing good bug reports

(Comments)

We all use software and systems we didn't create, and we all find ourselves in the position of having to report bugs and seek support. Surprisingly, support staff and software engineers are human beings, too, and human beings have moods and feelings just like everybody else. They probably spend a good portion of their work day dealing with such things, and they can easily have any number of negative reactions: frustration, impatience, or condescension. This is rude, clearly, but we still need their help to get back to what we were doing. Filing good bug reports and doing your part as a good user can help ensure that you get the most helpful response. Here's how.

1. Read the error messages

Some software or usability engineer somewhere actually took the time to craft an error message. Surprisingly often, users just don't read those messages, even if they can help the user solve their own problem without needing to ask for help. All these users know is that they got an error they didn't expect. Before you file a bug report or contact technical support, you ought to take a moment to actually read the error output. Not only will providing that error output help the support engineer diagnose the problem you're having, but it shows that you yourself are trying to be helpful and self-reliant. Support staff truly does appreciate that.

One time, a developer working with a Django project that I was supporting, they reported that their development environment just stopped working. He pasted the output from the command he'd run, and after a long series of log messages at the very bottom, it read:

django.db.utils.OperationalError: could not connect to server: Connection refused
Is the server running on host "localhost" (127.0.0.1) and accepting TCP/IP connections on port 5432?

This is a great error message. It tells the user what manner of problem occurred (django.db.utils.OperationalError), and the user could surmise from that module name that this was in the database subsystem. It describes the low-level error that the system reported ("could not connect to server: Connection refused"). And it even suggests a first step toward remedying the problem by checking to make sure the database is running and listening to where Django is looking to find it.

The first question I'm going to ask my developer is whether they tried this first step and what they found out. If they haven't and haven't even read this message, any support engineer would feel a little frustrated. So read the error output. It's there to help you help yourself.

2. Google

If you're using a piece of commercial or open-source software, paste the error message into Google along with the name of the product reporting the error. Sites like Stack Overflow, Github, and Quora have a long spanning history of people reporting errors and bugs - it's entirely possible somebody had the exact same issue that you had. Also, I can pretty much guarantee that the support engineer is going to do this if he/she doesn't immediately know what the problem is. Save them and yourself some time by doing the preliminary searches to see if there's an easy and quick fix.

3. Try again

Lots of errors are simply intermittent. They could occur from a random confluence of unrelated events, a temporary outage, or some other fluke. Many times, you can get past the blockage by simply trying your operation again.

I host a number of web applications (including this blog) on Heroku. Heroku's deploy process is notoriously unreliable, in terms of how long it takes to do a deploy, whether or not a deploy will time out, or any number of mysterious failures inside the black box of a PaaS provider. When I get such a failure, my first reaction is to try the deploy again. Nine times out of ten, it works the second time.

This advice is safe to skip for transactions that deal with very sensitive or volatile data, like financial transactions. But for everything else, there's probably no harm in trying again.

4. Help the support engineer reproduce the problem

If support staff doesn't know what's wrong, they will have a much harder time fixing it. The best and more comprehensive way to help a support engineer help you solve your issue is for him/her to be able to reproduce the problem. To do that, you'll need to provide three critical pieces of information:

  1. What steps did you take that resulted in the error or bug?
  2. What did you expect to happen when you took those steps that didn't happen?
  3. What actually happened instead?

For the first one, be specific and give details. What URL did you go to? What commands did you execute? What button did you click on? What form values did you provide? Were you logged in or not? Any and all steps that the support engineer would need to take in order to reproduce the failure you observed will be helpful to enumerate.

For the second one, what the support engineer will be looking for is to understand your intent. Maybe the steps you took aren't the ones that are necessary to achieve your outcome. Maybe there's a workaround to do what you're trying to do. And if there is a bug that needs to be fixed, knowing what you expected to happen helps the support engineer identify that the problem is fixed.

And for the third one, this is where the support engineer will start looking for the source of the failure, so providing him/her with all log output, error messages, screenshots, etc. will be crucial to making sure that the engineer can confirm that they've reproduced the issue and can identify where in the process the breakdown began.

Providing all of the information about your actions, your mindset, and your desired results will help the support engineer get on the same page as you toward fixing it.

5. Be kind

We're all human beings, and there's never any reason not to be kind to one another. You're talking to a support engineer because you need help, and the support engineer is on your side. Remember that the support engineer is a human being with feelings like you, and you'll find almost without fail that they'll be kind and helpful in return.

Do you have other suggestions for writing helpful bug reports? Feel free to leave your stories or ideas in the comments.

And related, check out my Celerity colleague Janet Leon's series on the true cost of bugs, an outstanding argument for why QA needs to be a process, not just a person.

(Photo credit)

Currently unrated

Comments