The good way to file a bad bug

Great bug reports are the the oxygen of quality control. Take them away and everything turns sour.

As a Customer Support Lead, I spend a lot of time in technical discussions with our customers, ranging from API debates to complex SDK edge cases. Our end goal is a happy customer, but the approach will vary from conversation to conversation.

Sometimes I just need to explain something, sometimes I need to write some sample code or debug theirs, sometimes we need to update or clarify our own documentation, and occasionally I need to file a bug to get something fixed quickly.

Good bug reporting streamlines an investigation for whichever product team owns it and lets them triage quickly. A repository of well documented bug reports gives valuable context if a bug re-occurs, or a similar incident happens.

The features of a good bug report

We use Github Issues to log our bugs, but the advice remains the same regardless of what service you use.

A great title

A great title can be scanned by your colleagues to immediately get a sense of what’s happening, where, when, and how. Remember, this is going to have to be picked out from a list of other issues during triage. A descriptive title will also ensure that duplicate reports of the same bug will be quickly spotted.

A bad example: User list is bust for this user
Why? How could anyone know what this is without having to click through and read more?

A good example: User list not scrollable in Linux on OSX FF34
Why? This has lots of detail including the operating system and browser being used.

A concise, precise description

Be as detailed as possible as to what is actually happening, and how the user experiences it. Be careful not to take the user’s description of events as gospel. Often they’ll only talk about what they experience, not what is actually happening. For example, a customer might say, “The message stopped sending”. Did it actually stop sending, or did the UI tell them it stopped sending?

As we use Intercom to provide support for Intercom we always link to the relevant conversation where the customer highlighted the issue.

How to reproduce

To reproduce a bug you need to document the relevant conditions it occurs in and the steps necessary to trigger it.

Conditions are everything from Browser/OS/Mail Client through to customer state (e.g. paying, new, or old). Make sure to include clear steps to reproduce i.e any actions taken on screen to cause a broken state.

Label it

Labelling is also important so that issues can be prioritised. We tag bugs with the following labels:

Severity: We have labels for problems (p-levels) ranging from “Drop everything and fix this” through to “Tidy this up when we’re next changing that flow”.

Team: Which product team is impacted. If in doubt over which team should be notified we add tags for both. The key here is to ensure the relevant product manager and engineering lead is alerted straight away (we send GitHub notifications to the relevant team’s Slack channel).

Customer frequency: Our team tag frequent issues if they occur daily or weekly. These problems may not be critical to the product (in terms of p-level) but are so frequent that it’s in everyone’s interests to fix it quickly.

Cause: We want to determine why this bug got in the wild and depending on the cause what we can do to make sure it doesn’t happen again. Some bugs you can’t prevent but some are preventable with proper testing or QA. Example tags here includes acceptable, missing automated tests, wrong design/dev decision or missing human QA.

Console investigations

Our customer support team has many engineers so naturally we investigate ourselves before escalating. Tools like FullStory’s Dev Tools lets us see what the console logs looked like for real visitors on our site at the time of their issue. Including console output from our investigations is really helpful for transferring context. The receiving engineer can then either build on that logic or decide its wrong and close it – either way, time has been saved.

Gifs and screenshots

Screenshots are the currency of UI bugs and gifs are the equivalent for interaction bugs. If something is janky, such as a rough animation or broken transition, a Gif is the easiest way to communicate it. Show, don’t tell.

Check the Javascript console

If there’s a problem in the UI it’s always good to check the Javascript console. We have a saved reply for this, as it’s such a valuable piece of information from our users.  For anyone non-technical it means hitting <Alt>-<cmd>-<J> – anything red is bad. Again, a screenshot of the offending text is valuable.

Other considerations

• Does this bug happen every time? Or is it sporadic. This will influence the severity of the bug, and the complexity of the investigation.
• What were you doing right before this occurred? Often times there’s an offending prior state that goes unnoticed (e.g. I arrived here by clicking this link in an email)
• If the bug causes a crash, does the crash log offer any clues as to what might be the underlying cause?
• What plugins are in use? There are many, many, popular but badly maintained plugins that cause browsers trouble.

The eternal rules of bug reporting

In the QA mindset, Rands outlines the golden rules of bug reporting. Put concisely:

• If you see something wrong in the product, however big or small, report it as best you can.
• It is good practice to take the time to report the issue as descriptively and thoroughly as possible, but it is more important to report the issue.
• It is also good practice to check if the issue has been reported by someone else, but it is more important to report the issue.

No matter how rudimentary it is, a bad report is better than no report. Everyone in your company, from the CEO to the summer intern, should know how and who to alert if they spot a problem in your product.