Archive for the ‘Documentation’ Category

When Low Priority Bugs Should be Fixed

So you file a UI issue. No functionality is broken, the site looks ok. The deadline is close, and it makes sense to launch it. No one wants to delay a deploy because of a few pixels. When do you put your foot down? Maybe it isn’t worth fixing the color on a banner that only shows up for 5% of people using the application, or optimizing for an outdated browser that few people use. But not all issues that are marked as low priority should be ignored.

Low priority issues should be fixed when they make you look sloppy. Sure, everything works, but if your alignment is off, your headlines are the wrong font size, or spelling / grammar could be improved – the overall look of the application is sloppy. People will not remember that the log in process worked just fine, they will remember that your news story hung halfway off the page, that they had to scroll down the page to get to information or that your content blocks were not lined up.

low priority defect on Facebook

Low priority issues should also be fixed if they they add frustration (now or in the future). If your page can only be viewed well in one or two browsers, you may lose users who don’t want to download and use an unfamiliar browser.  Anything that can cause frustration to your users or waste the customer service department’s time helping people find work around is better off fixed.

Your site should be polished- functionality, and look and feel. If adjusting a low priority defect gives a more professional overall appearance, or saves you from having to have a detailed troubleshooting page- they should be fixed before launch. And it is worth the QA departments’ time to make a stand for them.


What I’ve learned about Agile Testing

I started on an agile team last August. In the past few months, I’ve be exposed to a wide range of agile-ness as we have learned to adapt it for different projects and struggled to make it work for us. I have found great worth in some of its practices, and unfinished mess in others. As a team, the company has had to switch practices and courses a few times. The cool thing is that we are able to. No one is so stuck on procedure as to not admit that it could use tweaking. And where procedure was too lax, people have asked for more.

Here is what I’ve learned as the QA department has grown up and around the development teams:

– Agile doesn’t mean “No Processes” it just means flexible processes

Documentation is more a struggle. Shortcuts made in the name of agileness and flexiblity have been huge road blocks in my testing effort. We are still sorting out the right place for them. Something that allows for creativity and adjustments, but still leaves us with a solid document to test against.

– Trying to keep meetings short by cutting out discussion is pointless and unhelpful. Structuring meetings to allow for quick agendas and then discussion time is priceless.

Transparency between departments and code reviews are QA’s best friend, even if it takes more time at first.

– Short iterations mean quick test cycles. This means you need to have as much ready before had as possible – test plans, test suites – both automated an manual, report templates etc can all help keep you on track.

– Managing an agile team means knowing when to step back. The team is responsible to each other, not to you.

– Be flexible. This should be obvious, and maybe it is to some. But in my experience, learning agile meant learning to contort your testing efforts to the time and space available. That doesnt mean you should let your team get pushed around, but it means that you have to be willing to do the agile dance along with everyone else.

– QA needs to make its needs known. Don’t let yourself get deadlines pushed back and back until you are the one left struggling to pick up all the slack. Just because deadlines can change mid cycle now, doesn’t mean that your team has to sacrifice thoroughness in the name of changing requirements and speed.

– Communication is even more important – with things changing and people having so much freedom with their work, if you have a bad communicator, your whole project is at risk.

I am sure this list will grow in the coming months. For those of you who are new to Agile testing- what have you learned? Was it what you expected? Have there been any roadblocks on your project?

Write bugs that get fixed

It seems simple enough- find an issue, file a bug, wait for it to be fixed. The tough part should be finding the bug in the first place, but sometimes it isn’t.

1- Time constraints. When deadlines loom, the team may be anxious to get the project out the door and ignore “minor” issues that can be fixed post launch.

2-Complexity. Even simple sounding bugs have have complex integration with other functionality or require a lot of rework – and no one likes rework.

3- Dismissal. No one likes to hear this one, but it happens. Sometimes bugs get dismissed because a) the developer doesn’t trust the QA or hasn’t seen the issue b) it feels like a rare corner case c) it is hard to reproduce d) there are a lot of other demands on the team at the time and your bug simply falls off the radar.

How can we avoid these issues?

We can write clear bugs. Reproduction steps are one of the strongest tools we have. One of my favorite QA interview questions involves getting candidates to write reproduction steps for an unexpected action. It is a really important skill for QAs. Project managers and engineers need to really understand how we got to the issue we did and be able to see it themselves- not to mention how much it helps us if we have to explain our bugs to someone.

We can follow up on issues. How many times have you written a bug just to let it slip by unnoticed while you found more issues? Checking on the status of issues isn’t as exciting as finding new ones, but it is important if you are trying to get something fixed. If a bug gets put in deferred, make sure you know why. If it is fixed- does the fix match the documents?

We can be knowledgeable. Sure, we aren’t developers- but it helps if we know about about the software they are writing. If there are lots of false alarm bugs or low priority issues labeled as high priority, the team will start to trust us a little less. Knowing about the documents, the software and the issue make you a much more creditable source and seems to help your bugs get fixed.

We can “file” some bugs off the record. I’m not really promoting this- especially if your company is big on formal policies. But, if it makes sense for your team, letting developers know about smaller issues first, offline, not only gives them a head start, it puts us all on the same team. We aren’t filing bugs against developers, we are working with them to make sure everything runs right.

Does anyone else have tips for making sure bugs get addressed?

Documentation on Agile Developement Teams

Agile Teams are all about flexibility, right? Rules evolve with the project, people’s suggestions can taken on, deadlines can be adjusted internally – and within that one week sprint, the team is pretty much free to set their own pace and schedule. Its great. It allows for creativity, changes in plans and new ideas. But sometimes, it seems like documents lose some importance.

I know, I know. People get tired of hearing QA whine about documentation. But there is a reason.  In a world where you are free to find your own way, it is good to have some idea where you are finding your way too – so that you- and your teammates – all end up at the same destination. Documents provide that overview, that roadmap. They also help others to follow you- either when making revisions in the future, or when testing the final output. No matter which development style your team follows, the importance of good documentation should not be lost.

In waterfall development, documents are turn by turn directions -carefully planned and harder to change. Agile developement needs the same directions, but the exact path can be filled in as you go. By the end of the project (by the time QA starts testing), Agile teams should sh0uld have produced the same kind of documentation that waterfall teams have.

But what about QA teams? How do you go about designing your test plan when you don’t know exactly how things are going to work yet? The documents won’t be ready in advance, which means you have to keep your plan flexible… agile. You should have a good idea of the final product, so you can plan your approach, your general tests, your schedule etc very early on. As documents are updated, update the test plan. This was a big adjustment for me, as I was used to knowing in advance what all will be done. But really, the philosophy is the same- you just work in phases right behind the developers. They finish a piece of functionality, you finish your test plan and test that piece while they start filling in the documents on the next piece. QA defiantly has an important place here – and keeping an eye on the requirements, performing ambiguity review at each stage and applying those to an agile test plan are all part of it.

Because with out requirements, development really is like that old elephant story. (An awesome version of which is reposted below from this website)

It was six men of Indostan
To learning much inclined,
Who went to see the Elephant
(Though all of them were blind),
That each by observation
Might satisfy his mind.

The First approach’d the Elephant,
And happening to fall
Against his broad and sturdy side,
At once began to bawl:
“God bless me! but the Elephant
Is very like a wall!”

The Second, feeling of the tusk,
Cried, -“Ho! what have we here
So very round and smooth and sharp?
To me ’tis mighty clear
This wonder of an Elephant
Is very like a spear!”

The Third approached the animal,
And happening to take
The squirming trunk within his hands,
Thus boldly up and spake:
“I see,” quoth he, “the Elephant
Is very like a snake!”

The Fourth reached out his eager hand,
And felt about the knee.
“What most this wondrous beast is like
Is mighty plain,” quoth he,
“‘Tis clear enough the Elephant
Is very like a tree!”

The Fifth, who chanced to touch the ear,
Said: “E’en the blindest man
Can tell what this resembles most;
Deny the fact who can,
This marvel of an Elephant
Is very like a fan!”

The Sixth no sooner had begun
About the beast to grope,
Then, seizing on the swinging tail
That fell within his scope,
“I see,” quoth he, “the Elephant
Is very like a rope!”

And so these men of Indostan
Disputed loud and long,
Each in his own opinion
Exceeding stiff and strong,
Though each was partly in the right,
And all were in the wrong!


So oft in theologic wars,
The disputants, I ween,
Rail on in utter ignorance
Of what each other mean,
And prate about an Elephant
Not one of them has seen!