Posts Tagged ‘Software Testing’

Test Escape Reports

We just had a big launch at work, which of course leaves me thinking about test escape reports.

I have mixed feelings on test escape reports, and for a long time I didn’t want to use them here. My reason being that if you have a small test team, and you know everyone is really dedicated and working hard, then the occasional test escape is to be excepted and we shouldn’t harp on them.

However, I think over time I’ve seen the other side of this issue. Test Escape reports ensure stakeholders that you are on top of any issues both before launch and after. They hold the team accountable, not just for the issue but also for the resolution and prevention of it in the future.

 

The format we use is simple:

Issue:

Severity:

Reason missed:

Steps taken to cover functionality:

Test cases numbers for verification:

 

The point being not to look backward, or to blame someone for missing a bug- but to look forward into how we can test that in the future and how we can avoid over looking similar areas on other projects. The goal is to improve visibility- we admit our mistakes, take steps to correct them and let everyone take comfort in/participate in the improved coverage.

How do you handle missed bugs? Are test escapes’ a whole team problem or an individual tester’s problem? Is there a formal process surrounding them?

Sauce Labs

We got set up with Sauce Labs last week and are starting to play around with it. It seems like it will be a great addition to our automated testing. We are in the process of redefining how we use Selenium here, so that it is used more effectively and strategically.

Really, our greatest challenge has been getting everything smoothly integrated through Hudson, which I think will make a huge difference. The whole team worked through it together and we learned a ton.

At Agile 2010 I heard Jason Huggins speak about using selenium to make demos of functionality for shareholders. I loved this idea. What a great plan  to give people an easy way to keep track of a project’s current status and to nail down the functionality for some high level tests. We are trying a similar idea now for a widely used site to help users get a good handle on the features. We are hoping it is our first finalized example of the new automation process with sauce labs integrated. I am really excited!

Become a Better QA

What do you do to become a better QA?

It is easy to get distracted by the rush of every day work as a tester. Over time, you get practice and your skills develop and you progress. But it seems such a passive way to go about improving yourself and your career. There are numerous books and blogs and communities out there, not to mention whatever is available locally in your area. So how do you chose?

When I started at my current job, I made it an ongoing personal goal to learn as much as I could about testing and related technologies that will make my job easier, make my team stronger and help our testing efforts go further.

In the last year, I have met a lot of mini-goals along that path- becoming  more comfortable with shell scripting, starting to learn ruby, reading at least two testing books per quarter – but I still feel like I have a lot to do! It is both exciting and overwhelming to dive into open-ended self education.

I frequent blogs that put out fresh new ideas and share experiences, I follow groups online and attended an awesome conference. I’ve considered taking evening classes at a community college (I loved my evening ruby class last spring) and trying to get more involved with the local community. I think both of those may come into play next year. My focus for the next year is both on strengthening my agile testing skills as we have teams switching over to SCRUM, as well as to continue learning Ruby and of course, reading about new QA techniques and approaches.

But where do you turn to become a better tester? How do you decide what to work on and where to improve your skills?

Agile Conference 2010

I arrived in Orlando with my team last night. We are looking forward to a week at the Agile Conference 2010. Our company has been moving more and more into true agile all year, and we are learning as we go, so this conference will be a great chance for us to pick up new ideas.

Finding out a testers place on the team has been an ongoing and ever changing process. Which is why I was so excited when I learned that there is an amazing testing track with classes I can’t wait to take and speakers I am looking forward to hearing. I hope to learn a lot to bring back to the team at home and try out for ourselves!

Selenium IDE breaks firefox update 3.6.6

The new firefox update won’t start if the Selenium IDE is enabled.

System:
OSX 10.6 (snow leopard)
firefox 3.6.6
Selenium IDE

I updated my firefox version, then tried to start the browser. nothing happened. I opened firefox in safemode with all add ons disabled. It started fine. I added my add ons back one at a time until I added selenium IDE again, and it couldn’t start after that.

I also went into about:config and added a new Boolean  setting: extensions.checkCompatibility.3.6b (false) as recommended here. Nothing seems to be working so far.

Has any one else had this problem yet?

The “Mythical” QA schedule

I just finished reading The Mythical Man Month by Frederick Brooks. This classic was still amazingly relevant, even thirty some years later.

Some of the tactics he suggests have just been made obsolete by the advances in technology – however, his use of a surgical team as a model for software teams seemed surprisingly agile and modern (not that agile and modern are synonymous). What was most interesting to me was the importance he placed on software testing. In his planning for software schedules, he allots half the project time to testing and bug fixes.

What could you do with that kind of time? Could better fixes be implemented for bugs? Would more bugs be fixed pre-launch? I find that as development schedules get pushed out, it is testing that get shortened.

Squished between milestones that can’t move or running up against a launch date that has already been announced, testing is often a race against time. Sometimes – new projects are hard to estimate for. Sure, you know how long it will take you to run through your test cases, but what about the debugging time? That varies greatly depending on the bugs found. Having half the schedule devoted to testing (or even just a greater percentage) would certainly be a change.

Would such a schedule just stretch things out or get shortened itself? Would it hinder the creativity of the engineers by shortening their time? Or would it work to allow a team with greater importance and visibility placed on testing? A place where quality was not just a “QA” thing, but a collaborative time of testing and bug fixing that could bring the importance of quality at every step of the development process to the forefront.

Maker’s Schedule vs Manager’s Schedule

I just read Maker’s Schedule, Manager’s Schedule by Paul Graham. It got me thinking about how readily it applies to QA and to so many of the projects I have worked on- especially as I get more integrated with the development team. Here is an excerpt, but reading the whole article is well worth it.

There are two types of schedule, which I’ll call the manager’s schedule and the maker’s schedule. The manager’s schedule is for bosses. It’s embodied in the traditional appointment book, with each day cut into one hour intervals. You can block off several hours for a single task if you need to, but by default you change what you’re doing every hour…

But there’s another way of using time that’s common among people who make things, like programmers and writers. They generally prefer to use time in units of half a day at least. You can’t write or program well in units of an hour. That’s barely enough time to get started.” -Graham

As software testers, we are often makers in a manager’s world. But it is different for us than for engineers, because we need to fill both roles to some extent. We need to produce test cases, comb through documents and create creative and comprehensive test plans. But we also need to sit in on meetings, communicate with other departments, understand release schedules and be focusing not just on our days work- but on following up on a previous releases issues and planning for next releases test cycle. We have to balance needing a maker’s block of time to create and a manager’s hour long shifts to get to meetings and do planning.

What do you do, as testers, to mitigate the clash of schedules and are you happy with how it works in your company?