Sauce Labs

We got set up with Sauce Labs last week and are starting to play around with it. It seems like it will be a great addition to our automated testing. We are in the process of redefining how we use Selenium here, so that it is used more effectively and strategically.

Really, our greatest challenge has been getting everything smoothly integrated through Hudson, which I think will make a huge difference. The whole team worked through it together and we learned a ton.

At Agile 2010 I heard Jason Huggins speak about using selenium to make demos of functionality for shareholders. I loved this idea. What a great plan  to give people an easy way to keep track of a project’s current status and to nail down the functionality for some high level tests. We are trying a similar idea now for a widely used site to help users get a good handle on the features. We are hoping it is our first finalized example of the new automation process with sauce labs integrated. I am really excited!

Advertisements

Become a Better QA

What do you do to become a better QA?

It is easy to get distracted by the rush of every day work as a tester. Over time, you get practice and your skills develop and you progress. But it seems such a passive way to go about improving yourself and your career. There are numerous books and blogs and communities out there, not to mention whatever is available locally in your area. So how do you chose?

When I started at my current job, I made it an ongoing personal goal to learn as much as I could about testing and related technologies that will make my job easier, make my team stronger and help our testing efforts go further.

In the last year, I have met a lot of mini-goals along that path- becoming  more comfortable with shell scripting, starting to learn ruby, reading at least two testing books per quarter – but I still feel like I have a lot to do! It is both exciting and overwhelming to dive into open-ended self education.

I frequent blogs that put out fresh new ideas and share experiences, I follow groups online and attended an awesome conference. I’ve considered taking evening classes at a community college (I loved my evening ruby class last spring) and trying to get more involved with the local community. I think both of those may come into play next year. My focus for the next year is both on strengthening my agile testing skills as we have teams switching over to SCRUM, as well as to continue learning Ruby and of course, reading about new QA techniques and approaches.

But where do you turn to become a better tester? How do you decide what to work on and where to improve your skills?

Time for QA to fully embrace Agile

My favorite idea from Agile 2010 was that “QA should be the headlights for a project” A quality guide, not a quality police.

We can sit at the end of the schedule, run tests and file bugs. Or, we can jump out in front, help define requirements, learn from the programmers as they work on the project, offer advice and keep the user experience and project quality on everyone’s mind from beginning to end. Which helps the project more? Actually, which is more fun? Because personally I’d much rather get involved and get people excited about turning out a good product in the end, than just force quality on them right before release.

So how do you change your company culture to embrace that? I am presenting at a lunch and learn next week, with the hopes of kicking off this next phase of the QA process. I am so inspired and excited for the change after a week of being surrounded with agile talks… but how will the company take it? I think I have gotten my department on board, but the hard part will be the other departments who have to put up with us a lot more now. No one likes change, and organizations are harder to change than people.

So how do you get everyone excited about it? How do you show everyone how important this will be? I haven’t finished my presentation yet, but I’m trying to think back to those really inspiring sessions at the Agile Conference and give it a go. Heres hoping!

Agile Conference 2010

I arrived in Orlando with my team last night. We are looking forward to a week at the Agile Conference 2010. Our company has been moving more and more into true agile all year, and we are learning as we go, so this conference will be a great chance for us to pick up new ideas.

Finding out a testers place on the team has been an ongoing and ever changing process. Which is why I was so excited when I learned that there is an amazing testing track with classes I can’t wait to take and speakers I am looking forward to hearing. I hope to learn a lot to bring back to the team at home and try out for ourselves!

Selenium IDE breaks firefox update 3.6.6

The new firefox update won’t start if the Selenium IDE is enabled.

System:
OSX 10.6 (snow leopard)
firefox 3.6.6
Selenium IDE

I updated my firefox version, then tried to start the browser. nothing happened. I opened firefox in safemode with all add ons disabled. It started fine. I added my add ons back one at a time until I added selenium IDE again, and it couldn’t start after that.

I also went into about:config and added a new Boolean  setting: extensions.checkCompatibility.3.6b (false) as recommended here. Nothing seems to be working so far.

Has any one else had this problem yet?

The Tester’s Scientific Method

We all remember the scientific method from jr. high science labs.  But as a tester, we still use it every day. The steps may entail different things, but the underlying philosophy remains the same. Testing is an interesting blend between scientific and creative, between destructing and constructing. It requires a flexible yet ordered mind and, as with any scientific approach, it requires curiosity.

1. Ask a Question: The most valuable thing a QA can do is to ask the right questions. Is this feature supposed to work this way? Could this functionality be broken if I do this?  How will this affect existing functionality? What does this mean for the user experience?

2. Do research: Talk to the engineers, look at the documentation, discuss with your fellow testers. Stand up meetings are a great time to hear what developers are up to and to ask any questions you have run into on those features.

3. Construct Hypothesis: Determine how the feature should work and all the ways you need to verify that it passes when it is supposed to pass and fails when it is supposed to fail. Write your automated and manual test cases

4. Test: Run test cases

5. Analyze Results: Evaluate what is failing and why. It is not enough to know that a feature has an issue, you need to know what exactly reproduces it? If you spend a moment looking at it, perhaps you will find related issues or other ways it can be reproduced.

6. Report results:  Carefully fill out your bug report. Stick to a bug filing standard. Spell check. Make sure that anyone could understand your bug and the information you have gathered from it.

7. Be prepared to test again. Bug fixes always bring with them the risk of inserting more bugs. Don’t just verify a bug fix, test the bug fix. It should get the same attention that the original feature did. Be aware that you may have uncovered a cluster of bugs or a series of new bugs that have appeared since the fix, so look at it with fresh eyes.

Extending Lighthouse bug tracking

We use lighthouse for bug tracking at my company. It is simple, flexible and has a great api that allows us to hook it into testlink to track bugs from our test management system. The search functionality, however, seems limited. I have often wished there were a dashboard feature that allowed for more of an overview. It might be different if I worked on only one project, but as a QA lead,  I care about all the projects all the time. So when my company decided to do a hackathon day, I wanted to find a way to extend lighthouse into a dashboard.

We had 8 hours and 4 people to complete our hackathon task. Since a number of us had just completed a basic intro to Rails class, we decided to make a separate page that pulls data from Lighthouse. We were able to get the info out of lighthouse pretty easily with the API and Rails (with web themes) let us get a basic page up quickly. With so much data coming in, we had to figure out how to paginate the results so that the system didn’t run into any lag while trying to display the results. And the profile creation and filtering process took us a while.

By the end of the day, we had a working profile up with dynamic info coming in from our Lighthouse projects. A user can log in and see a list of their current tickets for all projects, with priority and status displayed and link those back to lighthouse to update them, or they can go to a more project specific view and see tasks assigned to them – with details for each project.

We have more work to do- we want to work on the sorting so that tickets appear in different bins by status, by priority, date updated etc. We also want to make a “team view” so that leads can log in and see the release status for all the members of their team. Eventually, we hope to do similar integration with testlink to display test status, test cases assigned and test cases created by you. We also want to integrate with Pivotal Tracker, which is also great from the project perspective, but needs a cross- project dashboard for those of us who are always working cross- team.