As part of the rush to release software earlier, our attitude towards testing and consumer feedback has evolved. Traditional QA departments are being scaled back in favor of beta testing. There are good online alternatives to expensive focus testing. Closed beta tests start earlier in development and last longer than they used to. Open beta testing follows and it’s not uncommon for beta to be the longest period in the development cycle.
Online tools and services make it easier than ever before to conduct your own beta tests, but there are some important things to consider when you are drawing up a plan.
Preparing for beta success
To begin with you need to detail your objectives and consider your metrics for measuring success.
How many people do you have interested in helping out on a beta? Can you rope in friends and family? Do you have details of potentially interested parties who have used previous software releases from your company? Do you have some advertising budget to promote your beta and get the level of sign up you need?
Do you need a broad participation or are you going for depth over breadth? Is your software fully accessible from day one or is it designed to accumulate data and expose more advanced features over time? In our case it was the latter and so a six month closed beta period with a small group of testers was appropriate.
Screen your testers and engage them
Once you have a prospective list of beta testers you need to run through the screening process and select candidates you believe will add value to the process. You need testers who are fully engaged and prepared to give you some time and share their honest feedback. A small group of committed testers is far more valuable than a large group of testers who give minimal feedback and don’t continue to use the software over time.
You also need to make the testing process as easy as possible for them. It should not be a hassle to report bugs or make suggestions. It is easier to set up access to a bug tracking database or a forum to allow them to report issues directly. Then you can import the issues you want to work on into your existing bug database. Don’t give them access to your internal bug database because it will be full of sensitive information about the development. You should also be prepared to accept bugs by email and just take the time to rewrite and enter any pertinent reports into the system yourself.
Remember that beta testers will expect a fast turnaround on the bugs they report. They are giving you some of their time and the benefit of their opinion so if they enter a bug and it appears to be ignored they may become jaded about the process. The last thing you want is a bitter beta tester. When an issue comes in let them know that you value that input and, if you are able to, give them a time estimate for a fix.
In the early closed beta period there may be whole areas of the software that are not yet functional. Make sure your testers know this so that you can manage their expectations. Good communication will also yield more useful information and you may find it worthwhile to set up one-on-one sessions with your most active beta testers in order to uncover their overall opinions about how to improve the software.
Path to a polished product
The value of collecting the opinions of your target consumers relatively early in any software development should not be underestimated. It’s not just about finding a cost-effective solution. You can tailor the software and channel your resources into creating a polished and highly usable end product that does exactly what the consumer wants it to.
If you haven’t yet jumped onboard with the beta testing trend then it’s about time you did.