By Ian Landsman

There seems to be some misunderstanding about the beta process in the MicroISV community and even less understanding of how running an effective beta can be the foundation for a successful product launch. Hopefully I can clear up a few things and give a few tips to help you along the way.

Why do a beta?

Many ISV’s think the beta is just a box on the checksheet to release. This kind of thinking will get you in real trouble. The beta is a critical step in the testing of your product. A beta before a 1.0 is perhaps the most important beta you’ll ever run since, after hours of pouring over code, you may lose site of the likely trouble areas or parts of code that may require additional testing. At this point your code has never been run out in the wild, and ‘real-world’ usage is critical to ferreting out hidden problems. You think you know your code but you don’t, so don’t fool yourself into thinking that you understand all the potential trouble spots!

How long should a good beta last?

In my opinion, you can’t do a full beta in less than a month. Less than that and there’s simply not enough time for users to get comfortable with the program, for you to make adjustments, and for users to install subsequent beta builds. In fact, I’d suggest 2 months if you can afford the time. That will let you do about 4-5 beta releases, which should be enough to work out the major kinks. The HelpSpot beta ran 6 weeks and in that time was very successful in eliminating many huge bugs with the installer and other system components.

Finding great beta testers

Finding great beta testers really isn’t any different than finding customers. If you started your marketing early (via blogs, forums, etc) then you should already have a group of interested users. Trying to find beta testers the week before your test begins isn’t going to work. A great beta tester is someone who needs your product and is interested in its success. People who agree to do it but don’t really need your product are relatively worthless to your test. They have no stake in the outcome, they won’t have good ideas for improvements, and they’ll generally be a waste of your time. There’s no magic solution to this problem other than hard work, smart planning, and early contact with your potential customer base.

So you’ve made the early connections to your customers…what attrition rates should you expect?

Plan for at least half of your testers to simply fall away being either too busy to install it or having already found a solution to their issue. You need to plan for this. You should aim to have double the number of beta testers as you think you actually need. In my case, I had 170 companies lined up for beta testing with about 130 actually downloading and about 80 providing quality feedback.

The beta should be polished and ready for prime time

A beta product should have a complete user interface. If you’re still working on your UI then you’re not ready for a beta. Beta tests aren’t the place for testing out which color is best or changing graphics between each build. The focus should be on working out technical kinks, don’t make the beta testers job more difficult by changing how things look between each build. Remember, this is a dry-run for the real thing so your product should reflect such.

Getting feedback

This is probably the biggest problem people seem to have with running a beta. They think their not going to get feedback or haven’t gotten feedback in their past betas. The solution….. build the feedback form into every page of your application. Every single page should have a prominent ‘beta feedback’ box. Make the feedback process as obvious as possible. They’re also more likely to give feedback for smaller items, which would not have been reported if doing so was tedious. It’s a little more work for you, but well worth it.

In HelpSpot’s case the box had 3 links in it. One for providing feedback about bugs, one for feature ideas, and one link to the general discussion forum.

Discounts

After your beta testers have suffered through all your bugs, provided feedback, and validated fixes/improvements they deserve a reward. They deserve to be given a discount on your software, they do not deserve to receive it for free. If you followed my advice for finding beta testers then what you have are not just a group of testers, but your first customers. They would have purchased this wonderful product at full price had it been available so asking them to pay half price is more than fair. Remember they’re vested in the success of your product. Having spent time learning and maximizing your product for their needs, they don’t want your company to go out of business.From personal experience, this initial revenue is very good for your psyche. After you’ve spent 4 months finding your first customers (beta testers) do you really just want to give the software away and start all over looking for your first customers? No!

The HelpSpot beta testers continue to be some of my best customers and are still providing great feedback. I’ve never had one complaint about the fact that they had to pay for the software, because they use and love the software. And shouldn’t that be your goal, to create software people want to pay for. If you’re unable to get them to pay for your 1.0 product perhaps it’s not ready. Listen to your testers, find out where the product falls short and address those issues. You’ll be rewarded with a better product and loyal customers.

—

Ian Landsman is the founder of UserScape, creator of HelpSpot Help Desk Software a modern web based help desk application.

Popularity: 1% [?]