Crowd testing now a critical resource for apps developers

Just such, to my mind, is Testbirds. It’s a crowd testing model. Enthusiastic device owners sign up on the site (http://www.testbirds.com/) and may then, after an aptitude test and other screening, become one of a crowd of app and website testers.

According to Testbirds’ Managing Director, Philipp Benkler, tresting apps is done on three levels. First there is internal testing where the software is put through its paces by a group of people within the developer. Then automated testing can soak test the application to make sure it doesn’t break. But the third level is in many ways the most critical and the most demanding. This is the human testing to make sure it actually works for the people who use it without absurdities or creeping in. This is what crowd testing is good for.

The Testbirds site has now signed up over 20,000 testers who between them own 30,000 devices. This is the sort of scale you need to cover an increasingly diverse and fractured field. Think about it. There are dozens of major devices (scores more of obscure ones) and these may be running different versions of their operating system. Then start doing the exponential sums to determine how many testers you’d need to really test that an app works on a particular phone or tablet - you get the picture.

Testing has therefore become a fraught activity for apps developers. It’s no longer OK to put an app in an app store and use the first 100 downloaders as the bug-finders. Once an app has got its awful scores, that’s usually an end to it. Nobody buys or downloads.

The answer is, of course, to do some solid testing to make sure the thing works before it’s put up for sale, but as we’ve outlined above that can be a huge logistical headache if you attempt to do it right. So apps developers are happy to outsource the crowd-testing to a rigorous process.

Philipp reports big pick-up on Testbirds services from developers (about 100 per cent growth per year) and heightened interest too from developers of apps for the big verticals where rigour is crucial, including media companies and banks where clearly product has to work perfectly before it can be released into the community.