Software commando usability test for $30 in Cliff Bars

Usability testing, required for rapid iteration, can be expensive, requires advanced planing and diverts programmers’ valuable time.  A better solution is to hire a commando usability tester like me who can work directly with your programmers to determine what features need to be tested and analyze user feedback.  A proficient tester in a public venue can generate quality feedback quickly for very low cost at almost any time of day.  Short surveys taken after the usability test  provide critical demographic information. Get your first usability test free by contacting me.

Last Sunday I ran a usability commando test for an iPhone running application.  In exchange for $30 worth of cliff bars, twenty people tried the application over two hours.  Commando testing proved to be highly effective, very affordable and a nearly instantaneous way to get feedback.

How to Run a Commando Test

1)      Write down your goals

2)      Choose a good location and move if necessary

2)      Create signs on the fly to test what gets people over

3)      Engage people with simple and relevant questions to get their attention

4)      Start the hands-on test as soon as possible

5)      Get email address with an offer after the test for follow up and marketing

6)      Have a short survey for more substantive feedback

Mike is the creator of RoadBud, a new iPhone application that helps runners track their workouts, listen to music, and keep safe while on the road.  Mike and I setup a table at a 5k race and had runners put their hands on the application and perform specific tasks.  Mike got a lot of really useful feedback that will inform his next development iteration.

Presentation

The setup was very home made.  We used a small square table to put out an email signup list, two iPhones, a survey and some promotional materials.  His daughter drew a sign on a 3 x 4 foot piece of paper.  We stood by the sides of the table talking with people as they walked by.  The extremely budget nature of the product was not detrimental.  We were overwhelmed by people wanting to play with the application.

Location is critical to a successful commando test.  You need to have enough traffic walking by your location.  Move the table around if necessary to get the right traffic.  And scoping out the location before the test begins is a smart move that we didn’t do.

Signage

By writing the advertising sign onsite, we were able to test out different displays to see what was more effective.  We first started with a sign that said “Test a new iPhone running app”.  We specifically did not offer any reward.  As people walked by we would ask them if they ever ran with music.  We found that by not offering a reward for their participation that people felt we were selling them something and were not inclined to participate.

So we drew up a new sign and said “Test a new Iphone running app, get a FREE cliff bar”.  This was more successful.  Now, as people walked by and we asked them if they listened to music on an iPod while running, they would come over to check out the app.  The best technique would be to present the iPhone and get the person touching the application as quickly as possible to retain their interest.

Conversation

A successful usability test requires the ability to approach strangers, be friendly,and politely guide them  into your test.  It was very easy to get sucked into a general conversation about the application or features the user would like to see.  It is also easy to get disappointed when someone walks by the table or leaves half way through the test.  But don’t get discouraged!  Discipline was required to get the needed results.

Email Signup List and Feedback Survey

No one wants to be the first person to sign up on the email list.  So we put a few names on the email signup list and people were then much more comfortable putting down their names.  Also, if you offer a real reason to sign up, such as a discount code you can mail to the person or offer news on the product, they were much happier to write down their email.

To do a follow up poll or survey, the most important thing is to have it on one piece of paper and to ask only the most critical questions.  Check boxes are much more effective than asking for long hand responses.  And limiting the number of check boxes make it easier for people to fill out and get a relevant response.

Setting up goals and deliverables

The next time out I will make sure there are specific goals for the tester.  This was RoadBud’s first usability test so we didn’t feel the need to have a specific set of questions we wanted answered.  But having goals for a test will make it more efficient because you can get guide people to the part of the software you want tested and solicit specific feedback.  Furthermore, being very specific and direct made engaging the tester simpler and more effective.

One also needs to be careful about the line between usability and customer development.  While both aspects are important to the lean startup, it is more effective to test your software with one concept in mind at a time.

I’d love to talk with you about running a commando test for your software and helping you save time and money.

Tagged with: , , , , ,
Posted in Testing ideas in the real world
5 comments on “Software commando usability test for $30 in Cliff Bars
  1. Beau Finley says:

    Nice. I can't wait to read the book "The Morrow Way: The Way to get Ahead in the World of Tomorrow"

  2. Well done. This was the first “commando test” I've ever done – and it was a hoot.

    We could have spent time on professional graphics and the rest, but this turned out to be unimportant. Provide a small gift/favor for people and they're happy to help.

    Eric – you did a terrific job running the whole test – from gathering people in, to leading them through the test, to compiling results and adding your ideas. I learned about flaws in our app that I could not see and have already made fixes in response. Nicely done.

  3. Simon says:

    Great tips.
    maybe you can elaborate more on the intersection between usability and customer development / lean startup method.

    greetings from Germany,

    simon

Leave a Reply

Your email address will not be published. Required fields are marked *

*