Add-On to a usability evaluation

Depending on the situation, your big company has an in-house UX lab facility with the respondent database with the team of the facilitator, or moderator.   However there is another situations, you simply don’t have the these resources.  Maybe in some cases, you have to go to the foreign countries to conduct research.  In these cases, it is common practice to use the field research or market research agency for the recruiting and facilities.

I would like to share some experience about these processes using the agencies.  There are many firms specializing in these job.  For example, there is WatchLab in Chicago.  Typically they are located in the prime central place with convenience access and has luxurious interior.  For it is basically real-estate business like hotel business, location is very important.  As you can imagine, the reason is that many people has to visit this place for the first time, and the time & cost to visit this place should be optimal.  So it makes sense to have an office at the down-town prime space.

For the first timer, price is very important for budget planning.  However there is nothing like average cost.  It really depends on the situation.  For example, pilot or medical doctors are expensive.  Smart phone users will be relatively cheap.   However as an example, an incentive for the consumer electronic product can be $100 for two hour.  And you have to pay additional $100 for the recruiting to the agency.  Sometimes you will want a professional facilitator specializing in UX evaluation, which can cost $1200 a day for 4 session of 1 hour long.  The nice rooms with the mirror-window which only researchers can see the participant with lots of AV equipment costs $1000 a day.  And maybe there will be a transcript service for $100 for an hour.  It is extremely busy, so catering maybe a good idea.  Add the sum yourself, and you can find it is not cheap process.

So you know the price, how can make it valuable process?  In my opinion recruiting( and analysis for next time) is very important.  For example, if you are conducting UX evaluation about the smartphone. You would like to include two groups of people, say novice users and advanced users. It will be very tempting to say I would like to recruit novice and advanced users in the meeting with the professional research recruiter. However it is very subjective what constitutes novice and advanced user group. So the busy research recruiter may have an entirely different point of view about the each group, and prepares inadequate samples for the each groups. So it is designer’s role to prepare more numeric and quantitative guideline for the recruiting, which characteristic each group has. Say novice group has less than one year of smartphone usage experience, and uses less than 5 apps daily, with the age group of 40~50, with the equal distribution of the male/female, and does not use social media apps using their smartphone,… and so on. And during the recruiting process, involve with the recruiter actively during the process, and requesting participant survey for confirmation is good idea.

Also in my opinion, determining the right number of participants are not so important than it sounds. More often than not, it is determined by the budget and resources to conduct the research. Remember more energy is required in the analysis of the result. So it is very practical that having a number of the users, designers can personally identify and “View” the entire session (which can be really boring after say 3 person), find the design improvement point, and do an analysis over the session.  I would like to also remark that it is physically demanding task.  Part of the reason is that it may start after say 6pm due to the working participant.

Finally I would like to briefly mention 2 services, which can help during the process.

The first is eye tracking.  For the software review, knowing the pattern of the user’s eye movement can help detect the problematic area, which maybe not available without the use.  It shows the inner working logic of the user mind by showing which area got attention and didn’t.  It is possible to track eyes for the desktop app and mobile app.  Also there is a rental firm of equipment and they also provide total service including the tech personel during the process and analysis report.

The second service is the streaming service.  Sometimes designers or stakeholders are far-away.  By streaming data real-time and providing video archive for the later reference, it is possible to participate during the process.  It also provides stereo channel where left side can be original sound, and right side can be real time translation.


6 thoughts on “Add-On to a usability evaluation

    • ^^; I would like to suggest the fact that there is no overlapping material here with the textbook as an evidence of reading, for I carefully avoided what already mentioned in the textbook.

      • You are too funny! 🙂 Now, be smart, too. Write a brief reading reflection that is about the readings. Change the title and the category for this post – and, voila! now you have 2 posts instead of 1. Do so as soon as possible.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s