RAA#4 Personas Revisited

Title & Author

Friess, E. (2012, May). Personas and decision making in the design process: an ethnographic case study. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’12). ACM, New York, NY, USA, 1209-1218.

Purpose of the research

Personas are well-known technique for the interaction design process. But as we can see the example of the RAA#2, the current status of adoption among the professional designers are unclear. This paper tries to investigate this problem using the ethnographic case study.

This paper received a CHI 2012 Best Paper Awards.

Methods

The author investigates the problem using ethnographic methods. He participated and observed how the designers use personas during the design process. To enable objective view, she used a linguistic analysis. The reason she used an ethnographic methods was that even though there are many claims about the personas, it is not yet empirically supported by the practice. (Remember at the IBM half of seasoned designers thought that personas was useless and didn’t used.) That means that the actual behavior of the designers can be differ from what they insist or think about personas.

So this paper begins with the summary of the previous claims about the personas like “Personas engender interest and empathy toward users” or “Personas help diffuse conflict or disagreement among team members when discussing possible design solutions.” Will it be really the case in real world? To see this, she participated in the design process of top tier design consulting firm, where personas are well accepted. She recorded every meeting where design decisions were made, and analyzed the script.

Main findings

Q: How often the personas were used in their talk?
A: About 2%

Q: Who invoked a persona?
A: Mainly designers. Not clients. In detail, there were 4 designers. And 2 of them participated in the field work and developed the personas. Not surprisingly, it was those 2, who developed the personas, who mentioned the personas most frequently. (85% of all persona mention)

Q: Which personas were invoked?
A: There were 8 personas. Only 3 of them explains the 97.8% of design talk. (This can lead to the practical insight of the number of personas).

Q: When were personas invoked?
A: During the scenario meeting for cognitive walkthrough. In other words, they made a story how each persona character will interact with the product.
Analysis

Q: How were Personas used in Decision-Making?
A: Mainly for Role playing(48%) and Focusing(34%).

  • An example of Role playing

“So, Dr. Samuels, she would probably not do that. She checks her timing, looks at the scree, and makes the diagnosis.”

  • An example of Focusing

“Well, that might be ideal for Georgia, but for now we needed to close in how autopilot can make this thing foolproof for Michael.”

Other than Role-playing and focusing, there were meeting maintenance(11%), empathy(3%), clarification(3%) and approximation (1%).

Analysis of the result

Despite the fact that this group spent several week interviewing end users and developing personas, they make relatively few mention in the actual design process except the meeting for the cognitive walkthrough. And the “common language benefit” is overly idealized for the clients never referred personas.

Instead of the persona (3%), they used their own opinions(25%) and story telling(21%) during the design process, for example to persuade design solution.

Conclusion

I have an opinion about the persona process. I think it is a like novel-writing using research data. Why designers do that? They do it to process the data mentally in higher level. During the process of personas creation, the designers unconsciously analyze the research data to create a story.  This enables deeper processing and understanding, which leads to the longterm memory.  So what is important is not the persona but the writing process itself.

So for me, it is not surprising to see that the persona itself is not so much used in the decision making process even in the top-tier firm where personas are widely accepted.  Because the designers “gained”, “digested” and “internalized” the insight about the target users during the personas process, they don’t have to rely on the personas for any specific purpose. The insights are shared among the designers who participated in the personas creating process. So they can focus on the design-domain specific issue using design language.

And it is not surprising for me, that those who created personas, used them most frequently(85% of all mention). My view is also supported by the author, who says “It appears that those who are involved in the creation of the persona have a better understanding of the personas and the user-at-large than those who are mere recipients of the personas.”

Also for me, designing using the personas developed by others are completely wrong use of the technique.

Finally one thing interesting to me is that an ethnographic study is usually done by the designers to understand the users. However in this paper, the study subject is the designers themselves.

“This reflexive move enables a descriptive assessment of how designers themselves actually use personas, not just how designers think they use personas or how designers think persons should be used.”

Advertisements

RAA#3: Web Experience Analysis

Title & Author

Website Experience Analysis: A New Research Protocol for Studying Relationship Building on Corporate Websites by Mihaela Vorvoreanu

Purpose of the research

If we think about the paradigm of the design again, there is an engineering and usability. Engineering means the product is working and usability means it is easy to use. But is that all? No. There is an UX which means the overal experience like pleasant to use. If we think the purpose of the website is relationship building with the public, it goes beyond usability. This paper proposes a process, web experience analysis to analyze this. For example, if one website wants to sell something to the public, it should build a relationship with the potential customer so that the company is trustable and the product usage will be interesting or pleasant. Previous methods only focused on the usability of the website, but in this case usability alone is not sufficient to persuade the customer. So the author proposes a new research protocol to measure what element of the website contributes to the relationship building.  Also previous methods try to analyze the contents of website itself, but it is the process of interacting with websites that builds relationship.

Methods

WEA tries to find the dimensions of the communications involved. For this paper, five dimensions which is trust, commitment, involvement, dialogue and openness. According to the these dimension, survey questionnaire was constructed. For example of trust, the following 2 question is asked to the participant after they review the site.

  • Do you feel you can trust this organization?
  • What on this website makes you feel this way?

They selected 9 corporate website according to the size. So there is large company, medium company and small company. The selection according to the size was done to ensure that the familiarity with the company name will not interfere with the each judgement. 6 participant was selected for each corporate websites. The number 6 is derived from the Nielsen’s suggestion that almost all of usability issue can be found with the testing of 5 users. They used a homogenous group of students who attended a class together.  This characteristic of the participant assures that the participants belongs to the same interpretative community.  The analysis focuses on the answer about which aspect of the sites influence the participants’ judgement about each relationship building dimension.

Main findings

According to the WEA analysis, we can find which element of the website can contribute to the each of five relationship dimension.  The main findings from the research was summarized below.

Trust: information about the organization’s history, background and value, visual characteristic like white background, quantity of the information

Commitment: consumer orientation,  maintaining only business relationship with its public, contact information

Investment: donation to charities, the very nature of an organization’s activity, graphics and pictures

Openness: detailed information about the business, quantity of the information, the availability of contact information

Dialogue: the availability of the contact information

But more important than the findings about specific value, the paper proposes a new research protocol, which is beyond a specific research example.  So the conclusion is that we can and should measure which element of website affect the relationship building process.

Analysis

I think human relies on as much as emotional experience as logical analysis.  For example, if I have to decide which graduate school to attend, I will rely not only the logical analysis but emotional impact of the website.  So in this perspective, I think this experience-centered analysis proposes a meaningful direction for the website or product analysis.

However, in my opinion, the choice of the dimension seems a little arbitrary.  For example, the difference between the dialogue and openness is not clear to me.  Also the dimensions can be different according to the goal of the website.  For example, the websites for the information delivery and persuasion will appeal to the different aspects of communication dimension.

And as the author points out, “the potential risk of an interpretive, experience-centered perspective are endless relativism and subjectivism”.  The author argues however there are interpretive communities who share a set of interpretive strategies.  The concept of clustered people is what I am personally pursuing.  In my opinion, I would like to investigate how different communities perceives or experiences the same website elements.  For example, if  use 3 websites and 3 different groups of 6 people can show this aspect.

Finally one participant’s comment made me think about the more holistic approach to the relationship building process.

“Well I would have to know more about the organizations.  The appearance of a website can be deceiving.  I would have to research the organization more to determine if I could trust them.”

As we are learning the online identity management, the same can be applied here.  Nowadays a website is not alone.  If I would like to know about a company, I will google the company.  When there is many negative contents associated with a company, then no matter how the company website try to give the trust, it would fail.

 

Conclusion 

I would like to add Web Experience Analysis protocol  to the user testing of previous CGT website.   The reason is that the decision to select graduate school is important and maybe feeling like emotional attachment can play important role in the process.  Due to the lack of the resources, we will have to choose most important dimension in using CGT website in the intention of graduate school selection.  It can be an trust or fun.  Our team will have to identify the key dimension to ask.

One minor detail I learned is that I have to delete cookies and temporary internet files.  The reason is that it can affect the loading speed or browsing experience.  We can apply that instruction in our usability testing.

RAA#2 Persona: The Journey is the reward.

Caution: If I say, someone misunderstood something, it implies that my view is right and theirs are wrong.  However I understand that there is always possibility that it is actually me who is wrong.   I would like to inform you that I understand this.  But I will say “They misunderstood” instead of “Even though there is possibility that I can be wrong, I think that they are misunderstood” to save typing, so saving the earth, and shortening sentences so that it is more easy to read.

So without further ADO, let’s start!

Title & Author

How Do Designers and User Experience Professionals
Actually Perceive and Use Personas? by Tara Matthews, Tejinder K. Judge, and Steve Whittaker

Purpose of the research 

Persona seems to be the source of the debate.  Is it effective for design?  If effective, in what aspects is it effective? Does real-world experienced designers use this technique in their design process?  If not, why?   This research paper tries to answer this kind of answers.

Methods

They gathered 12 very experienced designers and UX professional into 2 hour interview.  It seems they belongs to the IBM. First they asked about their background and experience with the persona.  Then during the process, the authors asked the designers to design something providing persona.  And the authors watched their discussion and analyzed their conversation to check whether persona is used in the design process.

Main findings

They categorized the participants according to their opinion about persona, which is Persona champion, Persona Moderate and Persona pessimist.  After analyzing the interview script, they found that most designers DID NOT use persona during the design process.  They just tried to the extract user information like age, role, degree of skills and so on, and relied in these information to design.  However everybody agreed that Persona is great for communication with other people like stakeholders or developers. The reason they didn’t use persona is, they thought persona is too abstract or too-much personal detail which is a fiction is misleading or distracting other characteristic that is based on the user research.  Interestingly the three designers who attended  Cooper University or have created persona themselves was persona champion and had an opposite view against about these reasons why other persona pessimist didn’t use persona.

Analysis

Frankly speaking, I was so surprised how many misunderstanding about the persona process is still there.  Also there was very dubious industry practice was developing which is distinguishing UX and design, and UX doing research and persona and passes them to design.  Let me explain this more.

First the authors requested the designers to use persona developed by others during the process.  It seemed to me that the authors themselves did not understand the persona developing process.   The persona is like writing novel based on user research.  Why we do that?  It is a high-level processing of the information to make structure in our mental model and longterm memory so that it can be better understood and more easier to recall.  Simply speaking if we have to memorize a book for exam, it is better for us to make a structured mindmap than just reading it again and again.  So the whole purpose of the persona is writing the persona.  The process is to promote the sympathy of the designer over the user.  In other words, designers tries to be in user’s shoe to understand them more thus making better design for them.  One similar approach is if designing for the blind, the designer himself will try to close his eyes while doing daily task so that he can understand better about being a blind.    So the whole value of the persona lies in the developing it.  The journey is the reward.  It is not outcome of process but the process itself, which is important.

To be honest with you, I rarely use personas myself… I find them useful for requirements gathering and I actually find them useful for the client. But I don’t design with them or design off of them. The reason is that it’s a great tool to get people to start thinking about the thing they are building in terms of users but in my experience, personas tend to be overly optimistic. They describe the best-case scenario for the perfect user who is incredibly enthusiastic about the system… Personas are not helpful because the users don’t match up to the personas.

-> The reason he thinks persona is not reflecting the user is because he/she didn’t participate in the persona generation process including user research.

You can’t design to somebody else’s understanding of the problem. You need to be part of the user research to design something. That is where the best designs come from. –E3, Designer

What I don’t like is how distilling something into a persona, so for example if I’m making an e-commerce app and I can take all 40-something women shoppers in the Mid-West and turn that into ‘Katie’ my persona, I feel like the generalization that is being made at that point, makes me feel slightly uncomfortable, rather than just having the body of research to start with. – M1, Designer

Also the distinction of the role between UX and design makes me confusing.  Does design means interaction or visual design?  UX means interaction design or user/usability research?  It seems that companies are trying to make distinction between those who conduct user research and those who do the design.   However according to the cooper, the essence is the designer participates actively in user research.  And the persona is outcome of their understanding of the user research.   Comment like because the persona is too abstract, I will use user research data itself when  there is enough time, makes me wonder if he/she understood the concept of persona.

Finally about the detail, if the designer himself participated in the user research, he will know what is important user characteristic and what is an interesting make-up of detail to make it real.  However if the designer didn’t participated in the user research, the designer does not know this, and what he tries to do is doing this process in his mind using the stereo-typical assumption about the user.  That was the pitfall where persona wanted to avoid.    The details of persona is like a detailed gesture of actors who would like to convey the characteristics of the role nonverbal way.

 

Additional comment – Added later

After reading UXB chapter 17,  I began to think maybe I am focusing only on the small domain of the interaction design, where small design team, like Cooper or IDEO completes the interaction design part and pass it to the dev team.  There maybe other domains, where the engineering part holds the key or it has a large organization where the role of the design is limited in to changing existing designs.   For example, if I am an interaction designer in Google, and if I would like to change the labels, which I think confusing, I can’t tell them change it.  I would have to gather evidence to support my claim like user testing of 100 people showing it is really confusing.   I am beginning to drop my prejudice and be open minded to accept the new learnings from the book.  It is quite refreshing how reading books changes the understanding.

If I apply this understanding to the this article and industry trend, it makes sense that, at least for some people, persona is useless.  Because persona was developed in the context of small design team completing interaction design process, it does not fit in the other situation, where the engineering may hold keys or such a large organization with existing product.

RAA1: Beauty is in the eye of beholder. Cultural difference about usability.

Improving Performance, Perceived Usability, and Aesthetics with Culturally Adaptive User Interfaces

Katharina Reinecke and Abraham Bernstein, University of Zurich

Purpose of the research 

We all know the aesthetics plays important role in the usability and performance.  Using pretty interface, the user likes it better, performs better.  However the problem is beauty is really subjective thing.  And certainly culture is one of the factors deciding the standards for the beauty.  I remember doing class presentation with team about excise, and Eddy showed Baroque-like chair from china and modern zen chair as an example of different perception of aesthetics. Then how about showing the users interfaces built according to their cultural norms, instead of showing same interfaces for the all global users?  Will the customers like it better and perform better?  This paper tries to answer that question.

Methods

First, how we can decide the one’s culture? It is certainly different from the country of origin.  The paper made a average of the countries where one user stayed.  So the more one user stayed one country, the more he/she is adapted to that culture.  So that is user model.

How about the cultural difference in the design of the interface.  There is very famous work by cultural anthropologist Hofstede*, where he studied the characteristics of IBM employees from different culture.  According to it, there is 5 dimension in the culture, like Power distance, Individualism, Masculinity, Uncertainty Avoidance, Long Term Orientation.  According to the this dimension, they made a specific design rule according to the high/low value.   That becomes a rule for the that culture.  So that is the culture/UI rule.

And finally we need a test platform to do this.  The author made a service website devoted to the study called MOCCA. It is basically to-do-list website.  They gathered 41 international participants.  Basically they ask users where have you been, and generated rule for the interface, and adapt interface according to the rule and show it to the user.

Main findings

Not surprisingly, the result shows that users were 22% faster using the culturally adapted interface, needed fewer clicks, and made fewer errors.  Also subjective methods (asking users to evaluate themselves), shows the users thought the adapted version was easier to user.

Analysis

When we think about the design paradigm, the fact is that the bars are getting higher.  At first something working reliably working is welcome.  Then it should be easy to use to win the competition.  Finally at the fierce competition around matured product,  it should be pleasant or fun to use to survive in the market.  And aesthetics plays important role in the pleasant experience.   For example, Apple is one of the most successful company, for they are really good at providing beautiful experience.

However when we think about aesthetics, it is really subjective, isn’t it?  So the previous paradigm, where the omnipotent designer worked really hard to create universally absolutely beautiful product is not applicable any more.  Rather the design output can be a function of various inputs like culture, rather than a static point.  It is this papers value that shows diligently this direction.  It is easy to assume it.  But to test this assumption, they made a working sample of website and developed rule for the adaptation.  And invited 41 users to test it, and evaluated their result to show that this assumption is valid.  It is not a small task.

However I would like to extend this theme a little more.  For me,  culture is certainly one dimension in the optimal design function.  Other can be sex/age/income and so on.  At this point it becomes more complex multidimensional statistical problem.  So which dimension affects how much becomes hard to tell. Then finding a hidden theme becomes more of user clustering or machine learning.  There is many users of various taste.  Even person from same culture shows different taste.  So this concept of culturally adapted interface should be extended to the taste adapted interface.

Personally aesthetics has been always my weak point.  As you can see my motto “Intuinno” , I work to make innovation (which is science or engineering driven) intuitively (which is about usability), but that’s all.  I never approached the aesthetics with the same confidence with other two discipline.  That’s where team work shines.  If there is world class team, like in TV series in CSI, I can play the usability part (Anyway my engineering is also under average, even though I can make it barely work. )  and Jonathan Ive can do the aesthetics part.

Finally here is some interesting pictures of the beauties from all over the world.   You can see we are different.

Reference 

Katharina Reinecke and Abraham Bernstein. 2011. Improving performance, perceived usability, and aesthetics with culturally adaptive user interfaces. ACM Trans. Comput.-Hum. Interact. 18, 2, Article 8 (July 2011), 29 pages. DOI=10.1145/1970378.1970382  http://doi.acm.org/10.1145/1970378.1970382

Photos examples of Beauties.  National Geographic channel. The link is  http://www.allvoices.com/contributed-news/3798150/content/36379222-beauty

Hofstede, G. 2001. Culture’s Consequences: Comparing Values, Behaviors and Organizations across Nations 2nd Ed. Sage Publications.