RAA#4 Personas Revisited

Title & Author

Friess, E. (2012, May). Personas and decision making in the design process: an ethnographic case study. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’12). ACM, New York, NY, USA, 1209-1218.

Purpose of the research

Personas are well-known technique for the interaction design process. But as we can see the example of the RAA#2, the current status of adoption among the professional designers are unclear. This paper tries to investigate this problem using the ethnographic case study.

This paper received a CHI 2012 Best Paper Awards.

Methods

The author investigates the problem using ethnographic methods. He participated and observed how the designers use personas during the design process. To enable objective view, she used a linguistic analysis. The reason she used an ethnographic methods was that even though there are many claims about the personas, it is not yet empirically supported by the practice. (Remember at the IBM half of seasoned designers thought that personas was useless and didn’t used.) That means that the actual behavior of the designers can be differ from what they insist or think about personas.

So this paper begins with the summary of the previous claims about the personas like “Personas engender interest and empathy toward users” or “Personas help diffuse conflict or disagreement among team members when discussing possible design solutions.” Will it be really the case in real world? To see this, she participated in the design process of top tier design consulting firm, where personas are well accepted. She recorded every meeting where design decisions were made, and analyzed the script.

Main findings

Q: How often the personas were used in their talk?
A: About 2%

Q: Who invoked a persona?
A: Mainly designers. Not clients. In detail, there were 4 designers. And 2 of them participated in the field work and developed the personas. Not surprisingly, it was those 2, who developed the personas, who mentioned the personas most frequently. (85% of all persona mention)

Q: Which personas were invoked?
A: There were 8 personas. Only 3 of them explains the 97.8% of design talk. (This can lead to the practical insight of the number of personas).

Q: When were personas invoked?
A: During the scenario meeting for cognitive walkthrough. In other words, they made a story how each persona character will interact with the product.
Analysis

Q: How were Personas used in Decision-Making?
A: Mainly for Role playing(48%) and Focusing(34%).

  • An example of Role playing

“So, Dr. Samuels, she would probably not do that. She checks her timing, looks at the scree, and makes the diagnosis.”

  • An example of Focusing

“Well, that might be ideal for Georgia, but for now we needed to close in how autopilot can make this thing foolproof for Michael.”

Other than Role-playing and focusing, there were meeting maintenance(11%), empathy(3%), clarification(3%) and approximation (1%).

Analysis of the result

Despite the fact that this group spent several week interviewing end users and developing personas, they make relatively few mention in the actual design process except the meeting for the cognitive walkthrough. And the “common language benefit” is overly idealized for the clients never referred personas.

Instead of the persona (3%), they used their own opinions(25%) and story telling(21%) during the design process, for example to persuade design solution.

Conclusion

I have an opinion about the persona process. I think it is a like novel-writing using research data. Why designers do that? They do it to process the data mentally in higher level. During the process of personas creation, the designers unconsciously analyze the research data to create a story.  This enables deeper processing and understanding, which leads to the longterm memory.  So what is important is not the persona but the writing process itself.

So for me, it is not surprising to see that the persona itself is not so much used in the decision making process even in the top-tier firm where personas are widely accepted.  Because the designers “gained”, “digested” and “internalized” the insight about the target users during the personas process, they don’t have to rely on the personas for any specific purpose. The insights are shared among the designers who participated in the personas creating process. So they can focus on the design-domain specific issue using design language.

And it is not surprising for me, that those who created personas, used them most frequently(85% of all mention). My view is also supported by the author, who says “It appears that those who are involved in the creation of the persona have a better understanding of the personas and the user-at-large than those who are mere recipients of the personas.”

Also for me, designing using the personas developed by others are completely wrong use of the technique.

Finally one thing interesting to me is that an ethnographic study is usually done by the designers to understand the users. However in this paper, the study subject is the designers themselves.

“This reflexive move enables a descriptive assessment of how designers themselves actually use personas, not just how designers think they use personas or how designers think persons should be used.”

RAA#3: Web Experience Analysis

Title & Author

Website Experience Analysis: A New Research Protocol for Studying Relationship Building on Corporate Websites by Mihaela Vorvoreanu

Purpose of the research

If we think about the paradigm of the design again, there is an engineering and usability. Engineering means the product is working and usability means it is easy to use. But is that all? No. There is an UX which means the overal experience like pleasant to use. If we think the purpose of the website is relationship building with the public, it goes beyond usability. This paper proposes a process, web experience analysis to analyze this. For example, if one website wants to sell something to the public, it should build a relationship with the potential customer so that the company is trustable and the product usage will be interesting or pleasant. Previous methods only focused on the usability of the website, but in this case usability alone is not sufficient to persuade the customer. So the author proposes a new research protocol to measure what element of the website contributes to the relationship building.  Also previous methods try to analyze the contents of website itself, but it is the process of interacting with websites that builds relationship.

Methods

WEA tries to find the dimensions of the communications involved. For this paper, five dimensions which is trust, commitment, involvement, dialogue and openness. According to the these dimension, survey questionnaire was constructed. For example of trust, the following 2 question is asked to the participant after they review the site.

  • Do you feel you can trust this organization?
  • What on this website makes you feel this way?

They selected 9 corporate website according to the size. So there is large company, medium company and small company. The selection according to the size was done to ensure that the familiarity with the company name will not interfere with the each judgement. 6 participant was selected for each corporate websites. The number 6 is derived from the Nielsen’s suggestion that almost all of usability issue can be found with the testing of 5 users. They used a homogenous group of students who attended a class together.  This characteristic of the participant assures that the participants belongs to the same interpretative community.  The analysis focuses on the answer about which aspect of the sites influence the participants’ judgement about each relationship building dimension.

Main findings

According to the WEA analysis, we can find which element of the website can contribute to the each of five relationship dimension.  The main findings from the research was summarized below.

Trust: information about the organization’s history, background and value, visual characteristic like white background, quantity of the information

Commitment: consumer orientation,  maintaining only business relationship with its public, contact information

Investment: donation to charities, the very nature of an organization’s activity, graphics and pictures

Openness: detailed information about the business, quantity of the information, the availability of contact information

Dialogue: the availability of the contact information

But more important than the findings about specific value, the paper proposes a new research protocol, which is beyond a specific research example.  So the conclusion is that we can and should measure which element of website affect the relationship building process.

Analysis

I think human relies on as much as emotional experience as logical analysis.  For example, if I have to decide which graduate school to attend, I will rely not only the logical analysis but emotional impact of the website.  So in this perspective, I think this experience-centered analysis proposes a meaningful direction for the website or product analysis.

However, in my opinion, the choice of the dimension seems a little arbitrary.  For example, the difference between the dialogue and openness is not clear to me.  Also the dimensions can be different according to the goal of the website.  For example, the websites for the information delivery and persuasion will appeal to the different aspects of communication dimension.

And as the author points out, “the potential risk of an interpretive, experience-centered perspective are endless relativism and subjectivism”.  The author argues however there are interpretive communities who share a set of interpretive strategies.  The concept of clustered people is what I am personally pursuing.  In my opinion, I would like to investigate how different communities perceives or experiences the same website elements.  For example, if  use 3 websites and 3 different groups of 6 people can show this aspect.

Finally one participant’s comment made me think about the more holistic approach to the relationship building process.

“Well I would have to know more about the organizations.  The appearance of a website can be deceiving.  I would have to research the organization more to determine if I could trust them.”

As we are learning the online identity management, the same can be applied here.  Nowadays a website is not alone.  If I would like to know about a company, I will google the company.  When there is many negative contents associated with a company, then no matter how the company website try to give the trust, it would fail.

 

Conclusion 

I would like to add Web Experience Analysis protocol  to the user testing of previous CGT website.   The reason is that the decision to select graduate school is important and maybe feeling like emotional attachment can play important role in the process.  Due to the lack of the resources, we will have to choose most important dimension in using CGT website in the intention of graduate school selection.  It can be an trust or fun.  Our team will have to identify the key dimension to ask.

One minor detail I learned is that I have to delete cookies and temporary internet files.  The reason is that it can affect the loading speed or browsing experience.  We can apply that instruction in our usability testing.

Good Design: So you want to write a master piece?

 The purpose of the product:

The purpose of the product I want to introduce today is to compose a creative writing.  By creative writing, I mean the kind of writing for which there is no determined content yet.  For example, the daily business reports with given format is not a creative writing, while an academic paper or literature like novel or poem is.  One characteristic of such writing is that the emphasis is usually on the content not the format.  And it requires substantial human focus, for it is very easy to distract.  One way of explaining this distraction is that one’s brain is lazy, so it does not want to work or in this case ‘think’.  So it tries to escape creative writing requiring thinking whenever possible.

 The two product of comparison:

I would like to compare two great software for the writing, whose purpose is slightly different.  The two product of comparison is the Microsoft Word program and Writer program by Information Architect, Inc.  In the functional aspect, Word is a lot more powerful than Writer.  However does this functionality can be additive to the productivity of creative writing?  I don’t think so.  By offering so many possibilities in any given situation, Word program adds more distraction.  On the contrary the Writer program offers two modes to assist the creative writing.  One is full screen mode.  It hides everything except the neatly organized text.   And the other is focus mode.  This mode slightly fades every  text other than a paragraph cursor is located.

I will choose iA writer for creative writing.  As Antoine de Saint-Exupery said, “Perfection (in design) is achieved not when there is nothing more to add, but rather when there is nothing more to take away.”  By reducing the unnecessary distraction it enables more focused environment.  It is interesting that in the recent version of Microsoft Word, there is new function called “Full Screen”, and its function is similar to the Writer.

 

 

Do you think classroom activity is too fast?

Then check this out!

It seems that seasoned professional interaction designer also does it.  They gathered for 4 days.  And they worked hard 9 to 5 to produce a interaction design which is wireframe model.  There is two main course in the Cooper U, which is interaction design workshop and UX bootcamp.  I attended design workshop, and I hope to attend UX bootcamp.  Although it is quite expensive, I think it’s worth every penny.   Also this UX bootcamp was held on the Allen Cooper’s farm house.

RR UXB CH16~17

This week I tried another approach about reading reflection.  Previously I wrote a review after the reading the text book entirely, which can be too abstract or vague, or sometimes additive not mentioning the textbook.  So today I tried to write what I learned or questioned from the reading in the process of the reading.  The disadvantage is it is too detail and there is no whole subject in the passage.  The advantage is that it reflects more detail about the reading.  So I want some feedback about which is better.  If you are kind reader, please leave a comment, which do you prefer.

1. For iteration is basically circular process, it can scare managers.  Usability report showing each iteration is getting better can be something relieving this worry for managers.

2. Setting a quantitive target level for UX measure seems too arbitrary too me.  For it then proposes a difficult problem of how to setting target value.  I think maybe common sense will be better in this case.

3. I learned a lot reading the “Not All errors are created equal”.  I had a doubt about relying on only the quantitive aspect of performance evaluation.  However there is a various kind of errors and it will be impossible to eliminate all errors.  For example voice dictation can produce many errors.  But rather than reviewing and correcting all the errors, just showing the memo with error can be better solution, because users are usually busy when they are making voice memo and they usually don’t have a problem recognizing the content with errors.

4.  When to stop improving was a nice question during the Apple CTO presentation.  Even though the book mention this problem, there was just general remarks about it like when the team feels enough, the budget and time limit.

5. Formative data analysis method by Whitney Quensenbery was very interesting to me.  For what I felt the hardest part was gathering all stakeholder and let them watch the video patiently.   It would be perfect only if I could do that.

6.

“We suggest bringing in the problem analyst as early as possible, especially if the analyst is not on the data collection team.”

“We set up the evaluator team to ensure that someone with the requisite designer knowledge will be present during the evaluation session to include that information in the UX problem instance content that we now need in this transition to data analysis.”

This sentence really worries me.  I think my domain and the author’s domain is not overlapping.  Maybe it’s because I am follower of Cooper and IDEO method, which usually team of 2~5 members for the design.  In my standard, there is nothing like problem analyst and data collection team and designer.  The designer learns about the problem through User Research and he/she/team collects the data themselves, because he/she/team want to see the non-verbal subjective interaction themselves.   However there seems to be a trend in this specialization of the role, which is reflected in this Persona usage paper too.

7. The distinction between Critical incidents and UX problem instances  and was quite new, although they looked quite useless.

8. I learned how to decide how to decide priority between UX problem systematically.  The breakdown of factors like importance and cost and calculating ratio was new.  And distinguishing group cost and single cost was refreshing.  Putting the UX problems in quadrant was interesting,too.  Also the adding actual cost estimates so that we can get feedback is clever, even though it might be a little sophisticated.   The cumulative cost and the line of affordability concept was simple but effective decision method.

9.  Reading chapter 17, the author clear states that these report is for the internal use.  And if it should be for the other people than design team, it should be careful about it use.  Reading this, I began to think maybe I am focusing only on the small domain of the interaction design, where small design team completes the interaction design part and pass it to the dev team.  There maybe other domains, where the engineering part holds the key or it has a large organization where the role of the design is limited in to changing existing designs.   For example, if I am an interaction designer in Google, and if I would like to change the labels, which I think confusing, I can’t tell them change it.  I would have to gather evidence to support my claim like user testing of 100 people showing it is really confusing.   I am beginning to drop my prejudice and be open minded to accept the new learnings from the book.  It is quite refreshing how reading books changes the understanding.

10. I learned that there is an industry standard for the usability report by ANSI and ISO.  However these are for summative reports.  Even though most of the usability report is formative reports, the scope, audience, format is so different between reports, there is not yet standard for formative reports.

11. A individual problem reporting may contain following content

  • The problem description
  • a best judgement of the causes of the problem
  • its severity or impact
  • suggested solutions

12.  The section about “introducing UX engineering to your audience” is general, but contains practical advice for the situation, where you have to explain to sell your role to get support.  Even though UX has been popular nowadays, many other team may not be familiar with the concept yet.  So starting with teaching concepts is good approach.

12. There is many practical advice on how to survive office politics by “How to win friends and influence people” like manner.  Even though they are general and a little bit off scope of this book, it is useful.  I began to see this book as a handbook which tries to explain every detail.  It may not be the pleasant reading, because if you know it, it can be too verbose.   But it will be useful as reference, when you would like to make checklist.

13.  The section about how to effectively convince and sell the UX problem was interesting and useful.   After all, if you are not Steve Jobs, you have to convince others about the problem to fix it.   Because it is hot issue, there is no given answer.  Some prefer design recommendation, while do not actually use it.  Also an estimated effort to fix a problem was not effective.

14. Even though I knew that delivering report alone will not make any change, I would like to emphasize it again.  You should explain it again and again with face-to-face meeting.

So that’s all!  Thanks for reading.

Research and Design

There is something I don’t like about this UXbook.  And there was something in me, which prevented writing research article analysis.  Today’s talk is about it.  And it can be very abstract, which I don’t like. I feel sorry about you who read this.

The academic research and design is both intellectual creative process.  However they differ in the direction they pursue.  The academic research or scientific methods are essentially based on the replicability.  It means if some experiment shows some result, another one who does the same experiment should get same similar result.   So for example, if we conduct user research, and you conduct user research, you and I should make similar conclusion abou the user, for the users are same.  That’s where the assumption of the research lies.

However the design process is all about unique.  If two of the designer shows same design, you have fire one of them.  And it really matters who does it, than which methodology or process he/she uses.  Jonathan Ive is one of most famous designer.  If he designs computer, and I designs computer, it will be very different.  So it is this “Think different”, which is different from scientific method.

So why the designer conducts the user research?  He/she wants to understand the users, to create solution for the user.  Why the designer conducts usability testing?  To get feedback of his/her design to improve it.   The process of the writing persona and the process of the writing usability report is to get an insight.   That’s why hiring other’s to do the usability research or using persona made by other is not right use of the method.  User research and usability testing is not a goal but a tool.   Even if I make 100 persona, my design can still suck if I don’t have such creativity.

However there is some other professional who does the same thing like user research (or more generally market research) and usability testing.  Their goal is to assess the user and the product scientifically.  For these people the user research and usability testing is a goal.  And they usually present it with nice report or scientific paper, with chart and graph.

So what is the problem of the UXBook?  I think there is too less material about the creative design process (Or interaction design) and too much about the UX professional things.   This kind of the detail about the usability testing is just too much.  It is for the report for others, not the designer himself.   Even though there is nothing wrong with the making perfect report, it will take designer’s time and resource, and the perfect report will not help the designer for the designing of great product.  And that is the reason why I don’t feel natural about reading research article.

However after reading chapter 17, the author clear states that these report is for the internal use.  And if it should be for the other people than design team, it should be careful about it use.  Reading this, I began to think maybe I am focusing only on the small domain of the interaction design, where small design team completes the interaction design part and pass it to the dev team.  There maybe other domains, where the engineering part holds the key or it has a large organization where the role of the design is limited in to changing existing designs.   For example, if I am an interaction designer in Google, and if I would like to change the labels, which I think confusing, I can’t tell them change it.  I would have to gather evidence to support my claim like user testing of 100 people showing it is really confusing.   I am beginning to drop my prejudice and be open minded to accept the new learnings from the book.  It is quite refreshing how reading books broadens the understanding.