RR UXB CH16~17

This week I tried another approach about reading reflection.  Previously I wrote a review after the reading the text book entirely, which can be too abstract or vague, or sometimes additive not mentioning the textbook.  So today I tried to write what I learned or questioned from the reading in the process of the reading.  The disadvantage is it is too detail and there is no whole subject in the passage.  The advantage is that it reflects more detail about the reading.  So I want some feedback about which is better.  If you are kind reader, please leave a comment, which do you prefer.

1. For iteration is basically circular process, it can scare managers.  Usability report showing each iteration is getting better can be something relieving this worry for managers.

2. Setting a quantitive target level for UX measure seems too arbitrary too me.  For it then proposes a difficult problem of how to setting target value.  I think maybe common sense will be better in this case.

3. I learned a lot reading the “Not All errors are created equal”.  I had a doubt about relying on only the quantitive aspect of performance evaluation.  However there is a various kind of errors and it will be impossible to eliminate all errors.  For example voice dictation can produce many errors.  But rather than reviewing and correcting all the errors, just showing the memo with error can be better solution, because users are usually busy when they are making voice memo and they usually don’t have a problem recognizing the content with errors.

4.  When to stop improving was a nice question during the Apple CTO presentation.  Even though the book mention this problem, there was just general remarks about it like when the team feels enough, the budget and time limit.

5. Formative data analysis method by Whitney Quensenbery was very interesting to me.  For what I felt the hardest part was gathering all stakeholder and let them watch the video patiently.   It would be perfect only if I could do that.


“We suggest bringing in the problem analyst as early as possible, especially if the analyst is not on the data collection team.”

“We set up the evaluator team to ensure that someone with the requisite designer knowledge will be present during the evaluation session to include that information in the UX problem instance content that we now need in this transition to data analysis.”

This sentence really worries me.  I think my domain and the author’s domain is not overlapping.  Maybe it’s because I am follower of Cooper and IDEO method, which usually team of 2~5 members for the design.  In my standard, there is nothing like problem analyst and data collection team and designer.  The designer learns about the problem through User Research and he/she/team collects the data themselves, because he/she/team want to see the non-verbal subjective interaction themselves.   However there seems to be a trend in this specialization of the role, which is reflected in this Persona usage paper too.

7. The distinction between Critical incidents and UX problem instances  and was quite new, although they looked quite useless.

8. I learned how to decide how to decide priority between UX problem systematically.  The breakdown of factors like importance and cost and calculating ratio was new.  And distinguishing group cost and single cost was refreshing.  Putting the UX problems in quadrant was interesting,too.  Also the adding actual cost estimates so that we can get feedback is clever, even though it might be a little sophisticated.   The cumulative cost and the line of affordability concept was simple but effective decision method.

9.  Reading chapter 17, the author clear states that these report is for the internal use.  And if it should be for the other people than design team, it should be careful about it use.  Reading this, I began to think maybe I am focusing only on the small domain of the interaction design, where small design team completes the interaction design part and pass it to the dev team.  There maybe other domains, where the engineering part holds the key or it has a large organization where the role of the design is limited in to changing existing designs.   For example, if I am an interaction designer in Google, and if I would like to change the labels, which I think confusing, I can’t tell them change it.  I would have to gather evidence to support my claim like user testing of 100 people showing it is really confusing.   I am beginning to drop my prejudice and be open minded to accept the new learnings from the book.  It is quite refreshing how reading books changes the understanding.

10. I learned that there is an industry standard for the usability report by ANSI and ISO.  However these are for summative reports.  Even though most of the usability report is formative reports, the scope, audience, format is so different between reports, there is not yet standard for formative reports.

11. A individual problem reporting may contain following content

  • The problem description
  • a best judgement of the causes of the problem
  • its severity or impact
  • suggested solutions

12.  The section about “introducing UX engineering to your audience” is general, but contains practical advice for the situation, where you have to explain to sell your role to get support.  Even though UX has been popular nowadays, many other team may not be familiar with the concept yet.  So starting with teaching concepts is good approach.

12. There is many practical advice on how to survive office politics by “How to win friends and influence people” like manner.  Even though they are general and a little bit off scope of this book, it is useful.  I began to see this book as a handbook which tries to explain every detail.  It may not be the pleasant reading, because if you know it, it can be too verbose.   But it will be useful as reference, when you would like to make checklist.

13.  The section about how to effectively convince and sell the UX problem was interesting and useful.   After all, if you are not Steve Jobs, you have to convince others about the problem to fix it.   Because it is hot issue, there is no given answer.  Some prefer design recommendation, while do not actually use it.  Also an estimated effort to fix a problem was not effective.

14. Even though I knew that delivering report alone will not make any change, I would like to emphasize it again.  You should explain it again and again with face-to-face meeting.

So that’s all!  Thanks for reading.


RR UxB Ch14 & 15: Usability testing

I will share a few of my reflection over the books.  First, for me,  conducting an formal usability process with recruited participant using a paper prototype is too much and has a high possibility of the waste of money.  Maybe paper prototype is more of a designer language than a user/consumer language.  And formal usability testing is an expensive and long process.  So it should located in the later process. (Not final stage like Microsoft Windows 8)  However if we can define usability testing more generally, so that not only the formal usability testing with granular bar but also a hallway kidnapping of comrades to receive a feedback about certain interaction design feature,  paper prototype is certainly the best weapon for designers.  In this sense, family and friends are important asset for designer so that they can be exploited for testing.

Anyway I would like to briefly mention importance of the video recording.  It can help solving the tedious problem of performance time recording so that participating designer can focus more on the qualitative cues like “facial expression, mouse searching patter and untold problem the user may have.   For our objective is improving the design, a short clip can help persuading stakeholder or developer who thinks this state is good enough.  It is already working, why should I change this! Showing a repeated pattern of frustrated user struggling is a good method to change this thinking.  It is a good practice marking the video so that I can refer it later, if the software package permits. But even if you can’t mark the recording, leaving a short memo about this ,will remind you to search the video to see it again.

One thing I would like to point is that the suggested evaluation metric like performance time, number of error, user surveys are usually dealing with the usability part of the interaction design.  However human are not only logical but emotional.  And as you can remember in the paradigm of the design, emotional part like aesthetics, pleasant to use should be an important design paradigm.  If you can’t measure something, you can’t improve it.  So how we can evaluate emotional performance is getting more important.  I am expecting to see “Web Experience Analysis” by Dr. Mihaela Vorvoreanu to solve such a problem.


RR: Info Architecture; Prototyping

Chapter5: Information Architecture

Information architecture is organizing information so that it is easy to use.  If we apply paradigm of design we learned last week, information architecture deals with the paradigm of engineering and paradigm of usability. (You can see nice pyramid about paradigm of design here)  So it is about how to make information easy to find.   So many interesting theories like information foraging is used. It will be especially useful, when there is many information in certain places, when the user will have to guess where the related information is located.  Providing self-relevant categories and many hints(scent) about where the info may lies is key task for the information architect.

There can be a really wide variety of category over specific items.  However some of the user-centered techniques for creating such architecture includes card sorting ,closed card sorting, similarity matching.  They request users to categorize and label various items.  The reason these are user-centered is that from various categorization available, designer follows what the customer thinks naturally.  One concept useful in this approach that is not discussed in the article is the concept of user’s mental model and designer’s representation model.  By making these similar, we can achieve optimal organization.  And that’s why we designers are interested in the model in user’s mind.

There are many topology certain sites can have.  For example there is hierarchy(tree), linear (sequence), matrix, full mesh, arbitrary network and hybrid.  Even though there is no absolute answer to the topology, a topology is adequate at certain situation.  One thing I would like to note is the horizontal vs vertical organization.  If there is many contents, horizontal organization with many links at certain level which means short required level.  Also the vertical organization means at certain level there is small number of links, which means deep required level.  It seems horizontal organization performs generally better than the vertical organization.  It seems because there are many choices, the probability of error also increases.  So in vertical organization where the user has to make many choices, there can be more errors.  So heuristically sublevel below 3 is now recommended.

Chapter 11: Prototyping

Let me begin with the different types and fidelity levels of prototypes.  There is horizontal and vertical prototype. Also there is variations like T and local prototype.  In terms of fidelity there is low, middle, high-fidelity prototypes.  Also if we consider the interactivity, there is scripted and click-through, a fully programmed, Wizard of Oz type, physical mockup and animated prototype.  So there is many types of prototype, and finding a adequate one to try seems daunting task.  Or is it?  Well, it seems common sense and reality check with time, budget and man power can reduce many choices into a few feasible rational selection.

1. First, matching the level of fidelity with the stage of progress is good idea.  That means start with low fidelity, proceed to the high quality pixel perfect prototype.

2. You should know using programming makes it complex, because nature of program is that it should be perfect to work even for the prototype.  Pen and paper are easy to create variations.  So begin with the pen and paper and proceed to the programming if it is required.  Please keep in mind that even though the prototyping code looks functional, development will prefer starting fresh.

3. Vertical prototyping is nice to test a certain feature which requires sequential interaction like check out. Horizontal prototyping is good for the general look and feel and feature set.

4. There is times when high fidelity prototype is required.  When the time comes, you will probably know.  Such event includes VC pitch, Big boss meeting, request from technical sales.

However there are pitfalls when using prototyping.  I already mentioned pitfalls with the programming in the previous blog post. There is a few more pitfalls.  Interestingly, there is a pitfall with low fidelity prototype and pitfalls with high fidelity prototype.

1. Low fidelity prototype is nice if it is used with the people who understands the design process.  However there are many cases where people don’t know about it.  If the people is your boss, client or stakeholder, it can be problematic.  For example, they may think you are an inexperienced amatuer (Maybe it is true that you are inexperienced amateur.  Not in the sense, your design skill is not good enough but you are not experienced with clients, if you showed it to someone who can think like that. )

2. High fidelity prototype also has its own pitfall.  For example, the clients sees the prototype, they think it’s done and may even try to save money by declaring the job done.  Other than that if you have a nice slick look and feel, most of your feedback can be on the topic of detail about look and feel not the interaction flow.  Also people tend to be less inclined to criticize work if it looks done, for it is too late.

By the way, the topics about the paper mockup was interesting.  I wonder maybe our next class will be an art and craft with pen scissor glue developing paper mockup for the mini-hub.

RR Cooper Ch 7 : Know Thy Enemy

Knowing who is an enemy makes the difference between the survival and death during the battlefield.  At the design process, the developers are your enemy.  I regret I expressed it too strongly.  It makes it vulnerable to the counter-argument.  However I express it that way because it is true.

Why they are your enemy?  Because once they write a single line of the code, they will stick to it.  And worse, they can implement most of the user requirements in a few hours or days.  So they think interaction design takes too long, and are not sure if it is necessary at all.  While waiting, they got tempted to write a few line of codes to implement user requirement.  So how about beginning implementation while waiting?  Oops it started working.  And I finished 90% of function already.  So how about bringing a designer and ask him/her to make the existing product look better and declare done!  Nobody likes the delaying project.  And every management will love the guy who delivers project is done before schedule.   Oh, I forgot the guys interviewing people and sketching rough drawings on the wall.  They bring something to the table finally.  But who cares?  The project seems to be done, and no one in the management will like the news the existing product should be made again to reflect the change interaction designer brings.

That’s why those innocent looking, nice and kind developers are enemy in a nutshell.  For this Cooper warns strongly do not let the coding begin before the interaction design process.  Also that is why I kept interrupting Dr.V about the idea of two team (developer and interaction designer) working simultaneously.

So you know the identity of the enemy.  It’s time to learn their characteristics so that we can defend ourselves.  In this purpose, I recommend reading the Joel on software blog.  He is very smart like every other developers.  But what makes him really stands is that he has the rare talent of explaining simply with the humor using written language.

Reading Reflection Cooper Ch 7: About a tool

There is a Korean saying that only the unskilled worker blames the tool. It means that regardless of tools you may use, you can still achieve a lot. However I always wanted best tool I can afford. Because I am not a master, I want a tool to be effective so that they can help me achieve better. I will introduce three tools here.

The first toolset is Lamy Fountain pen with Alt-Goldgrün (Old Golden Green) from Rohrer & Klingner. The best thing about them is I can carry always, and it’s a great pleasure to write. So it encourages me to write something down.

The second toolset is the whiteboard. It is great tool for the collaboration or even alone. It is easy to use, easy to erase and modify. Nothing is better than the mindmap on the whiteboard to organize project status. I don’t know why but if I use projector and notebook to simulate similar environment, it is not good. Or the digital ones with touch technology was not satisfactory.  The drawback of the whiteboard is that I have to erase them. There is only one screen so to do another thing I have to erase. And it is hard to share with others using emails, and hard to store the progress. Cooper propose a digital camera to solve that. However I know a better tool, which is the digital whiteboard. Not the blackboard with the touchscreen. There is a kind of the whiteboard with scanner to allow scanning and printing. Some model has a USB slot to copy the scanned image. This is an example.

Finally the last toolset I would like to introduce is Balsamiq. It creates quiet unique low fidelity, but professional and aesthetic (at least in my eyes) mockup model. The advantage of using low fidelity model is well explained in the book. In my opinion, it facilities more open discussion about the possibility of the interaction framework than the realistic mockup. It takes almost same amount of effort to create realistic mockup using omnigraffle and low-fidelity mockup using balsamiq. However if we see the realistic mockup, we assume the design is final stage, and does not try to view in the out of the box. And more critical issues are when the stakeholders who does not hold the same knowledge about the design process tends to think about it is almost done, and set a unrealistic demand on the work schedule. And they tend to criticize the minor detail about color or shades, which the designer actually didn’t focused at all. So there is need of low fidelity model. And actually there is a promotion in the Balsamiq, which gives free offer for the classroom use. So if Dr.V is reading this, please visit the following link to see the condition.

Two story about Assumption

About Assumption.
Today I will talk about two short story about assumption.

Todays CGT class was about the user interview. With a limited budget and schedule, which is too typical in IxD, we had to finish the user interview in 3 hour with no budget. Dr. Vorvoreanu wanted us to find the ways to improve CGT website to increase the minority-ethnographic portion of the students.

So we worked in the rush and vigor, which is also too typical in the Dr.Vorvoreanu’s class. We made a turn to be an interviewer and an interviewee. When my turn has arrived, I happened to interview Stewart. We had already a few fruitless interviews with the local american males, who had never used CGT website as an aid for the decision for the admission for the Purdue CGT. Anyway they have finished their undergraduate here, and know everything about the department, so why do he care about the contents of the website so that more minority international students will be interested? So I declared it’s the end of the interview after first question. I was thinking that I will save time in this fruitless interview and will have more time in more potentially fruitful interview with say ,a Chinese woman. But I was wrong. First, I was the last. There will be no more interview. So I continued anyway. And I found that he knew and articulated about the CGT so well that he provided very valuable insight about how to improve the site by providing more appealing research sample of professor to the potential student. That was my first assumption. Though he was not our target user group, it doesn’t mean I don’t have to interview him.

The second assumption is more subtle. During the meeting with the Stewart, he mentioned that we should consider intellectual diversity more than the ethnographic diversity. I liked his point, and it made sense to question the fundamental assumption. Why do we have to increase the minority ethnographic share at first? What does the stake holder get from it? So I questioned Dr.Vorvoreanu about it, for she was the only stake holder in hand. It turned out the major push behind this was government, who wanted diversity. And you know what? What was important was the diversity between Americans like African or hispanic American. So all of our effort about how we can increase the interest of the Chinese or Korean or Indian girls to the CGT, found to be useless or out of the point, for we had an assumption about the “ethnographic diversity”. If we had the stake holder interview before this, we would prevent from this. That’s where the established process shines. And why Dr.Vorvoreanu could not explain this at first more specifically? In my opinion, it can be politically incorrect to say something like that in public, like the government pushes it and the ratio of african American graduate student is too low. So that’s why stake interview should be held in small setting where one can express their opinion more freely.

So that was the two assumption which was wrong today. We rely heavily on assumption in our daily life. However it is really hard to see it is assumption when the assumption is really nice and applies to almost all situation. It is like it is hard for us to perceive that there is an air, before there is no air.

As a human being, we all have assumptions. It means it is inevitable that we, interaction designer, will have some when we are doing user research. Though it is inevitable, it is important that we perceive that what we have is an assumption and check and evaluate it with real world data.  And now I remember from the readings they mentioned something like that.  So from the reading and real experience it becomes more clear now.

As a young man I always make mistakes. And what I like about University is that it is relatively safe place to make mistake. We can learn from the failure as much as we can from the success. But what is important is not repeating the same mistakes. Being a person with a humble memory myself, I hope this blogging and reflection helps me remember.

Reading Note 3-2 UX Book Chapter 13

UX Book Chapter 13

Interestingly the UX book chapter 13 was about rapid evaluation methods. As I pointed from last blog, there can be an difference of the opinion even between the experts. (Maybe I am not titled for an expert yet.) I will give another example. I am currently in the class of the “Human Factors in Engineering” by Prof. Robert Proctor. Last week’s homework was evaluating the Europa, the European Union’s website with the heuristics by Jakob Nielsen. Among the factors, there was an “Aesthetic and minimalist design”. I thought the website was minimalist in approach. Though there were many contents, (EU surely has lots of things to say.) they were well organized and there was no flashing banner or eye-candy to distract. While submitting I asked to TA about her opinion. And she said, there are too many contents so they violate the principles. (She is a president of student chapter in Purdue University of The Human Factors and Ergonomics Society (HFES). And this chapter received the HFES Gold or Silver award for Outstanding Student chapter since 2008 to 2011, which is 4 times in a row! Maybe I should submit to the authority.) As I said before, I don’t think I am right and she’s wrong. It just shows the limit of the tools we use. We were using same heuristic standard to different conclusion. And my view is supported by an academic research by Herzum & Jacobsen (2003), “which essentially states that the variation among results from different evaluators using the same method on the same target application can be so large as to call into question the effectiveness of the methods. One good side effect of this limitation is our job will never be automated by computer algorithms. And for those native Americans, their interaction design job will not be outsourced to the 3rd world cheap labor, for there is cultural differences.

And here is another story about Nielsen’s another heuristics that I couldn’t agree more. When I was at the university healthcare center for the immunization test, there was an computer error at the nurse’s computer. And guess what was an error message like. I am sorry that I didn’t take picture. There was an error code starting with 0x00 and a few sentences that error has happened, with big red stop sign. (Everybody knows that error has happened.) But the error message didn’t say any helpful message for user for possible solution or the nature of a cause. (I checked the error screen myself carefully for help.)


It was very obvious violation of the discipline which states that “Error messages should be expressed in plain language(no codes), indicate the problem precisely, and suggest a solution constructively.” And the poor old nurse, who I think must be a very intelligent lady, could not solve the problem until I leave after talking to a practitioner. She might felt frustration, which is not good. Wait lady, we, interaction designers, are coming to rescue!

And in the book, there is pros and cons of rapid evaluation methods. There is one more advantage of rapid evaluation with a few UX practitioners or experts over the full usability evaluation with many real users. For an interaction design to be successful, it is imperative that the UX feedback is delivered to the design or developer team as early as possible. That leads to the use of low fidelity of wireframe model of an interface during evaluation. However, real users tend to be annoyed or affected by the low quality of the visual than a trained experts do. So their focus or criticisms are more on the minute aesthetics, which can be easily fixed at later stage. Though the facilitator can explain to the normal users that this is just for the evaluation, their ability to augment the humble visual is limited when compared to experts.

I also agree with the view, that what is important is follow-ups. As mentioned it the 496 page of the text, “Now that you have found the usability problems, what is next?” John and Marks (1997) ,applying the lessons from the usability is a whole new issue requiring organizational view. Sometimes nobody reads usability reports (Worst case scenario). Many cases, it is too late anyway. (Developer team is already behind launch schedule.) And that’s where a dictator like Steve Jobs could excel compared to more democratic or hierarchical organization. If the CEO of the company was the participant of the rapid evaluation, who can ignore the reports?


This week I ended up a long review. I feel sorry to Dr.V about it, while also thanks her for reading. If feels nice, that there is somebody who actually reads long reflection like this.