A photograph of a design critique, with designs on a wall and two peopel looking at them.
One source of evaluation is other designers.
Chapter 8

How to be critical

by Amy J. Ko

So you have a design. How do you know if it’s any good? We’re going to talk about three ways to know in this book: 1) critique, 2) empiricism, and 3) analysis.

When you’re asked to give feedback on an idea, how can you give useful, constructive feedback? In the culture of design studios, designers give feedback via  critiques 3,4 3

Irandoust, H. (2006). The Logic of Critique. Argumentation.

4

Kowitz, B. (2014). The key to happy, productive designers: Teaching your team to critique. VentureBeat.

. As some have noted 8 8

Wolf, T. V., Rode, J. A., Sussman, J., & Kellogg, W. A. (2006). Dispelling "design" as the black art of CHI. ACM SIGCHI Conference on Human Factors in Computing (CHI).

, critiques are not just evaluation of designs, but collaborative deconstructions of what makes a design successful and what makes it fail.

In many ways, being critical is easier than being generative. Our society values criticism much more than it does creation, constantly engaging us in judging and analyzing rather than generating and creating things. It’s also easy to provide vague, high level critical feedback like “Yeah, it’s good” or “Not great, could be improved”. This type of critique  sounds  like feedback, but it’s not particularly constructive feedback, leading to alternatives or new insights.

Design critiques have a number of unique features that try to ensure that feedback is useful:

  • Critiques are  two-way . It is not just one person providing critical feedback, but rather the designer articulating the rationale for their decisions (why they made the choices that they did) and the critic responding to those judgements. The critic might also provide their own counter-judgements to understand the designer’s rationale further.
  • The critic in a critique must  engage deeply  in the substance of the problem a designer is solving, meaning the more expertise they have on a problem, the better. After all, the goal of a critique is to help someone else understand what you were trying to do and why, so they can provide their own perspective on what they would have done and why. This means that critique is “garbage in, garbage out”: if the person offering critique does not have expertise, their critiques may not be very meaningful.
  • Critiques are both  divergent and convergent . They can generate ideas, or point in new directions. But they primary focus on revealing why something doesn’t work.

Because critiques are inherently negative to some extent, some approaches to critiques can be abusive. In some studios, people might call designs even design ers  by nasty names. This is often done to attempt to free people to provide deeply honest feedback about design, train designers to have “thick skin,” impervious but open to negative feedback. But it can be highly discouraging to many novice designers, making them question their abilities.

One way to avoid this harm, while still sharing harsh feedback, is to follow a simple rule: if you’re going to say something sharply negative, say something genuinely positive first, and perhaps something genuinely positive after as well. Some people call this the “hamburger” rule, other people call it a “shit sandwich.” Whatever you want to call it, finding something positive to say about something you don’t like forces you to consider the possibility that there is something actually good about the idea, even though all you might notice is what isn’t working. It’s your responsibility to search for both and share both good and bad aspects of an idea. This strategy has the added bonus of making people  much  more likely to listen to your feedback, because they’ll be primed by positive feedback.

Let’s look at an example of a critique. Most aren’t filmed, so I can’t show you one. Instead, I made one up. Let’s pretend you showed me this user interface mockup and we engaged in a critique.

A wireframe of an emale inbox showing folders, email subjects, and an email body.“
A basic email wireframe, for the purpose of illustration.

Here’s one possible design critique that would follow:

  • Me : Please tell us about what we’re looking at. 
  • You : Sure. So I set out to design a cleaner alternative to the traditional Microsoft Outlook mail client. I was going for something that was simpler, had fewer options, and as more learnable that the current design. 
  • Me : What do you think works? 
  • You: I’m really happy with the subject, to, and from information. I think the subject is really nicely prominent, and that’s because that’s the content people really care about when they’re trying to see a message. I also think the proportions of each of the tiled windows is nicely balanced, giving equal attention to all of the elements. 
  • Me : I agree, the balance is nice. It gives the greatest weight to content, which is what email is really about. I’m struggling a bit with the the large subject line in the mail body, however. Do you see how the subject appears in both the the message list and the message body? There’s a redundancy there that I can’t really see being helpful. Also, from a task flow perspective, someone would read the subject in the message list, and  then  select the message, so they’d already know what the subject was. It seems like wasted space. I do think the prominence of the font itself is nice, however, and could be a good place for other prominent metadata. Do you have any thoughts on what else might go there? 
  • You : that’s a good question. I guess from a task perspective, the first thing someone really wants to see after reading the subject line is the message itself. Maybe that whole container of metadata isn’t really necessary, or doesn’t need to be as prominent. 
  • Me : I wouldn’t go that far. Some of that metadata like who else received the message, could be pretty important. Maybe try putting the names of the other recipients.

There are several things to notice about the exchange above. It’s respectful, with each person listening and accepting what the other person is saying. It’s collaborative, with each person contributing knowledge to the conversation. It’s grounded in design rationale and design judgement, focusing on  why  choices are made and why  different  choices were made, and how that might effect the success of the solution. 

Another form of critique that can be applied to design is  Socratic questioning . In this form of critique, the person giving the critique wants to deeply probe the designer’s way of thinking and dig beneath the surface of their design. Some types of questions to achieve this include:

  • Clarification   questions, which encourage the designer to clarify their thought process.
  • Questions that challenge the designer’s   assumptions  .
  • Questions that encourage the designer to consider   alternative perspectives  .
  • Questions that encourage the designer to spell out the   implications and consequences   of their design.

Now imagine a dozen other students observing this dialog. What would they get from it? They’d see, like you’re seeing on this web page, an example of how to share feedback, how to receive it, and deep domain knowledge about the nature of email as social media. This means that you have much to gain just by watching critique happen, in addition to participating in it yourself.

As we noted before, good critique isn’t just about method, but also expertise. One aspect of expertise that is critical is expertise in the  domain  a design is exploring. Recent evidence suggests, however, that peers in classrooms can get pretty close to more expert feedback when students get  feedback  on their feedback 5 5

Kulkarni, C., Wei, K. P., Le, H., Chia, D., Papadopoulos, K., Cheng, J., ... & Klemmer, S. R. (2015). Peer and self assessment in massive online classes. Design Thinking Research.

. There are limits to this though: while feedback on your critique can improve your critique skills, nothing can replace domain expertise, however, which is invaluable for understanding the structure and dynamics of a problem space. From a design justice perspective, this might mean arranging a critique session not with other designers, but with stakeholders, asking them to bring their lived experience and knowledge of their domain to critically analyzing your design. This can require some careful planning, as people who are experts in their lives and domains are often unwilling to be critical to someone who has designed something 2 2

Dell, N., Vaidyanathan, V., Medhi, I., Cutrell, E., & Thies, W. (2012). Yours is better! Participant response bias in HCI. ACM SIGCHI Conference on Human Factors in Computing (CHI).

. This phenomena, called participant response bias, means that it can be essential to have someone that is impartial, or at least perceived to be impartial, to receive the feedback.

There’s one critical aspect of critiques that we haven’t discussed yet, however. How does someone judge what makes a design “good”?In one sense, “good” is a domain-dependent idea. For example, what makes an email client “good” in our example above is shaped by the culture and use of email, and the organizations and communities in which it is used. Therefore, you can’t define good without understanding context of use.

On the other hand, there are some design principles that one might use to make more absolute judgements of “good” design. Let’s start with some common, but vague principles, that are not particularly useful:

  • Intuitive . Human beings are not born with much innate knowledge. What people mean when they use this word is that someone can  infer  from the information in a design what the purpose or intent of something is, based on all of the prior knowledge they’ve acquired in their life, including encounters with a long history of user interface conventions and domain concepts. That is not “intuitive,” but rather, closely mapped to someone’s knowledge.
  • User-friendly . This is another imprecise phrase. What does it mean to be “friendly” with a user? Nice? Supportive? Helpful? This phrase suggests a lot without meaning a lot, and does not facilitate precise design critique.

There are many design principles in broad use that are a bit more precise, even though they might not be universally good in all contexts:

  • Simple . This is a design aesthetic that prizes minimalism and learnability. These  can  be good qualities, reducing how much people have to learn to use an interface and how long it takes to learn. But simplicity isn’t always good. Should moderation tools in social media simple? There’s nothing inherently simple about regulating speech, so they might  need  to be complicated, to reflect the complexity of preventing hate speech.
  • Novel . In some design cultures (e.g., fashion design), the best design is the  new  design that pushes boundaries and explores undiscovered territories. Novelty is powerful in that it has the power to surprise and empower in new ways. It also has the power to convey status, because possession of new design suggests knowledge and awareness of the bleeding edge of human creativity, which can have status in some cultures. But novelty trades off against simplicity, because simplicity often requires familiarity and convention 6 6

    Norman, D. A. (1999). Affordance, conventions, and design. ACM interactions.

    .
  • Powerful . This aesthetic values the ability of designs to augment human ability. Take, for example, a graphing calculator. These are exceedingly complex little devices with thousands of functions that can support almost any kind of mathematics. It’s certainly not simple or novel, but it’s extremely powerful. But power isn’t always good. Sometimes power leads to complexity that poses barriers to use and adoption. Powerful designs can also amplify harm; for example, powerful saved searches on Twitter enable trolls to quickly find people to harass by keyword. Is that harm worth whatever other positive might come from that power, such as saved time?
  • Invisible . Some trends in design aesthetics value designs that “get out of the way”, trying to bring a person as close as possible to their activity, their information, and their goals. Designs that achieve invisibility don’t try to be the center of attention, but rather put the attention on the work that a person is doing with the design. Good example of designs that attempt to be invisible are the many intelligent assistants such as  Siri  and  Alexa , which try to provide “natural” interfaces that don’t need to be learned, personalized, or calibrated. All of this may come at the expense of power and control, however, as the mechanisms we often use for invisibility are automated.
  • Universal . The premise of universal design 7 7

    Story, M. F. (1998). Maximizing usability: the principles of universal design. Assistive Technology.

     as something that all of humanity should be able to access, prizing equality over other values. For example, designing a website that is screen readable so that people who are blind can read it often constrains the type of interactivity that can be used on a site. What’s better: power and novelty or universal access? Maybe there are some types of designs that are  so  powerful, they should only be used by certain people with certain knowledge and skills. Of course, universal designs are rarely universal; all design exclude somehow.
  • Just . The premise of design justice 1 1

    Costanza-Chock, S. (2020). Design justice: Community-led practices to build the worlds we need. MIT Press.

     is the purpose of design should not be to amplify inequities and injustices in the world, but to dismantle them. This might mean that a design that ultimately serves the enrich and empower the wealthy (e.g., Facebook Ads) might be deemed worse than a design that helps dismantle an unjust system (e.g., a social media network for small-business loan networking amongst Black owned businesses)

Of course, you can see by now that I take a  relativistic  view of design aesthetics. I think all design choices are made relative to a prioritized set of values. I think good design  process  makes these values explicit, consciously deciding which aesthetics supersede others, who to support, who to exclude.

References

  1. Costanza-Chock, S. (2020). Design justice: Community-led practices to build the worlds we need. MIT Press.

  2. Dell, N., Vaidyanathan, V., Medhi, I., Cutrell, E., & Thies, W. (2012). Yours is better! Participant response bias in HCI. ACM SIGCHI Conference on Human Factors in Computing (CHI).

  3. Irandoust, H. (2006). The Logic of Critique. Argumentation.

  4. Kowitz, B. (2014). The key to happy, productive designers: Teaching your team to critique. VentureBeat.

  5. Kulkarni, C., Wei, K. P., Le, H., Chia, D., Papadopoulos, K., Cheng, J., ... & Klemmer, S. R. (2015). Peer and self assessment in massive online classes. Design Thinking Research.

  6. Norman, D. A. (1999). Affordance, conventions, and design. ACM interactions.

  7. Story, M. F. (1998). Maximizing usability: the principles of universal design. Assistive Technology.

  8. Wolf, T. V., Rode, J. A., Sussman, J., & Kellogg, W. A. (2006). Dispelling "design" as the black art of CHI. ACM SIGCHI Conference on Human Factors in Computing (CHI).