10,000 feet

In February, Ted Chiang made a simple point about AI that struck a chord: “starting with a blurry copy of unoriginal work isn’t a good way to create original work.”

He made this argument about using AI to create a first draft of any written work, but the reality check felt bigger.

We’re living in an AI cyclone — where large language models are being positioned as capable of doing just about everything, and Ted’s point was simple: ahem…no?

It’s really appealing to look at a LLM and think: I’ll never have to go through the self-conscious process of plunking down an awkward first draft again!

Nonetheless, Ted (and later Ezra Klein) reminded us of something fundamental — that writing a first draft is neither a waste of time nor a speed bump that gets in the way.

Last week, after an intensive week of user research, I realized the same is true for qualitative analysis: starting with a blurry unoriginal analysis of users isn’t a good way to create original design.

7,000 feet

UXR platforms are dying to automate qualitative analysis

User research companies seem to be in a thirst race to automate analysis — in fact, Dovetail, Remesh , Insight7, TLDV have already launched an AI analysis feature.

The functionality is rudimentary today, but their end goal seems clear. Tired of wasting all those hours of your life reviewing what all those complainy respondents said? Simply press the analysis button and BLAM-O, neatly organized thoughts ready for the deck. You’ll never have to get lost in the muck of qualitative data again! Stop overthinking, and start designing today!

Sarcasm-aside, I understand why these companies are pushing in this direction. The hype of AI is driving intense competitive pressure and a fear of being left behind.

However, it feels like they need a Ted/Ezra-inspired reality check because analyzing user research is not an irritating chore that gets in the way of the design process. It is an act of design — an intellectual space where inspiration flourishes and where we prime our brains for original thoughts.

5,000 feet

Analysis wires your brain for design

Through analysis, we rewire our brain to think like our users.

After a long day of interviews, I sit in the sunniest part of my house with my dog at my feet. I re-trace the stories that I heard that day, hanging on each respondent’s wording and their explanations. Slowly but surely, connections emerge and understanding builds.

The objective is not to forcibly identify a few broad themes to pop into a deck. Rather, it is to study each respondent’s thought process. For example,

• Why did the patient avoid talking about their biggest fear when they walked into the doctor’s office?

• Why did this respondent move $500 to savings at the end of the month instead of paying her credit card balance down?

• Why did the pharmacist use that piece of paper to record inventory levels?

Even if AI could provide us with a nuanced analysis, and even if we dedicated days to studying it, we would still not have woven the neurological fabric that comes from doing the analysis ourselves.

It would be like reviewing another UX researcher’s output. Even if they had produced great work, you don’t gain a intuitive understanding by studying someone else’s answers.

That’s why, beyond the mere output, the essence of analysis lies in the process itself. Without it, we’re left to start the designing brainlessly.

That is, of course, unless the job of design itself is also overtaken by AI (more on that later).

I used to feel nervous at the beginning of every project because I would wonder to myself: what if we can’t find a unique design opportunity?

After I had a few years under my belt, that fear washed away. I realized, regardless of the industry or size of the client company, there was always something really obvious that they had missed about their customer.

That’s because entire industries are built on clichéd understandings of the people who use their product, overly simplistic narratives that fit neatly into the business machine.

The responsibility of the designer is to uncover the often overlooked, yet significant, jagged little insights that others may have disregarded due because it was just too spiky to fit with their smooth understanding of customer needs.

Meanwhile, as stated by Dave Karpf, AI’s large language models are literally cliche generators.

When you press the AI analysis button, it takes your hard-earned unique qualitative data, sands off all the nuance or insights, by filtering it through the lens of mass understanding.

And basic insights always lead to basic design.

Ground

For the sake of design research, we need to question the idea that AI is always helpful

If the social media era has taught us anything, it’s that no technology is neutral. On top of that, we have realized how bad we are at proactively understanding second/third order consequences of the technology we create.

The surgeon general just noted that social media has made us “unknowing participants in a decades-long experiment.”

However, I fear we are about to be participants in a far more dramatic experiment.

• Social media has touched the most fundamental human elements— like our ability to trust one another, see ourselves in a positive light, and feel present as we go through life.

• AI will try to take over the most fundamental human elements

That’s why I feel that this is much bigger than a button.

Imagine a scenario where AI possesses the ability to flawlessly understand us and our needs. Where would that leave all of us?

Think about how much we value our ability to empathize with one another — be it shedding tears alongside a close friend during a heartbreak, selecting the ideal gift for a partner that reveals our deep understanding of them, or delivering a speech at a wedding that beautifully captures the essence of the person being celebrated.

If AI also had the mysterious and beautiful ability to do this, it wouldn’t feel special anymore. We would have lost one of the things we value most about ourselves, one the “crown jewels” of our species.

That’s why I believe we have to ruthlessly examine AI now — even if may seem like nothing more than a silly little button — because it’s possible that most beloved parts of ourselves are at risk.

As designers, we may need to become protectors of the mysterious, beautiful realm where we humanity is most vibrant.

I’ll be exploring this topic in detail in the upcoming weeks; subscribe to follow along.