Design Thinking for Data Scientists — Need-finding

DSXL4AIML
15 min readSep 27, 2020

Author(s): Tara Su

Abstract. This report includes the need-finding results using participant observation, user interviews, and post-event protocols. The need-finding focuses on the pain point of using the iPhone mailbox app to check email in all situations, especially when available time is limited (under 10 minutes time span). The clear need for functionalities that facilitate organizing and filtering was identified. Users consider “smart sorter” to be a great-to-have functionality for the specific purpose, conditioned on additional requirements, such as “smart sorter needs to be truly smart”. The below sessions give a high-level summary of the need-finding exercise and the results. Detailed transcripts are included in the appendix.

Need finding exercise 1: participant observation

According to the plan I laid out during assignment M1, I went through the exercise of email checking in 3 different environments and recorded my feedbacks real-time.

The 3 environments I tested are morning in the bed, during lunchtime, and in between meetings.

Morning in the bed

  • The context of the task

This is the time I can actually concentrate on checking the emails without much distraction, although my time is normally very limited (5–10 minutes).

  • What is my major purpose?

My major purpose of checking emails at this time is to prepare for the day.

  • What do I need to achieve the purpose?

I need to remember all the important tasks I need to accomplish both at work and in personal life.

  • What specific tasks I do? What are the subtasks?

My high-level task is to quickly look through the new emails until the point I reach an email that I already checked yesterday morning.

Subtasks include: I make notes in the notes app on my cell phone if there is something particularly important I need to deal with either today or in the near future. I forward the important personal information to my husband. I quickly reply work-related emails to schedule meetings if I want to deal with it in the morning. I search for important people/companies’ names and titles just to make sure I am not missing anything from them.

  • What’s my thoughts and complaint?

So many unimportant information. I am sure I am missing something important because of all the noises, but I just don’t have the time and energy to care. I really wish there is some smart sorting to help me out and automatically organize the emails. I also have to switch back and forward to make notes. I hope there is a better way to help me track important info.

  • What hacks did I use to help achieve the goal?

I use the notes app as the note-taking tool, so I have to switch back and forth. I use my husband as the “reminder”, which is very effective, but I also get yelled at a lot. I reply to colleagues saying “please schedule a meeting to discuss with me” instead of actually schedule a meeting myself. I can’t really do it when the email is coming from client or leadership, so I have to switch to note-making for those ones. This definitely is not the most efficient way, but the scheduling integration is just not that convenient for me to deal with it when I need to finish checking all emails in 5 minutes.

During lunchtime and in between meetings (simplified to shorten this report)

Under these environments and contexts, I normally just have 1–5 minutes to check my emails. I feel even more strongly annoyed by the emails from promotion and social media. I scan through emails very quickly and ignore everything that is not work-related nor crucial personal information. I keep sending important personal emails to my husband, who acts like my “personal assistant”. I think a smart sorting with the functionality of notetaking and scheduling would help me immensely.

Avoid the bias

I am not my user, so I am going to use other need-finding methods to further explore others’ pain points without presumption. I also let myself be more objectively observing myself, so I was actually able to identify additional needs such as “notes taking” and “schedule meeting” besides needing help to filter information (smart sorting).

Need finding execution 2: user interviews

Figure 1: workflow and final script of the user interview

My second need-finding exercise was a user interviews. I recruited users who use iPhone and mailboxes app from my colleagues and my husband’s company, refined the scripts with my UX design and research colleagues, and finally conducted the interview (Figure 1). I enlisted the same 5 users for both user interviews and the 3rd need finding “post-event protocols”.

The detailed transcripts are available in the appendix. The results are summarized in Table 1 in the Data Inventory session.

4 out of 5 users voluntarily mentioned organizing emails is difficult with the current interface. The other user thinks it is manageable but not as convenient. All users voluntarily mentioned that they have too many promotional emails. All users mentioned searching for important emails is a problem, partially due to not robust searching function, as well as the amount of “unimportant” emails. All users mentioned the pain point missing important emails due to the unimportant emails.

“Painful. Very much. All the time. (I was just) waiting for quotes from the house improvement contractor. Didn’t realize it was already there.”

2 users mentioned that they want a timeline reminder. 2 users mentioned cell phone signal is less than ideal, which impacts their user experience of email checking. 1 user mentioned want a “pin email to the top” function. 2 users complained that the interface is not user friendly.

When talking about smart sorter, all users mentioned that they need to be “truly smart”. 4 out of 5 users want to be able to customize the folders. 3 out of 5 mentioned that the folders shouldn’t be too granular. 3 users mentioned that their needs change dynamically, so the folder structure needs to change accordingly. 2 users mentioned they want to be able to customize the sorting rules themselves.

To sum up, there is an overwhelming need for functionality/interface to help organize emails. Users want help to distinguish the important emails from unimportant ones. At the same time, they also want to be able to customize the sorting. Most users also demand a low error rate.

Avoid the bias

To avoid bias, I checked with colleagues who specialize in user research to make sure that my questions are reasonable and not leading questions. I also let the users go on tangent lines when time allows, so they can provide more spontaneous feedbacks. I let the interviewees do most of the talking, so I can objectively collect data without confirmation bias. I mentioned smart sorter but intentionally avoided showing my own attitude and my effort to developing one.

Need finding execution 3: post-event protocols

Right after the user interviews described in need of finding 2, I asked the users to spend less than 2 minutes to check their emails with their iPhone and mailboxes app. Immediately after that, I asked them for feedback about the app usage and pain points. The benefit of arranging interviews this way is that the feeling of using the app is still fresh in their mind, but I wouldn’t introduce bias by interrupting the normal usage pattern like in the “think aloud” format. I didn’t recruit for new users for this activity partly because of the cost (difficulty to find users who fit the profile and are willing to spend the time). One benefit of using the same users is that they will pay more attention to the topics we just discussed, which makes their feedback more relevant. This, at the same time, is a potential bias. The user’s focus may be more narrowed down than freshly recruited candidates.

Based on the result, the benefit outweighs the risk of bias. The users indeed give feedback relevant to the smart sorter and organizing the email box, but at the same time, they also gave additional feedback on items that were not restricted in this area. The major feedback points include suboptimal function discoverability, user’s detailed normal workflow, unsubscribe function, audio function to read out email aloud, poor cell phone signal, preference of mobile platform over the web interface, auto-reply suggestions functionality, the preview of attachment functionality, auto-extract important information and links from email bodies, etc. Many of these feedbacks can be important functionality of a broad definition of a smart sorter, such as unsubscribe function, the preview of attachment functionality, auto-extract important information and links from email bodies.

Overall, the users all feel managing important and unimportant emails is a challenging task. They recognize the value of smart sorters. They also provided feedback that can help the future flesh out the broad definition of a smart sorter.

Data Inventory

Table 1 data inventory is built based on the user interview and post-event protocols. The results of participant observation are already described in detail in the need-finding 1, so were not included here to save space. Besides the required 7 categories, I also included project-specific categories, such as pain points and smart sorter expectations. All the users recruited are right within the target profiles, so the specific feedback of pain points and smart sorters are closely relevant to the project. The only bias I want to point out while putting this table together is that the users I recruited are skewed towards the high technology population and high-income population. This is largely due to the accessibility of the users but could be an important watch out in the future data gathering and conclusions.

Table 1: data inventory from need finding 2 (user interview) and 3 (post event protocols)

Defining Requirements

After the user interview, I want to change my topic of “smart sorter” to “smart manager” to broaden the scope and address user needs more closely. It should include functions that help users to 1) sort emails into customized folders 2) making notes while reading emails 3) adding calendar events easily 4) summarize email and attachment content 5) extract important links such as unsubscription, account activation, etc. 6) highlight good deals.

Matrics I would uses include but not limited to 1) user engagement matrix (checking frequency, length of usage per app opening) 2) direct user feedback/rating (usability, error rate, discoverability, etc.) 3) efficiency: a path analysis of steps and time users take to achieve the same goal 4) versatility (the number of different tasks users achieve within the app).

Since I am targeting medium and power users, I would emphasize user experience (direct feedback) and efficiency.

Continued need-finding

I would like to conduct another round of user interviews, this time focusing more on usability. I plan to brainstorm more about potential design to address users’ needs first. Then I would like to use some low fidelity prototypes (sketched) to inspire more thoughts from the interviewees.

At the same time, I am concerned about the skewness of my users towards high tech and high-income population. I would like to use a survey to get a better distribution within my targeted user profiles.

My new questions are mostly focusing on the discovery of functional needs from the first round 3 need-finding exercises. I would like to investigate how broadly the needs are among more users. I will also focus more on relatively quantitative questions, such as “how many folders you think will be the right amount?”, “how many good deals you want to be recommended to you every day?”, etc.

Appendix

Transcripts

User 1

Profile

Age: 25

Education: Master degree

Income level: 75–100k

Occupation: Data Scientist

App using expertise: Intermedium level

Motivation: Keep contact with important but not very close people

Interview

  1. How often you use cell phone to check email?
  2. every other day
  3. Under what circumstance?
  4. When outside urgent
  5. What pain point do you have when using the mailbox app?
  6. Hard to open attachment. Switch to the another app to open it. Like to open it in the app
  7. Don’t know how to use some function such as add attachment
  8. Internet is slow. Didn’t know the photo is stuck in the draft. Like it to be cancelled at the moment. Then will switch to PC to do it.
  9. Interface is not as convenient as on PC.
  10. Screen is small. Junk trash are not very visible. Need to go back and forth.
  11. Hard to find the email if it is long time ago. Searching is not easy
  12. not sorting very often so hard to find the email
  13. How many emails you receive everyday?
  14. 4–6 emails, useful email once for 2–3 cdays. Use phone call and WeChat to contact with friend. Important info to email. Have promotional emails. Like Amazon’s promotion. Don’t look much.
  15. How you feel about mixing important emails and unimportant ones?
  16. Because don’t look at email much, so junk accumulate.
  17. What do you think about smart sorting?
  18. Smart sorting is going to be useful.
  19. What expectations you have for a smart sorting?
  20. All ad together. Look at it 1/2weeks
  21. Bills together
  22. School and work
  23. Conversation
  24. Job posting and interview related
  25. different time care about different things. Dynamically change your focus
  26. Deadline automatic identification
  27. Do you have any specific requirement related to smart sorting accuracy?
  28. Don’t Don’t put important email to advertisement.
  29. Not too many categories.
  30. Customize the sorting. And smart help. Like iPhone photo auto sorting.

Post event protocols

  1. Don’t know why not showing every emails. Emails at the Bottom doesn’t show.
  2. Just discovered that there is a “search”
  3. Too many discover card email. All over the place. Can sort into a folder based on name
  4. Need to go back and forth to select sent and inbox.
  5. Don’t know what the filter function is about. Never discovered it.

User 2

Profile

Age: 37

Education: PhD degree

Income level: 175–200k

Occupation: Quantitative Analyst

App using expertise: Intermedium level

Motivation: Make sure I am on top of everything

Interview

  1. How often you use cell phone to check email?
  2. Everyday more than 10 times
  3. Under what circumstance?
  4. In the middle of work, at lunch, morning before work, walking my dog. Any time except when driving
  5. What pain point do you have when using the mailbox app?
  6. Search can not find the desired result
  7. useful information goes to spam.
  8. How many emails you receive everyday?
  9. 30 emails every day.
  10. How you feel about mixing important emails and unimportant ones?
  11. No don’t feel important emails berried in unimportant ones.
  12. What do you think about smart sorting?
  13. Smart sorting must be smart enough to be useful. Need to be accurate. Hate when there is mistake
  14. What expectations you have for a smart sorting?
  15. Don’t mind unimportant things to be in important things. But don’t sort my important things into unimportant things
  16. No other thoughts.
  17. Hate web version
  18. want to be able to customize category
  19. Category shouldn’t be too granular. Don’t want to manage that many folders
  20. low error rate. Sort into the folder only when there is a high confidence

Post event protocols

  1. Open the inbox, look at every unread email from top to down
  2. Everything that is ads, delete
  3. Everything that want to read more will keep. Until reach last time’s reading point.
  4. Then go back to read all the details of the emails want to read
  5. Also keep the emails that knows need to keep, but won’t read now.
  6. Repeat this 10 -20 times. Less than 5 minutes every time.
  7. Why I subscribed to so many junk. Maybe my wife did it. But yeah, sometimes there will be one that is useful. It’s actually worth it to get all the junks just for that few useful ones.
  8. What if I want to unsubscribe but was not successful, then just throw them all into spam.

User 3

Profile

Age: 40

Education: PhD degree

Income level: 200–225K

Occupation: Quantitative Analyst

App using expertise: Intermedium level

Motivation: Make sure I don’t miss important emails including good deal

Interview

  1. How often you use cell phone to check email?
  2. 3 times per day
  3. Under what circumstance?
  4. work break, and morning, evening
  5. What pain point do you have when using the mailbox app?
  6. Too many promotional emails, hard to manage.
  7. Don’t want to do anything about them. Just leave them their. But there is a risk to have useful emails berried in them.
  8. Don’t want to spend time delete things. Too time consuming
  9. How many emails you receive everyday?
  10. 50–100 emails every day
  11. How you feel about mixing important emails and unimportant ones?
  12. Painful. Very much. All the time. Waiting for quote from house improvement contractor. Didn’t realized it was already there.
  13. What do you think about smart sorting?
  14. Smart sorting useful
  15. Would help me manage.
  16. Investment house. Business need.
  17. What expectations you have for a smart sorting?
  18. Get rid of or organize the seemingly junk emails
  19. Automatically detect the type of the email. Decide which folder to go into.
  20. Recommend to me to read or not.
  21. Management result easy to navigate. Intuitive presentation
  22. Minimize the click number. These are all chore for me. The fastest way to help me get through them.
  23. Do you have any specific requirement related to smart sorting accuracy?
  24. As accurate as possible. But save me time is more important.

Post event protocols

  1. Didn’t have enough time to go through all the emails
  2. Hated the amount of promotional emails, but liked the content. Some are very appealing. Awesome deals. Got distracted…
  3. A smart management is very much desired.
  4. ATT signal is not very good. waited too long to get it refreshed.
  5. Hope it has an audio function can read the emails for me.

User 4

Profile

Age: 37

Education: Bachelor degree

Income level: 150k — 175k

Occupation: Front end developer

App using expertise: Intermedium level

Motivation: Get important informations

Interview

  1. How often you use cell phone to check email?
  2. Daily
  3. Under what circumstance?
  4. Every where
  5. What pain point do you have when using the mailbox app?
  6. You check for the company email. Best way is the automatic notification.
  7. sound notification. I don’t like the sound
  8. Doing what it suppose to do.
  9. You can not do bulk mail
  10. Some time accidentally move to trash.
  11. How many emails you receive everyday?
  12. 8–10 emails.
  13. How you feel about mixing important emails and unimportant ones?
  14. Recruiter
  15. Subscriptions
  16. Marketing
  17. Don’t go through emails one by one.
  18. Lots of unwanted emails
  19. What do you think about smart sorting?
  20. That would be a good solution
  21. A new email account
  22. Create folders, automatically send to those folders.
  23. What expectations you have for a smart sorting?
  24. Smarter mechanism to sort, don’t want important emails to go to spam.

Post event protocols

  1. Nobody want to go to computer to check it. Easy to use
  2. a lot of unwanted emails. How they got my emails?!
  3. iPhone app is nice.

User 5

Profile

Age: 29

Education: Master degree

Income level: 125- 150k

Occupation: Data Analyst

App using expertise: Power user

Motivation: Keep track of all social activities

Interview

  1. How often you use cell phone to check email?
  2. 5–10 times / day
  3. Under what circumstance?
  4. At work at home first thing in the morning last thing at night.
  5. What pain point do you have when using the mailbox app?
  6. No push notification. Disabled. Frustrated initial, but when don’t have, you find it good.
  7. UI has been the same for the past years.
  8. Functionality not good
  9. A bug: when click in box, search, click on the first result, go back, it doesn’t go back to the search page, go back to general page. Randomly happening
  10. Multi mail box set up. Big mess.
  11. Managing not very intuitive.
  12. Idea of having multiple is good.
  13. 10 mail boxes, how to be very intuitive to mark some email box that is inavtive. Only push notify only some mail box. Priority
  14. Prioritize different mail box.
  15. you create account to separate your life. Now it’s all mixed together.
  16. Only go to job looking account when looking for job.
  17. UI not good.
  18. Not appealing
  19. Can’t create folder
  20. Can’t organize mailbox
  21. Have to go to the actual web mail box.
  22. It is light weight not very comprehensive
  23. Heavy user version, advance functionality
  24. How many emails you receive everyday?
  25. 1000/day 5–8 mailbox
  26. How you feel about mixing important emails and unimportant ones?
  27. Email types
  28. News letters
  29. Recruiters
  30. Promotion, marketing
  31. Personal
  32. Work
  33. Organizations
  34. School
  35. Select and delete, quick look to see whether you missing anything important
  36. Filters
  37. Smart sorter
  38. Important emails berried, miss important email because of these.
  39. Have to constantly go unsubscribe unwanted subscriptions. Small letter to unsubscribe. Select a couple of domains and automatically go to there web and unsubscribe. That would be so cool.
  40. Distinct sender. See some statistics. See which domain is putting all the junks in my emails.
  41. What do you think about smart sorting?
  42. (Skip. He mentioned smart sorting spontaneously)
  43. What expectations you have for a smart sorting?
  44. Creating rules from the app. No mailbox have it now. Custom folders. Want to customize rules from the app
  45. Folder for a specific mail box
  46. Only open iCloud email on phone
  47. Common function across emails.
  48. (Voluntarily gave a lot of other thoughts)
  49. Sudo email box, combine and show everything on that.
  50. Can not search all mail box at once. No advance search. No to someone, from someone. Other than that search is good. Type a name shows the name card.

Post event protocols

  1. As a heavy user, never-ending mail. Pin email at the top would be great.
  2. Reply auto suggest would be good.
  3. if have time in the email, would like a link to create reminder
  4. preview/summary of the attachment. Inbox app
  5. Linking other apps, reminders, show in the mailbox. Google keep notes integration.
  6. Shopping: shop now directly takes you to the app.
  7. Check score:
  8. Hyper link handy. Don’t even need to open the email.
  9. Share file easily. Gmail can do google drive easily.
  10. Change design. also give you the options.
  11. Smart folder
  12. Flight info extraction from the long email. Pin and set reminder for certain time. Already have google flight.
  13. Concert have similar functionality.

--

--