!!! (Brief version template for review) !!!

Project Overview

This project set out to assess how emerging technology can be used in the future of work. The ultimate goal of the project was to identify a user problem in current work practices and use emerging technology to create a conceptual solution.

The project resulted in the conceptualization of ContextClue, an AI-based app to help manage electronic communication notifications for remote office employees. Click the links to watch a video presentation of this case study, or read the full report.

Project Scope

This was an individual project completed over the course of 5 weeks. The project followed a Divergent/Convergent thinking iterative process. The process is divided into four development phases: Discover, Define, Ideate, Test & Deliver

 Phase 1: Discover

Secondary Research

I began the discover phase focusing on communication in virtual offices as the problem space. In order to understand the problem space, I combed through online research on communication practices in virtual offices.

Key Insights

• Working remotely is challenging because coworkers are out of sight and out of mind

• Lack of visibility leads to uncertainty and mistrust about employees’ productivity

• Remote employees feel they have to consciously put more effort into appearing productive and responsive or “always on“

• This pressure to be “always on” increases stress for employees

Qualitative Interviews

I conducted four interviews with remote office employees to get deeper insight into the refocused problem space.

Participants all had prior experience working in a physical office. I chose to conduct qualitative interviews because it allowed me to probe remote office employees on their current experiences with feeling “always on”, and how they signal and read coworkers’ availability.

Phase 2: Define

Interview Insights

To analyze the interviews, I descriptively coded the interview transcripts for common trends. The most salient findings were:

• All participants daily working hours varied by day

• Most participants use the same physical spaces/devices for work and personal activities

• All participants rely on availability statuses built-into communication platforms

  • No participant set availability manually

  • Most participants did not turn off notifications after working hours

  • All participant said they felt satisfied with current systems

• All participants’ choice of communication platforms is pre-determined by their organization and inflexible

• All participants admitted to screening notifications throughout the day

  • Motivations for screening notifications included: curiosity, self-control, anxiety

  • Most participants filtered and replied to notifications received after hours based on certain screening criteria:

    • Work Roles

    • Notification urgency

    • Time required to answer

    • Current availability

• All participants felt increased pressure to be “always on” since working from home that added stress to their life

Affinity Map

After I identified common user behavior trends from the interview insights, I created an affinity diagram to connect trends into larger themes and get a comprehensive look at the use case space.

I highlighted two potential pain points for users that could benefit from a technology solution:

1. Employees had difficulty separating work and home life

2. Employees feel an increased pressure to be “always on”

“How Might We” Statement

After I zeroed in on the problem, I reframed the problem with a “How might we statement” brainstorming session. Options focused on leveraging current tech platforms and reducing “always on” anxiety.

How Might We…

1. Reduce the need/want to screen work notifications for remote office employees?

2. Improve notification screening tools to maintain a separation between work and personal time while not missing important information?

3. Redesign notification tools to include descriptive context?

User Personas

As the last step in the define phase, I created two user personas to ground any solutions in real use case requirements.

The personas varied by different levels of job urgency and flexibility, coworker socialization, living/working environments, office space set-up, and work/life balance.

Phase 3: Ideate

Proposed Solution Description

The goal in this phase was to create solutions for a device that translated context into notifications. I decided the solution should be a context-aware notification filtering app for remote employees.

The app, named ContextClue, would give users more information on the context of notification messages and help prioritize relevant notifications in order to reduce employees’ need to screen notifications throughout the day.

Building Off Current Tech Limitations

As the first steps to create a solution, I compared the notification filtering options for the most common platforms participants used: Slack and GChat.

I used these notification filters and the gaps they left unaddressed to inform my own solution.

Outlining “Context” Awareness

Outlining context-awareness would highlight the additional information my solution’s filtering options needed to incorporate. To do this, I researched existing context-aware AI frameworks. I then filled in the framework with screening criteria participants already used to manually filter their notifications messages. The context outline is listed below.

I also considered how the solution would collect the necessary information. Luckily, a lot could be pulled from the device, but I would need to use AI or collect other information from user input directly.

AI Model Training

I researched machine learning to determine how the solution’s AI model would be trained to collect information. I decided my solution would use two rounds of training.

I chose this process because it would allow for a modular approach that could easily be scaled up or down. Also, the process seemed relatively feasible conceptually.

 Lo-Fi Prototype

I wrote a user scenario focusing on the primary persona for the first lo-fi prototype. In the scenario, I describe an app that works on mobile and desktop, works with existing communication platforms, and allows users to select from a list what contextual information they would like their messages to be filtered with.

The scenario helped me decide on how the onboarding process should look (See Sketch 1), what contextual screening criteria filters should be included, and how users would know if the system is working (See Sketch 2).

Sketches

 Mid-Fi Prototype

I created a mid-fi prototype as a digital narrative video to assess the general concept of the app. The video briefly described the features of the app and follows the onboarding process.

Testing this conceptual prototype would help me learn if ContextClue was useful and understandable to users as well as if they had any reservations about using the app.

Phase 4: Test & Deliver

I used the mid-fi prototype for two rounds of user testing, each with two participants. In the testing sessions I watched participants as they watched the digital narrative video and asked them a short list of questions afterwards.

Testing Round 1  

Link to prototype: https://youtu.be/WLvdWWhqRYM

In the first round of user testing both participants thought the product would be useful. However, there was confusion about the list of screening criteria filters.

Round 1 Design Changes

Privacy

• Eliminate face tracking as a potential screening filter.

  • Face tracking was the main reservation for both users and I debated whether it was necessary to include. In further research, I found face tracking as a means to infer emotional state is not entirely reliable. Face-tracking could also potentially detect physical signs of depression on a user’s face and broadcast that information to coworkers.

• Add “Pause Services” prompt when users move locations 

  • I added an additional feature that asks users if they would like to turn the app’s location tracking off when changing locations. Giving users more control over data collection should help ease concerns over location tracking privacy.

Confusion on Screening Criteria Filters

• Emphasize screener criteria filters are preinstalled on the app

• Emphasize screener criteria filters are optional and selected by users

Testing Round 2

Link to prototype: https://youtu.be/W7Nh50P6btc

In the second round of user testing I found both participants found the app useful. I also found there was more confusion around the screening criteria filters.

Round 2 Design Changes

Screening Criteria Filter Confusion

• Make prototype screens larger, more visible and easily legible

  • Participants mentioned text on the prototype screens was difficult to read because it was too small. Future prototypes would need to be larger.

• Tier and organize the list of screening filters

  • I organized the list into three sections (See Below): User Details (information that need users have to fill in themselves), Basic, and Advanced.

• “Screening Filters” nomenclature

  • To make screening filtering function more obvious to users I renamed “screening criteria” to “screening filters”. I felt “filter” was a more easily recognized and understood term.

• Emphasize which are automatic and which require set up by users

System Integration

• Emphasize the app is set up with user profiles 

Final Prototype

Link to prototype video: https://youtu.be/h3U-2SPHFUk

ContextClue Notification Messages (Desktop and Mobile)

ContextClue “Active” Signal (Desktop)

ContextClue “Screening” Signal (Desktop)

Future Steps

Future Iterations

Moving forward, steps for future design iterations should test with a more varied sample of users. Also, future designs should address:

Cost 

Both participants in the second round of user testing were interested in knowing the cost of the app. The cost would be determined by the cost to manufacture the app. Research would need to be done on the cost of AI model training in order to create a feasible business strategy.

Test screening filter names for understanding

In both rounds of user testing there was confusion around screening filters. Further user testing processes like card sorting could help to make screening filter names more understandable to users. This could also help signify the functions of individual screening filters.  

Future Design Considerations

Future design considerations should take into consideration:

Privacy

In both rounds of user testing participants main reservation with the app was privacy. Being transparent about data collection practices and incorporating clear signals when certain information is being collected could help mitigate these concerns.

Screening Filters 

Throughout the user testing sessions, the screening filters were somewhat hard for users to understand. Future designs should consider building in concrete signifiers and incorporating feedback when screening filters are turned on/off. Both of these would help users know what screening filters did before selecting them and know when filters were activated. Adding direct and clear feedback could also help increase users’ perception of accuracy and trust with the app.

Accuracy

To ensure the AI model is accurate, future designers should first acknowledge that AI models are only useful and accurate if they grow alongside real user data. Future designs should continuously work to update the AI model based on user feedback and promote continuous model improvements.