Gig-driver Research Tool
Understanding Gig Driving & Matching Algorithm
During the summer and fall of 2019, I worked in the Machine Learning Center in CMU, conducting research on algorithmic fairness and transparency of matching algorithm in gig driving.
Visual Designer / User Researcher
1. Recruited gig drivers for participation and conducting interviews.
2. Created the flow and the UI of the website the team used for interviews.
3. Illustrated and animated diagrams to aid the participants’ understanding of our goal.
4. Analyzed the findings from interviews and making conclusions.
Our team created a website that makes a machine learning model for each participant’s practices and opinions, based on their response to varying scenarios. It asks participants what information the matching algorithm should take into account when matching drivers to requests. Then it puts the participants through a series of scenarios that incorporates the types of information they chose. At the end, we tested the machine learning model that was trained using the choices the participants made throughout the scenarios by ranking 5 scenarios in order that the participant would prefer.
Online Forums /
My team wanted to learn about gig-drivers’ misunderstandings of the algorithm and the disparity between the reality and perception of the algorithm. We started our research by looking at online forums of drivers and conducting in-depth interviews with a couple drivers. Later on, we also had in-person interviews for more in depth information.ㅁ
After the research and the interviews, we were able to conclude that gig drivers have a lot of assumptions and confusions around algorithm and how it works. No one really knew what and how the information affected the types of requests they received, but only assumed closest person gets the request.
Problems / Questions
The research led the team to two main problems and two questions we wanted to answer. How can we fill the gap between the drivers’ understanding of algorithms and reality? What are the driver’s expectations for the matching algorithm? With these in mind, we were trying think of ways to utilize machine learning to confront them.
Solution — New Research Tool
To address the problems and questions we decided to create an activity where drivers will be choosing types of information algorithm should consider, in what order it should prioritize the data, and at the evaluate the algorithm that was trained based upon the data they gave. Below is a brief overview of an exercise we created to answer each of the objectives.
We went through two phases of prototyping for usability testing. First, testing it ourselves by role playing, then trying it with two participant drivers. This helped to build the overall structure of the interview to be more stable and actionable.
- Participants tended to confuse the idea of “algorithm” with the “app” and “company” they are using and driving for.
- Participants found it boring to go through a number of scenarios, and it was hard keep their attention throughout the pairwise comparison.
- Participants became “professionals” through the pairwise process. The features they based their decisions upon became narrower.
- Participants were confused on separating the “personal” and “social” features.
Motion Graphics /
I created visual assets to help explain different aspects of algorithm and this research tool we have developed to the gig-drivers.
At last, I added the visual assets to the pages participants will be going through.