Project Outreach
A multi-phasic study how we might improve Facebook Reality Lab’s internal research process.
**This is an ongoing NDA study. Certain details have been omitted or intentionally kept vague.**
Summary
PROBLEM
Research org’s within tech corporations like Facebook must sometimes involve internal (employee) participants to conduct quick pilot studies or studies using NDA concepts or prototypes. Recruiting and studying with employees presents unique challenges for research org’s due to lower engagement stemming from differences in incentive structures and other internal policies.
METHODS
Phase 1:
Interview
Brief Usability Test
Thematic Analysis
Phase 2 (ongoing):
Survey
Quantitative analysis (tbd)
PRELIMINARY OUTCOME
Insights from Phase 1 data — including user needs and pain points — were presented to stakeholders and made immediate impact on how this problem space is viewed by Facebook Reality Labs. Larger scale research (Phase 2) is being done to substantiate findings and provide actionable insights for stakeholder teams.
RESEARCHERS
This project was scoped, designed, and is being conducted by a fellow Research Assistant (RA) and I. Contributions to this project have been evenly split between the two of us.
MY RESPONSIBILITIES
- Scoped and co-designed study
- Recruited participants
- Conducted 4 interviews
- Took comprehensive interview notes
- Analyzed data via quantitative analysis
- Conducted activity
- Ran 4 brief usability tests
- Co-created final deliverable
- Co-presented to stakeholders
Background
In the beginning of February 2020, I finished recruiting and co-interviewing over a dozen Facebook internal participants for a study I was helping run. Recruiting for the study was challenging because of the stratified recruiting method (personas) required for the studies as well as the lower response and engagement from internal participants, largely due to policy surrounding incentives.
As COVID-19 was escalating and labs were at imminent risk of closing, I anticipated an increased reliance and demand for internal participants. Identifying a research opportunity, I teamed up with a fellow Research Assistant (RA) I decided to approach this as a user problem. Insofar as these lower engagement rates were a matter of user experience, how might we improve the internal recruiting research processes to provide a richer, more engaging experience for our FRL employees? I was excited for the opportunity to create meaningful and immediate impact in FRL, thereby empowering fellow RA’s in our org.
Methods
PARTICIPANT RECRUITMENT
To ensure we had a diverse sample of participants, we stratified our recruiting efforts based on 5 different job types. Recruitment was then done via proprietary Facebook platforms and systems. A total of 8 participants were recruited this way.
INTERVIEWS
Prior to interviews, a semi-structured discussion guide – which included a simple usability test module – was created and piloted with an independent FRL employee. A total of 8 interviews were conducted remotely with FRL employees. All interviews were recorded for later analysis. Interviewing and note-taking duties were evenly split between my project colleague and I. Each interview was followed by a debrief between the two of us, wherein we reviewed our notes, discussed our impressions about the interview, and talked through what worked and what didn’t.
THEMATIC ANALYSIS
Upon completion of all interviews, we created analyzed qualitative video and written data to identify common themes, important insights, and key quotes provided by each participant. These were subsequently outlined in a high-level summary of all 8 interviews.
Phase 1 Findings
Although I’m unable to disclose specific results of Phase 1, our preliminary findings were presented to stakeholders and have begun to make an immediate impact on how our internal teams think about the research process. As we progress through all phases of the study, our data will contribute significantly to the revamp of our internal study recruitment and research processes.
Current Status
This study is currently in Phase 2, which consists of a large scale, mixed-methods survey to targeting our greater research org. We have thus far employed qualitative and quantitative methods such as interviews, surveys, usability tests, card sorting, affinity mapping, thematic analysis, and concept testing. Based on our larger scale findings, we will consider deploying other methods as appropriate, such as journey mapping or a formal usability study.
