Amazon experiment data analysis dashboard
Project Overview
I led the vision for optimizing data analysis dashboard
When I joined this project, the experiment data analysis dashboard — used by internal teams to monitor treatment performance metrics and make final launch decisions — had a major usability problem: it was essentially a wall of numbers, with all the statistics thrown together with no organization, making it very difficult for users to read and interpret the data. After joining, I conducted user interviews and stakeholder interviews, ran workshops to brainstorm solutions, developed a design vision, and helped shape the roadmap that led to launching a new dashboard. In the end, what was once an overwhelming wall of numbers became a clear, organized experience that let users quickly and confidently understand how a treatment was performing.
Design Solution
Make the data speak for itself, so users can decide with confidence
Experiment health
Once an experiment starts running, users no longer have to wait in the dark for data to arrive. Instead, they get a real-time view of whether it's actually running as set up.
Intuit data visulaization
Metric data for the variation being launched is displayed in a side panel for quick reference while writing. Users simply select the data they need, and a complete sentence is auto-generated and inserted without having to manually put one together.
Granular data to support further investigation
Users can drill into more granular data for each treatment to analyze perfromance.
The impact
A game changer for deciphering statistical results
The data analysis dashboard is one of the most critical features of the product, since it directly shapes whether teams can make the right launch decisions. After the new dashboard launched, clear data visualization and built-in interpretation helped teams easily understand their metrics — even without deep statistical expertise. Teams were no longer left confused or blocked, needing to track down an expert just to make sense of their own results.
My role
Solo UX designer
Duration
2023-2024
Tools
Figma
Team
Product, Eng, Data Scientist teams
Research and Exploration
I led the research and workshop to discuss direction
Through surveys and user interviews, I found that the current data analysis dashboard functioned like a black box, users couldn't easily extract or make sense of the information they needed from it. I brought these findings to stakeholders for discussion and brainstorming, and together we distilled them into the core pain points users were facing.
The North Star
Support the full lifecycle of data analysis
After conducting a workshop with stakeholders, we aligned on a North Star: Weblab should walk with users through the full lifecycle of analysis, connecting every step from launch to decision. That's not a journey users should be making alone, so we set out to bridge the gaps and support the full lifecycle, helping users move through each step with confidence. I mapped this into three phases of the data analysis journey, each becoming its own tab in the design so users have a dedicated space for what they're working on at each step.
The Experiment Health tab opens the black box of the silent first week, giving users a real-time view into whether their experiment is actually running as set up and flagging anything that looks off before it becomes a problem. The Results tab is where the call gets made, rebuilt around helping users actually reach a decision rather than just displaying data. The Explore tab brings deeper investigation into the platform itself, letting users run and save custom queries with visualizations, so they never have to leave Weblab to finish the analysis they started.
Dig deep to surface real problem
The real ask behind "just tell us what to launch"：helping users trust their own call
From past user surveys and conversations with users, one request kept surfacing: users wanted Weblab to give them a launch recommendation, to simply tell them which treatment to launch. Rather than take that request at face value, I dug into why it kept coming up. Most launch calls are made by PMs, who often lack the statistical expertise to fully interpret the data, and the dashboard offered no support to help them make sense of it. So when they said "just tell us what to launch," what they really meant was "I can't read this, so give me the answer." But a launch decision isn't purely a data decision, since teams bring context the platform doesn't have, like strategic priorities, engineering cost, and roadmap alignment, which aren't things Weblab could take off their hands. The real user need wasn't a recommendation, it was the ability to read the data confidently enough to trust their own call.
