Designing Eppo Reports

A familiar scenario

The CPO gets an email with an experiment report. It has a big wall of text and numbers, and gets a reflex response, “this is going to take a while to understand — Snooze.” The cognitive load of unnecessary metrics, statistical inputs, and a lack of context put the email at the bottom of the pile. The CPO’s lack of attention eventually becomes a narrative of “we don’t have an experimentation culture here.”

At Eppo, we believe intentional and opinionated design unlocks experiment culture. Our latest launch is a great example.

Experiments are for all: make it happen with Reports

Eppo provides trustworthy and user-friendly metric report cards supported by a best-in-class statistical engine. However, customers faced difficulties sharing those narratives with a wider audience. They’d spend hours copying metric results, charts, and data from Eppo into Google Docs to generate a curated report for their experiments. And after all that time, the Google Doc would be confusing and untrustworthy, with data going stale and there being insufficient context of the experiment. All of this strengthened our commitment to providing users with tools to create reports within Eppo, and export them for easy sharing.

‍

‍

We focused on these principles when designing and building Eppo Reports:

Flexibility — Meeting Users Where They Are: Drawing inspiration from Notion, we have adopted a similar block framework as the foundation for Eppo Reports. Blocks are a familiar way to structure the data model into defined units and convey the available functionalities to users. Whether it's text, lists, charts, metric dashboards, and more, all are presented as blocks, giving Product Managers and Data Scientists the flexibility to craft their narratives. Furthermore, this approach establishes a foundation for future collaborative features, including embedding Eppo blocks into Notion pages or other tools, facilitating comments, and much more.

Source of Truth — Up to Date and Context-Driven: We designed Reports so that a low-context user can jump in and know exactly what is being tested. Every experiment comes with an executive summary card containing the hypothesis, key takeaways, and the decision of the experiment, with more block types coming. We began with core context items such as images of the variants being tested, metric results, and slice-and-dice explore charts. These experiment-specific blocks stay up to date with every data refresh, making a report useful not only after an experiment is complete but also for the duration of the experiment run time for weekly experiment meetings.

Compelling, Readable, and Beautiful: Our overarching goal was to reduce the cognitive load of understanding an experiment and make people feel excited to share their work. We explored different page structures, such as modular dashboards or infinite whiteboards, but ultimately chose a document experience with a slight twist. Documents and presentations are the best way to tell a linear story, so we combined those two elements resulting in cards and blocks. Cards are containers for your blocks that create sections for easy content separation when reading it like a document. When you’re ready to present your experiment to the executive team or at your weekly experiment meeting, each card becomes its own page of a slide. Your Eppo report is now a beautiful presentation without any extra steps.

Discoverable, Institutionalized Knowledge: A significant aspect of nurturing an experimentation culture involves transforming experiment insights into institutional knowledge. Curated reports will be centralized within Eppo's knowledge base, enhancing their discoverability across teams and providing an accessible entry point for executive stakeholders to see what has proven effective and what hasn't. This comprehensive approach ensures that the fruits of experimentation are preserved and built upon, fostering a culture of continuous improvement.

Eppo Reports simplifies the way growth teams communicate their successes and learnings. While ad-hoc dashboards can meet a specific need, they often leave executives and low context team members scratching their heads, missing the essential "why" behind an experiment's significance. Eppo Reports facilitates a shared experimentation journey that technical and non-technical users alike can celebrate and leverage.

‍