ClickUp uses Eppo to save 12+ hours of analyst time per experiment across growth, product, and marketing use cases
About
ClickUp helps companies save time with an all-in-one productivity platform that brings teams, tasks, and tools together. As one of the fastest-growing SaaS companies in the world, ClickUp has helped more than 800,000 teams and millions of users lead a more productive life and save at least one day every week.
Experimentation has been a key component of ClickUp’s strategy as they’ve grown from $4 million ARR to more than $100 million ARR in just over two years.
The Problem
"Even minor tasks like adding new metrics to an old experiment was very time consuming. The analyst has to find the one-off Python notebook, piece together the historic context behind that code and experiment across multiple conversation threads, and then figure out a way to correctly add a metric and refresh the result. This workflow just doesn’t scale."
Sai Srigiriraju joined ClickUp to lead their growth product analytics team. Like every growth team, ClickUp had an endless list of questions about their customers: what type of questionnaire should we show during onboarding? Should we use a checklist or a video to drive activation? Once a user is activated, how can we get them to stick around and expand their usage?
Sai knew that running experiments would be crucial to answering these questions: “If you're a growth practitioner, you make decisions based on evidence and data. Experimentation is the best way to do that”.
Experimentation wasn’t new at ClickUp, but the current approach could not scale to support the experiment velocity required to answer all of the questions the growth team had. For each experiment, analysts would spend hours preparing Python notebooks, updating data, and manually sharing results with PMs.
The result was not just an inefficient use of analyst time but also low incentives to run experiments. Ultimately, low visibility into outcomes and the inability to self-serve results made growing an experimentation culture impossible.
Dedicated tooling is needed to scale
There were a few critical criteria for a successful solution. Firstly, it had to be easy to set up and run experiments. It was evident that a streamlined and standardized process was necessary, considering the plans to 4x the number of experiments run per quarter. Secondly, the results had to be reliable and robust. In contrast to other product analytics use cases where directionally correct results may be sufficient, in experimentation, decisions must be made based on single-digit percentage differences. There is no margin for data issues in this context.
Because all of Eppo's analysis runs on top of the source-of-truth data in ClickUp's Snowflake instance, Sai's team found it easy to trust the results provided by Eppo:
“Being warehouse native gives us incredible control over the data we’re sending Eppo and instills trust in the results we’re seeing. If we want to spot check or gain a better understanding of what we are seeing in Eppo, we can go to our warehouse console and query the data easily. This makes Eppo very accessible and not a black box.”
Finally, a solution had to make experimentation accessible and visible across the organization. Even PMs with no prior experience with experimentation needed to learn from past examples to understand how experimentation can improve decision making.
To solve all of the issues simultaneously, Sai brought in Eppo. Eppo’s data warehouse-native architecture and purpose-built user interface led to a clear shift in experimentation velocity and culture.
A single platform for all use cases
Eppo enables Sai’s team to assess the impact of experiments across the entire customer funnel, including onboarding, activation, retention, engagement, and monetization use cases. Here are some examples of the types of questions that Sai's team tests using Eppo:
Onboarding: what happens when we switch up the questions in our pre-onboarding questionnaire? What happens when we shorten the questionnaire? What happens when we change the designs in the onboarding flow? How does that affect not just onboarding completion, but activation and retention down the funnel?
Activation: How does using a checklist or an onboarding video move activation rates? If it does move activation, does it also move retention in a meaningful way?
Retention: How can we get activated users to drive further engagement? Should we make them engage with features that they have not already engaged with? Or do we not expose the new feature in their home screen?
Marketing: Does sending out a social proof email to users who've completed onboarding help move them to the next step of the funnel?
Pricing: What is the right pricing page? What are the right features that we need to show our users on our pricing page?
12+ hours saved per experiment
Eppo directly utilizes business entities, slice-and-dice dimensions and metrics in Snowflake, thereby eliminating the need for manual experiment analysis. As a result, the data team is saving well over 8 hours per experiment, and business stakeholders are saving 4-6 hours of back-and-forth with the data team. Product Managers can now fully self-serve experiment analysis and conduct in-depth investigations independently.
The value created from the saved hours is evident, but it does not stop there. With analysts no longer burdened with data requests, they can now devote their time to pursuing new opportunities to explore. This ultimately leads to more ideas being tested and more wins being celebrated: “Now my team can focus on working with PMs to understand the “why” behind results and form the next set of hypotheses to test.”
Unlocking a culture of experimentation
The biggest benefit of adopting Eppo for ClickUp has been the fostering of a culture of experimentation.
“The ongoing support we receive from the Eppo team in cultivating an experimentation culture has been instrumental in the success of our partnership. Apart from the benefits offered by the Eppo platform, we have greatly benefited from the availability of stats experts who are enthusiastic about assisting us.”
Experiment learnings are now posted in a public Slack channel to give broad visibility and grow other teams’ excitement about running their own experiments. Even Product Managers new to experimentation can be easily onboarded: “With experimentation, there is friction and fear of the unknown. Once a PM sees an experiment in Eppo, all of this fear disappears.”
Finally, this experimentation culture has started to grow even outside of growth. PMs across the org are using Eppo to test new feature releases, and the marketing team is starting to leverage Eppo to make decisions on email strategy.
Sai summarizes it best:
“In the end Eppo’s value comes down to efficiency. In today's environment, efficiency is of utmost importance. If I had three analytics resources, I would prefer to utilize them in discovering new opportunities instead of running experiments or querying data to assist PMs in understanding what is happening with the business metrics. I think this is the big win with Eppo - helping us scale and enhance operational efficiency.”
“Eppo is an incredible tool that not only saves us tons of analyst time and improves accuracy but also enables us to run all kinds of experiment analyses right on top of the data warehouse where we can enrich and join data to get insights that wouldn’t be possible in previous tools. It’s a whole new generation of A/B platform.”
