Better Experimentation at DeepL: Maternity Cover Interim PM for Desktop & Mobile Web

No items found.

The Client: DeepL

DeepL is an AI-powered language translation service that uses neural network technology to translate text from one language to another. It translates whole sentences and paragraphs while maintaining the context and nuance of the original text.

DeepL is known for its high-quality translations, especially in languages with complex grammar and sentence structures (e.g., German). It currently supports translations between multiple languages, with plans to expand to more languages in the future.

In addition to its translation service, DeepL also offers a range of other language-related tools and services, including a dictionary, a web-based text editor, and a browser extension.

The Mission: Interim Product Manager for Core Product Web Team

We joined DeepL to serve as interim Product Managers in the Core Product Web team. We were part of a team of engineering managers, front-end developers, and several product designers. With the original Product Manager on maternity leave, our role was to fill in the Product Management gap during this time.

The Core Product Web team is a customer-facing/user-facing focused team with a strong focus on experimentation and fast learning. They are responsible for the Desktop & Mobile Web user experience and the largest share of DeepL’s traffic - 100s of millions of MAU (Monthly Active Users).

We had several opportunities and challenges to support on:

  • There are various types of customers using DeepL with different levels of engagement and Jobs To Be Done (JTBD). It had been challenging for the team to prioritize and make the capacity to address relevant user types equally each cycle and make progress.
  • Due to some data collection challenges, the team needed help setting a North Star Metric (NSM) and a way to easily map their goals into the broader business context.
  • Development team members needed help contextualizing the experiments they were running to stay motivated and see the impact of the experiments they were creating.
  • An existing ongoing refactoring project that occupied development team bandwidth.
  • A side mission to encourage better collaboration between the Product team and Machine Learning Research team.

Our Main Quest

Improved Experience for Engaged DeepL Users

For approximately two quarters (6 months), we focused on improving the experience of various DeepL features for engaged users and improving the ways of working within the team.

Our first step was understanding the various user types, their Jobs To Be Done (JTBD), and which contributed best to DeepL’s company objectives. We used this to align with stakeholders and the team to select a user type to focus on with the most impact.

Introduced a North Star Metric (NSM)

Additionally, we interviewed team members to understand better why a North Star Metric (NSM) was unavailable and what improvements we could make to the quarterly goal setting. We’ve used the Narratives, Commitments, and Tasks (NCT) framework for this.

Changed Discovery and Ideation Process to Incorporate the Wider Team

We implemented several changes to improve collaboration and ensure that relevant team members had a voice in the discovery and ideation processes at the relevant time.

Democratized Results and Documentation of A/B Test Experiments

One of our challenges was that some team members were missing the necessary context for experimentation and the results of the experiments they were creating. Making this knowledge accessible and encouraging people to read it helped us get buy-in faster and motivated the team.

Importantly, we have improved processes around A/B experimentation planning and defined case in which experimentation wasn’t necessary and could be substituted by other ways of collecting user insights (user research, usability testing).

Stronger Collaboration Between Product and (Machine Learning) Research Teams

In the context of our Interim Product Managers engagement at DeepL, a key objective of the role was to bridge the gap between Product and (Machine Learning) Research teams and foster better collaboration between the two.

The Product team was responsible for creating and launching new features and products. The (Machine Learning) Research team was focused on developing new algorithms and improving the core Machine-Learning (ML) models that power DeepL's language translation services.

Our first goal was to understand if there’s a disconnect, then what it looks like, and which are the underlying causes, including historical reasons. A relevant and harder-to-change one was the different tempos of development. ML teams tend to look at 3-6 month increments, while for web teams, it’s ~1 month. Others were easier (and under NDA). We started iterating on different ways to work closer together and kept the ones that worked.

These efforts improved collaboration between both teams, as they could work more efficiently and effectively, with a shared understanding of the overall product vision and goals.

Mission Achievements: Delivered Outcomes

KPI uplift from our A/B tests and other internal metrics are under NDA, so we’ll focus on sharing process-related contributions.

💡 Introduced a North Star Metric (NSM) and Narrative Commitments Tasks (NCT) for our team
💡 Democratized Results and Documentation of A/B Test Experiments
💡 Changed Discovery and Ideation Process to Include more stakeholder teams, including Machine Learning Research

In the Client's Own Words

Space Crew of this Mission

VP/Director/Head of Product
Product Management Consultant
Associate Management Consultant

For Clients: When to Hire Us

You can hire us as an Interim/Freelance Product Manager or Product Owner

It takes, on average, three to nine months to find the right Product Manager to hire as a full-time employee. In the meantime, someone needs to fill in the void: drive cross-functional initiatives, decide what is worth building, and help the development team deliver the best outcomes.
If you're looking for a great Product Manager / Product Owner to join your team ASAP, Product People is a good plug-and-play solution to bridge the gap.