ACQUISITION & ACTIVATION

Gateway to Growth

The path to achieving a frictionless authentication experience through strategic A/B testing.

OVERVIEW

Unlocking ITVX Growth

Leading the Growth, E-commerce, and New Value (GENV) Product Design team, a key area of focus in advancing ITVX involved optimising user authentication to re-engage dormant users and increase monthly active users (MAUs) from 10.5 million in FY 2022 to 20 million by FY 2026. This case study highlights our strategic implementation of A/B testing to improve the sign-in process. My role required coordinating a multidisciplinary team, fostering adoption of new technologies and methods, breaking down objectives into achievable tasks, promoting data-driven design innovations, and maintaining stakeholder engagement throughout the project. These actions were geared towards minimising user friction and enhancing engagement and satisfaction with ITVX.

IMPACT OF AUTHENTICATION INITIATIVES IN 2023

5%

reduction in dormant users

less Customer & Viewer Service contacts

350,000

incremental MAUs unlocked

70%

About ITV

ITV, the UK's largest commercial broadcaster and streamer, launched ITVX in late 2022 as the freshest streaming service offering both advertiser-funded content for free and an ad-free subscription tier. As a modern extension of ITV's traditional broadcasting, ITVX aims to deliver a wide range of content across various platforms including web, mobile devices, connected TVs, and gaming consoles, thereby reaching a broader audience

Roles & Responsibilities

ROLE

Head of Product Design

Leading Growth, E-commerce, and New Value (GENV) initiatives, focusing primarily on:

Acquisition & Activation
Upsell & Retention
New Value Propositions
Partnerships and Distribution

TEAM COMPOSITION

The GENV Product Design team consisted of:

1 Lead Product Designer
3 Senior Product Designers
3 Product Designers
1 UX Writer
1 UX Researcher

ACQUISITION & ACTIVATION TEAM

Collaboration model:

Product Management
Design
Engineering
Data
Customer & Viewer Services
Cybersecurity

KEY RESPONSIBILITIES

A/B Testing Advocacy: Led the adoption of A/B testing amongst the design team to enhance decision-making and product improvements through actionable insights.

Training and Onboarding: Managed the integration of Amplitude, boosting the Product Design team's analytical and design capabilities by enhancing workflow integration.

Resource Allocation and Prioritisation: Addressed testing delays by reallocating resources and reprioritising tasks to sustain project momentum and maintain quality.

Stakeholder Engagement: Partnered closely with Customer and Viewer Services and Cybersecurity teams to align project outcomes with their requirements.

Feedback and Iteration: Directed regular design critiques and updates, promoting a culture of continuous improvement and collaborative refinement.

Initial registration modal when ITV recognised an account associated with an email entry.

Initial sign in modal when a user had entered incorrect details

Forgot password modal displayed when a user had entered an email address for recovery of a password that doesn't have an associated ITVX account.

PROBLEM STATEMENT & STRATEGY

Tackling Dormant Users

ITV launched ITVX in November 2022, replacing the older ITV Hub catch-up service. Subsequently, issues surrounding user authentication surfaced. This led to a surge in password reset submissions and a high volume of related customer service inquiries. The transition saw a significant number of dormant users, who had registered but remained inactive, attempt to engage with the platform during major events like the 2022 FIFA World Cup, often misunderstanding the process as a need to re-register rather than simply reactivating their existing accounts.

To address this issue, we implemented a “No Account Found” modal. This modal, a component from our Design System, required minimal development and helped guide users to register if they had no account.

Impact Assessment

We analysed the impact by comparing data from four weeks before and after the modal's introduction which resulted in:

A 132% increase in successful registrations.
Projected 35,000 additional monthly active users.
Contacts to Customer & Viewer Service halved.

The introduction of a “No Account Found” modal, although effective in the short term, did not address the underlying issues of user confusion and system inefficiencies. The solution was not sustainable without further strategic and systemic improvements. Lacking A/B testing and a control group made it challenging to solely attribute improvements to the modal. I advocated for implementing A/B testing across our organisation to enhance user satisfaction and reduce customer service contacts.

BUSINESS GOALS

• Provide a frictionless sign-in for dormant users.

• Reduce the number of dormant users.

• Increase conversion rate of registrations and sign-ins.

• Reduce event-driven contacts to Customer & Viewer Services.

CHALLENGES

• 72% of user base is dormant, only coming to ITVX every now and then for large events.

• Seamless transition of user accounts from ITV Hub to ITVX not well understood by returning users, creating UX bottlenecks.

• Absence of established A/B testing methodology.

STRATEGIC APPROACH

• Implement A/B testing for a more agile, responsive, and user-centred approach to refine sign-in processes and optimise user re-engagement.

• Use of phased approach, starting with browser-based experiments, ensuring learning is applied before broader implementation.

DESIGN PROCESS & DECISION-MAKING

Research and Insights

Our initial step involved conducting extensive research to pinpoint barriers within our registration and sign-in processes, involving user testing, customer service analysis, and reviewing competitor methods. Key findings revealed that:

48% of users attempting to register already had an account.
20% attempting to sign in don’t have an account or forgot their password.
25% mistakenly start password resets, thinking they have an account.
"Forgotten password" was among the top three customer service issues.
Many users expressed uncertainty about whether they had previously created an account, complicating their decision on whether to sign in or register.
11% of users ended up creating a new account as a workaround because they were unsure of their registration status or had forgotten their password after a period of inactivity.

These insights set the foundation for identifying critical improvements in our user interface to enhance user satisfaction and streamline access to our service.

Ideation and Prioritisation

To enhance team collaboration and strategic alignment, we conducted an ideation workshop with product managers, engineers, designers, and data analysts. The key outcome was the development of an Opportunity Solution Tree. This tool helped visually map potential pathways and prioritize high-impact solutions, optimizing resource allocation.

The Opportunity Solution Tree clarified decision-making rationales, essential for maintaining strategic alignment and justifying choices to stakeholders. For example, we deferred features like magic links and Single Sign-On (SSO) to focus on more impactful initiatives initially.

Early user testing validated our approach, confirming the simplicity and effectiveness of the designs and setting the stage for successful broader implementation.

Solution and Execution

We addressed our user authentication challenges through structured improvements, focusing first on streamlining entry points and then on optimizing password management.

Phase 1: Single Email Entry We introduced a single email entry field that intelligently directs users either to sign in or register based on whether an account exists for that email. This update minimized user confusion by eliminating multiple entry points and incorporated bot detection to enhance security. Additionally, we refined our guidance for users, directing them to the forgotten password flow only when necessary, thus improving the entry process.

Phase 2: Optimising the Forgotten Password Flow We further refined the user experience by carrying over the user's email automatically to the forgotten password field, reducing re-entry errors. The help text throughout the reset process was overhauled for clarity, and unnecessary steps were eliminated to expedite the process. Post-reset, users are automatically logged into their accounts, enhancing convenience and satisfaction.

These targeted improvements were designed to create a more seamless, user-friendly experience, significantly reducing the cognitive load on our users and enhancing security measures. Through these phases, we sought to not only improve user retention but also to foster trust by ensuring a smooth and secure user interaction with our platform.

Before and after the sign in/registration improvements using A/B testing to validate hypotheses

OUTCOMES & IMPACT

Measuring Success

For Phase 1, we implemented the Single Entry Sign-In, achieving statistical significance within 24 hours. This update led to a 12% increase in successful logins and a 14% rise in successful registrations, while login failures decreased by 11%. Additionally, password recovery efficiency improved by 7%, contributing to a significant boost in user engagement with over 250,000 additional monthly logins.

Following the success of the first phase, we launched the Forgot Password improvements which reduced the drop-off rate during the reset password process from 30% to 7% , and increased successful logins post-reset by 8%. The reset process itself saw a 56% reduction in failures, and user engagement post-reset increased by 11%. These changes resulted in 21,000 additional monthly logins and markedly enhanced user satisfaction by streamlining the reset process.

Cumulatively, improvements made in 2023 added approximately 350,000 MAUs and reduced customer service contacts related to authentication issues by 70%. We saw a 5% reduction in dormancy rates as a result of the authentication initiatives.

Future Direction

In the initial discovery phase, we identified mobile sign-in enhancements and inconsistencies in 'forgot password' email displays across various email clients. This led to a significant redesign, improving password reset success rates. We also explored keyboard-based entry for Connected TV (CTV) to cater to user preferences over QR codes. Future plans include integrating magic links and third-party Single Sign-On (SSO) solutions like Google authentication to refine the user experience further. These initiatives aim to apply web enhancement insights to CTV and enhance overall accessibility and functionality.

RESULTS

11%

decrease in login failures

76%

improvement in password reset drop off rate

increase in registrations

12%

increase in successful logins

14%

increased conversion to view

56%

decrease in reset failures

11%

“Chloe enables the team to perform at their best. She is clear and concise with introducing new processes and supporting the team in doing so. She is really great at providing that bridge of knowledge and communication cross-functionally, allowing for great collaboration.”

Tom Barrett
Product Designer

Fancy another?

View All →