Exit Confirmation Feature Increased Lesson Completion Conversion by 5-8%

1. Summary

We implemented a mechanism to retain users in lessons and improved the CR_lessons_end metric. A/B tests conducted on iOS and Android showed the following results: on iOS, the conversion rate for lesson completion increased by +5%. Of those who saw the pop-up, 41.11% clicked the "continue lesson" button at least once, and 68.69% clicked the "exit" button at least once. For Android, the lesson completion conversion increased by +8% for new users and +2% for paying users.

2. About the Project

We aimed to significantly impact lesson completion conversion by reducing interruptions during the lesson. To achieve this, we decided to revisit the current exit behavior in lessons on Android and iOS.

2.1. Current Behavior

On the lesson screen, there is an exit button “X”. When clicked, the user exits the lesson without any additional prompts and, upon return, continues from where they left off. However, this approach may confuse users, as some lessons will restart from the beginning, while others will resume from the stopping point.

2.2. Hypothesis

If we stop saving lesson progress and warn the user in advance, this could significantly improve the CR_lessons_end metric.

To test this hypothesis, we plan to launch an A/B test where:

1. The control group (A) remains unchanged: the current logic of saving lesson progress upon exit will be maintained, with no exit confirmation.
2. The test group (B) will not save user progress, and upon exiting the lesson, users will see a modal warning.

Monitoring Metrics:

1. lessons_end (average number of completed lessons)
2. lessons_start (average number of started lessons)
3. count_days (average number of active days)

3. Design

Users will see a warning window informing them that if they exit, they will lose all progress and have to start from the beginning. This should prevent some users from exiting and change their decision to stop the lesson. If a user exits right from the first screen of the assignment, we will let them go without a warning, as they have not performed any useful actions or engaged with the lesson. The modal warning will only appear when clicking the exit icon, starting from the second lesson.

5. Conclusion

By spending minimal development team resources and implementing a small change, we were able to improve the key metrics for our team.