This talk will help you understand the concept of A/B testing, stop confusing it with a redesign or staged release. For those who do not yet use A/B tests in their projects, we’ll show how to choose the right time to run them. We’ll analyze the examples of A/B testing made in Spotify, Airbnb and Amazon. We’ll also touch on our own implementation in Revolut: Android, Kotlin, Rx, FireBase Remote Config. We'll look at the problems we faced while making A/B tests:
- Tests implementation while working simultaneously on lots of new features in a large team;
- Maintaining unit tests up and running regardless of A/B;
- A/B test rapid development and deployment;
- Safe switching on/off of an A/B test;
- The complexity of collecting and analyzing the results. In conclusion, we’ll understand how to properly assess and present the results of A/B testing.