Leemay Nassery is an engineering leader at Spotify, focusing Home page personalization and experimentation. She is also the author of Next Level A/B Testing.
Upcoming Talks
14 May 2025
Continuous Design
Stage 2
Launching a product feature powered by a model requires one essential step: A/B testingIn this session, we’ll detail exactly what you need on the experimentation side to make sure your model is performing better, ideally, then the control. The challenge in experimentation isn’t running the tests themselves, but rather the nuances and details key to designing a well-structured experiment, enabling you to gain the right insights to evolve your product and inform your larger AI strategy. Discussion points include:
- Understanding model behavior offline before starting the online evaluation phase
- Verifying experiments before launch through canaries or spot-check tooling
- Basic design principles that are fundamental to launching an A/B test on the product
- Simple strategies we implemented for Spotify’s Home page to increase the quality of experiments launched on the product