A/B Testing and Experiments

Experiments the Right Way

Have you ever wondered how Google chooses a color for its home page? How do they decide between two variations? In business, how can you know if changing something will improve it or not? Whether it's a website or model that powers your business, there is a way to determine which changes are "good" and should be rolled out, and which changes are “bad,” and should be discarded. The A/B Testing and Experiments training will teach you everything you need to know to successfully run your own experiments.

A/B Testing & Experiments is perfect for

Data Scientists and Analysts who work in organizations that use testing and want to professionalize experiments. Familiarity with statistics is required, as well as basic experience with Python. If you’re not quite there yet, we recommend, the Python for Data Analysts course as preparation for this training.

What will you learn during A/B Testing and Experiments?

After this training, you will know how to set up and validate an A/B test and confidently explain the statistics behind it to colleagues and management. You will also learn best practices for replicating experiments in your own organization.

Program

The program consists of both theory and hands-on exercises.

Day 1

• The importance of A/B testing
• How to determine the success of your experiment
• How to design an experiment:
• How to split the population
• How to determine the right metric for your experiment
• Experiment pitfalls and how to avoid them
• Classical hypothesis testing
• Further techniques for conducting A/B tests

Day 2

• How long to run an A/B test and why early stopping is a bad idea
• Bootstrapping to estimate uncertainties
• Bandit algorithms for real-time A/B testing

You will learn:

Designing and Running Experiments

• How to design meaningful experiments.
• The intuition behind the statistics needed to make an appropriate decision.
• The practical skills necessary to run your own experiments.
• Common mistakes made in the industry and how to avoid them.

Scaling Up Experimentation

• How to run multiple experiments at the same time
• How long to run an A/B test
• Why stopping early is a bad idea
• How to bootstrap to estimate uncertainties

Improving by Experimenting

• Conduct statistical experiments using Python.
• Avoid common pitfalls that can invalidate your experiments.
• Have confidence that you are making real improvements.

After the training you will be able to:

• Conduct statistical experiments using Python.
• Avoid common pitfalls that can invalidate your experiments.
• Confidently make choices that lead to real improvements.

Data Science Trainers

This Data Science training is brought to you by Xebia Data. Xebia Data is part of Xebia, just like Xebia Academy. Xebia Data works with experts in their field who are always on the lookout for the most innovative ways to get the most out of data. Your trainer is a data guru who enjoys sharing his or her experiences to help you work with the latest tools.

Data Science Learning Journey

Your Data Science Learning Journey starts with a Foundation course, such as the Certified Data Science with Python Foundation or Certified Analytics Translation training. Continue your journey with Advanced Data Science with Python or three days of Deep Learning. If you want to proceed to Expert level, register for our 3-day Data Science with Spark course.

Yes, I want to do more with data!

After registering for this training, you will receive a confirmation email with practical information. A week before the training, we will ask you about any dietary requirements and share literature if there's a need to prepare. See you soon!

What else should I know?

• This training requires a laptop. The hands-on labs are run in an online environment, eliminating the need to install software.
• All literature and a nice lunch are included in the price.
• Travel & accommodation expenses are not included. 
• This course is brought to you by Xebia Data.