Explain Like I am 5 | Day 01: What is A/B Testing?
Explain Like I’m Five Series | Week 01 | Lesson 01—What is A/B Testing? One core experimentation concept, explained with clarity and practical intuition.
We are going to start a series “Explain like I’m 5” where we’ll break down the AB testing concepts that can be understood by even a 5 year old.
👋 Hey! This is Manisha Arora from PrepVector. Welcome to the Tech Growth Series, a newsletter that aims to bridge the gap between academic knowledge and practical aspects of data science. My goal is to simplify complicated data concepts, share my perspectives on the latest trends, and share my learnings from building and leading data teams.
Today we’ll try to understand a very basic question: What is A/B testing? But we won’t approach it from the angle of a data scientist. Instead, we’ll break it down for a five-year-old
Suppose you want to bake cookies for your classmates on your birthday next week. Your mom asks, “Should I add chocolate chips or not?”
You love chocolate chip cookies, but you’re not sure if everyone else will enjoy them too. You realize you have only one option: trust your gut feeling and hope that either the chocolate chip cookies or the plain ones are the right choice.
But your mom has a different idea. She decides this is the perfect moment to teach you a new concept, one that will help you make better decisions without relying on guesses.
She introduces you to A/B testing.
The Experiment
Mom bakes two types of cookies for your birthday celebration:
Cookies with chocolate chips
Cookies without chocolate chips
She asks you to bring both types to school and gives you clear instructions to follow:
Put an equal number of chocolate chip cookies and plain cookies on two separate plates
Give half of the kids the plate with chocolate chip cookies
Give the other half the plate with plain cookies
Whichever plate finishes faster is the type of cookie that kids prefer more
The Result
On your birthday, you discover that the chocolate chip cookies are gone much faster than the plain ones. You conclude that your classmates prefer chocolate chip cookies.
You’ve just learned A/B testing a technique that replaces guessing with real-life experimentation.
In Data Science Terms
A/B testing is the same idea, just applied to websites and digital products. We run A/B testing experiments to determine which version of something performs better and helps us reach our goals. Here’s how it works:
Version A (Control): The original version. For example, a blue call-to-action button on a website
Version B (Variant): The new version we want to test. For example, the same button but in red.
The process is like what you did with the cookies:
Split your website visitors randomly between the two versions
Compare how each version performs using a success metric such as how many people click the button
The version with the higher click rate wins
Some more relevant examples are:
Ecommerce: Checkout Page
Experiment: Comparing a multi-step checkout (where shipping, billing, and payment are on separate pages) against a single-page checkout.
Goal: Reduce cart abandonment issues
Streaming Platform: Content Thumbnail
Experiment: Showing different artwork/images for the same movie or TV show to different segments of users
Goal: Maximize watch time and engagement
SaaS: Pricing Display
Experiment: Showing Annual Billing Pricing vs Monthly Billing Pricing of a service
Goal: Increase Average Order Value
Conclusion
A/B testing is a framework for making data-driven decisions instead of relying on intuition. It lets you test different ideas in the real world, compare the results, and confidently choose what works best.
Just like you discovered that chocolate chips win with your classmates, companies use A/B testing to discover what truly resonates with their users.
Quick Question:
What if you want to:
test 5 different colors of call-to-action button?
test different font styles and colors of a call-to-action button simultaneously?
Stay tuned for answers in the next post.
If you’d like to dive deeper into experimentation, here are a few of our learning programs you might enjoy:
A/B Testing Course for Data Scientists and Product Managers
Learn how top product data scientists frame hypotheses, pick the right metrics, and turn A/B test results into product decisions. This course combines product thinking, experimentation design, and storytelling—skills that set apart analysts who influence roadmaps.
Advanced A/B Testing for Data Scientists
Master the experimentation frameworks used by leading tech teams. Learn to design powerful tests, analyze results with statistical rigor, and translate insights into product growth. A hands-on program for data scientists ready to influence strategy through experimentation.
Master Product Sense and AB Testing, and learn to use statistical methods to drive product growth. I focus on inculcating a problem-solving mindset, and application of data-driven strategies, including A/B Testing, ML, and Causal Inference, to drive product growth.
Not sure which course aligns with your goals? Send me a message on LinkedIn with your background and aspirations, and I’ll help you find the best fit for your journey.