Intuit Wasabi – A Scalable A/B Testing Solution

What is A/B Testing?

A/B testing or split testing is a way to statistically analyze and compare two versions of an app or a webpage to identify the better-performing one. It is a type of experimental testing where different page variants are shown to users randomly. You can then determine which one has a large user base or gives more profitable metrics with the results achieved.

Advantages of A/B testing

1. Strong connect between the company and the customer: A product analysis with A/B testing leads to better customer engagement. It sets the premise for a strong bond between the company and the customer.
2. Better conversion rate: A campaign based on results attracts a wider customer base, improves conversion rate, and acquires more paying customers.
3. Flawless product: First, a pilot test and then identifying profitable scenarios could lead to a robust and flawless product.
4. Simple metrics analysis: Since A/B testing is preferred on atomic or very small-scale components, it’s easy to analyze the data.
5. Leads to a surge in sales: The measures, taken based on the metrics derived from the A/B testing, eventually increases the product’s sales, leading to more profit.

Problem Statement

I was always looking for solutions to attract more customers in a product-based organization. Its product and marketing team were experimenting with minimal textual changes on the atomic components like page heading/buttons.

Initially, they were doing those experiments using Google Optimize. However, the results were troubling me in ways mentioned below:

1. Faced a lot of performance-related issues: Only after the page was loaded, per variant changes were observed. The site was then loaded with flickers, impacting the page performance. It led to the loss of a potential customer.
2. Not possible to do experiments with structural changes: The type of experiments possible to execute was only text variants. They were not ready to implement findings of any investigation involving structural changes to the experiment.

For example, check the different button text variants in the screenshot given below.

Solution

We had to find a solution to save ourselves from all the above problems. We wanted to conduct experiments by observing changes in atomic and molecular levels.

Luckily, we found a tool named Intuit Wasabi to solve all our needs. It was scalable and modular to carry out experiments on a more significant level.

Since it is an API-driven tool, one can easily integrate the APIs without impacting page performance.

1. Page performance was the topmost priority of our application, so we were already building applications in the SSR (Server-Side Rendering) way. We added the logic of executing an experiment at the back-end and directly showed the corresponding variant on the front-end, which saved us from the flicker.
2. We started with a POC and carried out atomic experiments only like checking the conversion rate of the customers with just button text change. Eventually, we succeeded with those experiments and decided to do the experiments to a greater extent, doing structural changes and finding out the user ratio per variant.
3. At a given point in time, multiple experiments with multiple variants can be executed. Then, find out which one of the variants per experiment gives better results.
4. It is free, open-source, and easily customizable.

Conclusion

With the help of Intuit Wasabi, we can execute various experiments with lots of customizable options. It helps the company identify the customer conversion rate and build a strong product for customers, and finally, earn better revenue. Do try this tool and share your experiences with us. Happy coding!