Why you shouldn’t do A/B price testing?

You can put off your potential and existing customers

Due to the randomness of your traffic, the A/B test can lead to problematic customer service scenarios. If a sales specialist who got caught by your 10€ pricing is encouraging his or her boss to purchase software that to this person shows for 20€, you will most likely anchor them in the lower pricing point. It will be a nightmare converting them to higher pricing if it turns out to be the optimal one.

Now imagine another company where neither of these 2 was lucky enough to land on the cheaper plan. 

They became regular users, and through word of mouth, they figured out that they could've paid two times less. I'm sure you wouldn't like to be in their situation, and most likely, you'd raise questions. The more significant the price difference between plans, the harsher the questions will be.

To avoid any bad customer experiences, you need to be sure that they:

• use only one device to browse your website, 
• don't revisit it, 
• are the only employees visiting the pricing plan, 
• and never discuss software prices with their colleagues or anyone else. 

If you can't be sure about even one of the above, then A/B tests will likely expose you to friction in the sales process.

You're overcomplicating things

If your pricing testing methods consist of 3 different plans, then the A/B test will double the number of buying possibilities and create a total of 6 segments. 

On the marketing level, you're making a different analytical thing to cover. From a sales perspective, you're splitting your pricing policy, which is yet another thing to take care of by the sales team (just like the discount policy).

Cohort analysis is already complicated, and by multiplying it, you make it even harder. With A/B testing, you need to deal with two pricing plans that will affect your day-to-day sales and marketing efforts.

You're limiting your business potential

The ultimate goal of A/B testing is to discover which of the two variables performs better. You're revealing if the first one is more successful than the other in A and B's duel.

This, by any means, doesn't indicate that the winner is an optimal setup for your business. You can understand if A is better than B, but you don't know if A is good at all. You're limiting your global vision to nothing but multiple price points, which are just a part of complex pricing plans.

The perfect pricing point is found through surveying your existing and new customers and gathering their feedback. You need to understand what features they perceived value most and which they want to get rid of. And ultimately, what is their Willingness to Pay for the product?

If you limit your pricing strategy to nothing but A/B testing, you're overlooking multiple factors that are significant in setting up an optimal pricing strategy. 

Or, as Vilgefortz, one of the sorcerers from The Witcher's world, said: you mistake stars reflected in a pond for the night sky.⭐

What we have learned

A/B testing is an excellent tool for comparing conversion rates of website visuals, such as CTA buttons or different web designs. It's suitable for landing pages where significant amounts of people convert in short periods.

If your business is not in this stage yet, the A/B test won't provide you with valuable conversion rate optimization insights. On the contrary, it will expose you to sales, marketing, customer service, and PR challenges.

To avoid those risks and to start working towards a suitable pricing strategy, I encourage you to read this article first. And in case of any questions, don't hesitate to reach out to us via the contact form below.

FAQ

#1 Is A/B price testing illegal?

A/B price testing methods can be illegal if it involves price discrimination based on a customer's nationality. Charging different prices solely because of someone's country of residence can be against the law, such as in the EU where price discrimination is prohibited.

#2 Why does A/B testing require a significant amount of traffic?

A/B testing requires a large enough sample size to ensure statistical significance. To obtain reliable results, you need a sufficient number of visitors to your website or users of your product. Small sample sizes may not provide conclusive information.

#3 Can A/B price testing turn off potential customers?

Yes, A/B price testing can lead to customer dissatisfaction if they discover they could have paid a different, lower price. Inconsistent pricing experiences can create confusion and undermine customer trust in your brand.

#4 Does A/B testing overcomplicate pricing strategies?

A/B testing can complicate pricing strategies by increasing the number of price segments and creating additional complexity for marketing and sales teams. Managing multiple pricing plans can be challenging and may lead to confusion.

#5 Does A/B price testing limit business potential?

A/B price testing focuses only on comparing two variables and doesn't consider other important factors in pricing plan, such as customer feedback, willingness to pay, and feature preferences. It can restrict your understanding of optimal pricing.

#6 Is A/B testing suitable for all businesses?

A/B testing is not suitable for all businesses, especially those with low traffic or limited conversion rates. It is more effective for websites with significant traffic and higher conversion rates where meaningful insights can be obtained.

#7 Can A/B testing create customer service challenges?

A/B testing can lead to customer service challenges if customers discover different pricing options and feel they were not given the best price. Inconsistent pricing can result in negative experiences and difficulties in upselling.

#8 What are the risks of relying solely on A/B testing for pricing decisions?

Relying solely on A/B testing for pricing decisions overlooks various factors crucial for setting an optimal pricing plan. Important considerations such as market demand, cost-plus pricing, and consumer expectations may be disregarded.

#9  How does A/B testing impact marketing efforts?

A/B testing affects marketing efforts by creating additional analytical and strategic considerations. It can complicate cohort analysis, split pricing policies, and require separate sales and marketing approaches for different price segments.

#10 What alternatives should be considered instead of A/B price testing?

Instead of A/B price testing, businesses should consider surveying customers to gather feedback, understanding their preferences, and determining their willingness to pay. This comprehensive approach provides a more holistic view of the pricing plan.

‍