Objective

This guide is designed to walk you through the step-by-step process of setting up an A/B test via Kameleoon to measure the business impact of the iAdvize solution through the implementation of “Conversation Starters*”.

*Conversation Starters are pre-formatted questions integrated directly into product and category pages. By proactively targeting potential visitor questions—such as composition or usage advice—they simplify access to essential answers right from the landing and foster customer engagement in the conversion funnel. Learn more.

The added value brought by iAdvize can be measured through:

• Add-to-cart rate
• Conversion rate
• Revenue per session or per visitor

Test principle:

• Group A: visitors not exposed to iAdvize
• Group B: visitors exposed to iAdvize

Visitors in groups A and B must be limited to those “eligible” to view iAdvize.

This guide enables you to configure an iAdvize A/B test via the Kameleoon platform on your website, regardless of the device (desktop, mobile, tablet).

Technical prerequisites

Kameleoon setup

• Kameleoon is correctly installed on your website
• Key goals are configured: add-to-cart, conversion, average cart value, revenue per visitor
• A visitor only enters an A/B test after giving their consent via the CMP

👉 Need help? Contact your Kameleoon CSM

iAdvize setup

• iAdvize is properly installed on your website
• Visitors can interact with iAdvize notifications called “Conversation Starters” and initiate a conversation with the Copilot.

👉 Need help? Contact your iAdvize CSM or Customer Support

Conditions for a successful test

An A/B test is considered successful when it shows a statistically significant improvement on a clearly defined KPI, with reliable and unbiased results. Here are some recommendations:

1. Define a clear objective

• Which KPI do you want to improve with iAdvize?
• What is the minimum expected gain on this KPI (in %)?

2. Optimize the experience of eligible users before launching the test

Before launching, to fully reveal the added value of iAdvize, it’s essential to:

• Validate the UX of the Conversation Starters on product pages (see our tips)
• Validate the post-conversation UX along the entire user journey (ensure that iAdvize does not hide key CTAs on other pages or checkout)
• Check the quality of the content (knowledge base) used to generate the displayed questions
• Ensure the Copilot’s answers are relevant

👉 Need help? Contact your iAdvize CSM or Customer Support

3. Set up a well-calibrated test

To ensure the statistical validity of the test and optimize its duration, you must:

• 2 groups: with iAdvize / without iAdvize
• 50/50 traffic split
• No other tests running simultaneously
• 100% of eligible traffic is dedicated to this test
• Test duration is predetermined and adapted to the traffic volume

Minimum recommended duration based on monthly eligible* traffic:

Assumptions: 50/50 split, minimum detectable gain of 5% on a 4% conversion rate (alpha error 10%, beta error 30%). The conversion rate is calculated based on eligible iAdvize visitors.

To conduct your own analysis, use the Kameleoon simulator.

*How to evaluate your monthly volume of eligible iAdvize visitors?

iAdvize is implemented on all your product pages?

👉 the number of eligible visitors corresponds to the number of unique monthly visitors who:

• have viewed at least one product page
• have accepted cookies via the CMP banner

iAdvize is implemented on product pages of only certain categories?

👉 the number of eligible visitors corresponds to the number of unique monthly visitors who:

• have viewed at least one product page from those categories
• have accepted cookies via the CMP banner

4. Run a thorough validation before and after launch

To ensure a robust A/B test, check:

• The 50/50 traffic split between groups
• The experience of “with iAdvize” group visitors and correct display of iAdvize notifications throughout the navigation journey
• The experience of “without iAdvize” group visitors across the full site

Test configuration

Create a trigger

• Go to https://app.kameleoon.com/triggers/dashboard
• Create a new trigger. We’ll name it: Display of an iAdvize engagement. This trigger will be used during targeting. It checks every few seconds for the display of an iAdvize Conversation Starter.
• Select Condition → Custom JavaScript
• Select Run condition asynchronously
• Enter the following code:

window.iAdvizeInterface = window.iAdvizeInterface || [];
window.iAdvizeInterface.push(function (iAdvize) {
  iAdvize.on('engagementNotification:displayed', function (notification) {
    if (notification.type == 'EMBEDDED_CONVERSATION_STARTER') {
      setTargeting(true);
    }
  });
});

Create the test

Variations

To facilitate result analysis (clear group comparison), create two variations:

• Without iAdvize
• With iAdvize

Edit the “Without iAdvize” variation

1. Select Without iAdvize
2. Click on Edit mode

3. Double-click the Conversation Starter element

4. Click on Select another relative
   
   
5. Choose #embedded-conversation-starter-container

6. Then click Remove element

7. Click Finalize

Set up targeting

We need to ensure that only users eligible for iAdvize are included in this test.

1. Choose to target specific visitors
2. Setup:
• Trigger → Matches → “Display of an iAdvize engagement”

Configure traffic allocation

Adjust traffic distribution between the two variations.

View results

Once the test is running, you will find data in the Kameleoon interface on the dedicated results page of your experiment.

To make it easier to interpret the results, you can set the “Without iAdvize” variant as the reference.