Objective

This guide is designed to walk you through the step-by-step process of setting up an A/B test via AB Tasty to measure the business impact of the iAdvize solution through the implementation of “Conversation Starters*”.

*Conversation Starters are pre-formatted questions integrated directly into product and category pages. By proactively targeting potential visitor questions—such as composition or usage advice—they simplify access to essential answers right from the landing and foster customer engagement in the conversion funnel. Learn more.

The added value brought by iAdvize can be measured through:

• Add-to-cart rate
• Conversion rate
• Revenue per session or per visitor

Test principle:

• Group A: visitors not exposed to iAdvize
• Group B: visitors exposed to iAdvize

Visitors in groups A and B must be limited to those “eligible” to view iAdvize.

This guide enables you to configure an iAdvize A/B test via the AB Tasty platform on your website, regardless of the device (desktop, mobile, tablet).

Technical prerequisites

AB Tasty setup

• AB Tasty is correctly installed on your website
• Key goals are configured: add-to-cart, conversion, average cart value, revenue per visitor
• A visitor only enters an A/B test after giving their consent via the CMP

👉 Need help? Contact your AB Tasty CSM

iAdvize setup

• iAdvize is properly installed on your website
• Visitors can interact with iAdvize notifications called “Conversation Starters” and initiate a conversation with the Copilot.

👉 Need help? Contact your iAdvize CSM or Customer Support

Conditions for a successful test

An A/B test is considered successful when it shows a statistically significant improvement on a clearly defined KPI, with reliable and unbiased results. Here are some recommendations:

1. Define a clear objective

• Which KPI do you want to improve with iAdvize?
• What is the minimum expected gain on this KPI (in %)?

2. Optimize the experience of eligible users before launching the test

Before launching, to fully reveal the added value of iAdvize, it’s essential to:

• Validate the UX of the Conversation Starters on product pages (see our tips)
• Validate the post-conversation UX along the entire user journey (ensure that iAdvize does not hide key CTAs on other pages or checkout)
• Check the quality of the content (knowledge base) used to generate the displayed questions
• Ensure the Copilot’s answers are relevant

👉 Need help? Contact your iAdvize CSM or Customer Support

3. Set up a well-calibrated test

To ensure the statistical validity of the test and optimize its duration, you must:

• 2 groups: with iAdvize / without iAdvize
• 50/50 traffic split
• No other tests running simultaneously
• 100% of eligible traffic is dedicated to this test
• Test duration is predetermined and adapted to the traffic volume

Minimum recommended duration based on monthly eligible* traffic:

Assumptions: 50/50 split, 10% alpha error, 50% power.

4. Run a thorough validation before and after launch

To ensure a robust A/B test, check:

• The 50/50 traffic split between groups
• The experience of “with iAdvize” group visitors and correct display of iAdvize notifications throughout the navigation journey
• The experience of “without iAdvize” group visitors across the full site

3. Test configuration

Step 1: Create a trigger

• Go to https://app2.abtasty.com/dashboard
• Navigate to “Library” then “Trigger”

• Create a trigger we’ll call “iAdvize engagement display”. This trigger will be used for targeting. It checks whether an iAdvize Conversation Starter is displayed on the page.

• Select “Expert”

• Choose “Code” (via drag and drop)

• Enter the following code, then save:

window.iAdvizeInterface = window.iAdvizeInterface || [];

window.iAdvizeInterface.push(function (iAdvize) {

  iAdvize.on('engagementNotification:displayed', function () {

    abResolve(true);

  });

});

Step 2: Create the test

• Go to “Web experimentation”

• Click “Create” to start setting up the new test

• Select “A/B Test”

1- Main information

• Fill in the name and hypotheses of the test.

• Enter your website’s URL, required for test setup.

• Select “Code editor”, then click “Save & go to next step”.

2- Code editor

• Enter the code below and paste it into the editor

• Click “Save & go to next step”

• This code creates a “Without iAdvize” variation in which Conversation Starters will be hidden.

#embedded-conversation-starter-container {

  display:none!important;

} 

3- Objectives

• Select the test objectives, then proceed to the next step.

4- Targeting

• Segment: all visitors

• Page(s): target all pages of your site
  A broad scope is recommended to cover all possible iAdvize displays. The Trigger defined in Step 1 is enough to target the right pages, no need to specify URLs.

• Trigger(s): select the “iAdvize engagement display” trigger

• Frequency: All the time

• Click “Save & go to next step”

5- Traffic allocation

• Adjust the traffic split between the 2 groups (50/50 is the ideal distribution)

• Edit the name of Variation 1 → Without iAdvize, then proceed to the next step

6- Advanced options (optional)

7- QA & Preview

• Perform QA of the test experience on both desktop and mobile

• Then finalize the test setup

Results

Once the test is live, you’ll find the data in the AB Tasty interface, in the Reporting page dedicated to your experience.

You can change the reference group for result analysis to “Without iAdvize” (or Variation 1) to make the results easier to read.

When hovering over “Variation 1”, the “Reference” button appears. Simply click it.

The selected reference group is automatically saved for future result views.