Split Test WordPress Custom Post Types

Nelio A/B Testing 4.1 introduced support for Custom Post Types. As described by the Codex, Custom Post Types help you to “hold and display many different types of content”. Examples of Custom Post Types include WooCommerce’s Products, Staff Members lists, or other content types such as Movies, Reviews, or Articles.

WordPress can easily be extended in many ways, and we want you to test all those extensions. Nelio A/B Testing Plugin will help you test Custom Post Types, splitting your traffic between the alternatives you define, tracking the conversion rates of each alternative, and analyzing the results for you. A/B Testing will give you confidence that each change made to your WordPress site is generating more revenue, more subscriptions, and more clicks! Take a look at our installation, activation, and configuration guide and get the most out of Nelio.

Instructions for Split Testing Your Custom Post Types

Once you have installed, activated, and configured your A/B Testing Plugin, adhere to the following directions to begin split testing!

Create a New Experiment for Split Testing Your Custom Post Types

  1. Under the Nelio AB Testing menu, click Experiments to view and manage your experiments. Please note that the first time you access this page, you will not see any experiments.
  2. Three Different Ways to Begin:
    • If you have not yet defined any experiments, click the Create one now! link.
    • Click the Add New button placed beside the title, Experiments, in the upper left hand corner.
    • Under the Nelio AB Testing menu, click on the Add Experiment option.
  3.  You will now see a list with all the possible kind of experiments you can create. For each of them, you’ll see its name, an icon, and a description.
    • Click New A/B Test for Custom Post Types.

Setting Up Basic Information

Fill out the Basic Information of an Experiment. In this particular case, the basic information includes:

  • Name: a meaningful and descriptive name for the experiment that will help you identify it in the future.
  • Description: any additional information about your experiment that will better help you identify it.
  • Custom Post Type: you may have more than one custom post types defined in your WordPress installation. First, select the Custom Post Type of interest and, after that, select the actual post to test.
  • Original Custom Post: once you selected a Custom Post Type, you may select the actual post that will be tested and that will serve as the basis for the alternatives that you create (e.g. a Product or a Video).

Create Alternative Custom Post(s) for Split Testing

Create Alternative Custom Post(s) from Scratch

Under the Alternatives tab, click New Alternative (empty) which will allow you to create a new custom post from scratch.

  1. Name the alternative custom post beside Name (e.g. “Single Action Button” or “Blueish Landing Image”).
  2. Click the Create button right below Name.
  3. Create as many alternatives as you want and then edit them using the Save Experiment and Edit Content that appears below their names.

Create Alternative Custom Post(s) from an Existing Custom Post

Under the Alternatives tab, click New Alternative (based on an existing custom post) which will allow you to select a custom post whose title, content, and metadata will be copied.

  1. Name the alternative custom post beside Name (e.g. “Different featured image” or “Focus on testimonials”).
  2. Then, select the custom post whose content will be duplicated beside Source.
  3. Click the Create button right below Name.
  4. Create as many alternatives as you want and then edit them using the Save Experiment and Edit Content that appears below their names.

Goal Set Up

In order to define and configure which actions should be counted as conversions, go to the Goals tab. Actions are organized into Goals*. You may add additional Goals by clicking the Add Aditional Goal button. The first time you access this tab, you’ll see the Default, main goal. Actions are added as follows:

  1. Click on the New Action you want to create (for instance, if you want your users to view the Basic Package or the Professional Package page, add a new Page action).
  2. Configure the action properly (in our example, you’d have to select the select the Basic Package page, or the Professional Package, as well as any additional info required by the action*).
  3. Repeat the process for all the actions you need.

* For more information about how goals work and the available actions, check our Conversion Actions and Goals page.

Start Experiment

Once you have defined the experiment completely (that is, you defined the basic information, added one ore more alternatives, and organized one or more actions into goals), you may start the experiment. In order to do so, just go to the Experiments page, hover over the experiment you want to start, and click on Start to launch it.

* Please note that experiments cannot be edited once they’re running. You may edit the contents of the alternatives, even though this is not recommended. Changing the definition of an experiment while it’s running may impact on the performance of each alternative and, thus, invalidate the statistic results obtained.

Understanding Progress Information and Results

Find the Results of Your Experiment

Running experiments are shown in the Dashboard as simple cards where relevant information is shown. For this kind of experiments, the following information is available:

  • Page Views: how many times have your visitors viewed the tested page (either the original version or one of its alternatives).
  • Alternatives: number of alternatives defined in the experiment (original version included).
  • Original Version’s Conversion Rate: conversion rate of the original version.
  • Best Alternative’s Conversion Rate: among all the alternatives (original version excluded), it shows the highest conversion rate.

If you want to see more information about the Progress of the Experiment, either click on the experiment’s card in the Dashboard or go to the Experiments page, hover over the experiment, and click View.


The first part of the Progress of the Experiment page offers all the relevant information you need. With it, you’ll be able to know how each alternative is performing during time, which alternative is winning (if any), how confident we are of it, and some interesting data about the overall number of page views and conversions.

  1. In the Summary section, you’ll have all the relevant information you need at glance:
    • Colored Icon: it helps you identify the progress of your experiment quickly. It can be green (we are pretty confident that there is a clear winner), yellow (there is a winner, but we have a low confidence level), or red (there is no winner).
    • Current Winner: the winner of the experiment (if any) as well as our confidence level.
    • Best Alternative’s Conversion Rate: the percentage of times that visitors went to the designated goal page o post when shown the best alternative* (without including the original version).
    • Original’s Conversion Rate: the percentage of times that visitors went to the designated goal page or post when shown then original page.
    • Conversions/Page Views: the ratio of total conversions to page views for all alternatives.
    • A Timeline of the conversion rate for the original and each alternative.

* The best alternative is defined as the alternative that has the higher percentage.

Experiment Information

  1. Detailed information about the experiment:
    • The Name, Description, and the Actions of the selected Goal of the experiment.
    • The Original and Alternative elements you defined.
    • The winning alternative is highlighted in blue.
  2. Quick Action Buttons:
    • View heatmap link allows you to view the heatmap and clickmap for the page that is being tested.
    • Edit link is shown when the experiment is running and allows you to modify the experiment.
    • Apply link replaces the Edit link when the experiment is finished and allows you to override the original page with the contents of the winning alternative.
    • Stop Experiment Now button is shown when the experiment is running and allows you to terminate the experiment.


The second part of the Progress of the Experiment page shows three simple, yet powerful graphics, that will help you understand how every alternative is performing (specially compared to the original version of your experiment):

  • Information about the number of conversions and page views for the original and alternative elements.
  • Comparison of the conversion rates for the original and alternative elements.
  • The improvement factor of the alternative element with respect to the original element.

Advanced Section

Finally, you’ll find a Chart with the exact numbers and percentages used in the graphics are neatly displayed in the chart, as well as information about the type of statistical test used to compute the experiment (along with a link to an explanation of the test is displayed).

David AguileraWordPress Split Testing Custom Post Types