Skip to content
Article

Is Optimizely actually working?

Optimizely can show a healthy visitor split while the action you care about is never recorded. Check the snippet, the bucket for this visit, and the network call before you trust a zero in the results.

TLDR

  • Accept cookies and reload, then confirm the Optimizely script tag is on the page.
  • Confirm window.optimizely.get is a function and your event is in the project catalog.
  • Use a new private window and confirm this visit is in the experiment before you click.
  • Listen for the event, then confirm logx.optimizely.com/v1/events returned 204.
  • URL targeting does not carry from one host to another.
  • A zero in the first minute is a delay, not a failed event.

Optimizely can show a healthy test while the action you care about is not being recorded. Check three things, in order: the snippet is on the page, this visitor is in the experiment, and the event left the browser.

A healthy visitor split is not proof that events are recording. On a redirect test, visitors are counted before the browser leaves the first page.

Is the snippet on the page?

The snippet is a script tag. If it is missing, nothing else in this list can run. It is added only after the visitor accepts cookies, so accept them and reload. Changing consent without a reload leaves the tag off the page. The tag is also left off for one region on purpose. Look for the tag before you debug the experiment.

document.querySelector('script[src*="optimizely"]');

Is the client alive?

After the tag loads, window.optimizely.get should be a function. get('data').events is the list of events the project knows about. If the name you expect is not in that list, the event was never created. Clicking will not record it.

console.log(typeof window.optimizely.get === 'function');
console.log(window.optimizely.get('data').events);

Am I in the experiment?

getCampaignStates({ isActive: true }) shows the campaigns running for this visit. getRedirectInfo() tells you whether this visit was redirected. A variation name of null is a masked label, not a missing bucket. Read the variation id next to it.

On a redirect test, confirm the redirect before you click or submit. An old bucket stays with the browser. Use a new private window and let the redirect happen on its own. Opening the destination by typing it does not put you in the test.

var state = window.optimizely.get('state');
console.log(state.getCampaignStates({ isActive: true }));
console.log(state.getRedirectInfo());

Did the event fire, and did it leave the browser?

Add a listener before you take the action. Then click or submit. The console should print the event name immediately.

window.optimizely.push({
  type: 'addListener',
  filter: { type: 'analytics', name: 'trackEvent' },
  handler: function (event) {
    console.log(event.data.name);
  }
});

The console only proves the page fired it. In the Network tab, filter for logx.optimizely.com/v1/events. A 204 means Optimizely accepted the request. The event key in the payload should be the name you meant to send.

Two things that look like a broken test

URL targeting does not follow you from one host to another. An experiment aimed at staging does not run on dev or production until those hosts are in the page match.

Results trail the console. The event can be in the network log and still be missing from the report for a few minutes. A zero in the first minute is not a failed event.

Checklist

  1. Accept cookies and reload. Confirm the Optimizely script tag is on the page.

  2. Confirm window.optimizely.get is a function, and that your event is in get('data').events.

  3. Open a new private window. Confirm an active campaign. On a redirect, confirm the visit was redirected before you do anything else.

  4. Add the listener, then take the action. The event name should print in the console.

  5. Confirm logx.optimizely.com/v1/events returned 204, and the payload key is the event you sent.

  6. Wait a few minutes before you treat a zero in results as a failure. Confirm this host is in the experiment's URL targeting.

Drag to pan. Use +/− or Ctrl/Cmd + scroll to zoom. Pinch to zoom on touch devices.

Drag to pan. Use +/− or Ctrl/Cmd + scroll to zoom. Pinch to zoom on touch devices.