TLDR
- The variation redirected from Landing Page A to Landing Page B.
- Experiment code runs only where the experiment activates, so the listeners died on the redirect.
- Clicks were added on a Page that matches either URL, not on the experiment.
- The lead and scheduler listeners were added in Settings, then JavaScript.
- The experiment code editor was emptied.
Optimizely Web Experimentation loads a snippet on the page, splits visitors into an original and a variation, and can change what the variation sees. One of those changes is a redirect to a different URL.
We used that redirect. About 80 percent of visitors stayed on Landing Page A. About 20 percent were sent to Landing Page B. Both arms had visitors. Every conversion was on the original. The variation was zero.
The Redirect
Variation #1 was Optimizely’s redirect from Landing Page A to Landing Page B, not a change to the same page. The listeners for the clicks, the lead form, and the scheduler lived in the experiment code editor. Experiment code runs only where the experiment activates. Activation was a URL match on Landing Page A.
On Landing Page A, the snippet bucketed the visitor and ran that JavaScript. On Landing Page B, the snippet loaded again, but this experiment was not targeted there, so its JavaScript never started. The snippet being present is not the same thing as the experiment running.
The variation was bucketed on Landing Page A, the listeners attached, and the browser left for Landing Page B. A redirect is a full page load. Those listeners died with Landing Page A, the same way any other listener on that page would. The lead listener was waiting on a dataLayer push. The scheduler listener was waiting on a window message from an iframe. Neither was still registered when the action happened. The original never left Landing Page A, so it kept recording. Visitors were counted before the redirect, which is why the split looked fine.
A few other settings looked like the cause. They were not. The code was attached to the page the variation immediately left.
// Experiment code editor. Runs only where this experiment activates:
// Landing Page A. Variation #1 redirects away, so this listener
// is already gone when the booking happens on Landing Page B.
window.addEventListener('message', function (e) {
if (e.data === 'scheduler:booking') {
window.optimizely = window.optimizely || [];
window.optimizely.push({
type: 'event',
eventName: 'scheduler_booking_success'
});
}
});The Fix
Activation stayed on Landing Page A. Landing Page B stayed out of it. Someone who opens Landing Page B on their own is not in the test, and the visitor who was redirected is still the variation they were assigned on Landing Page A.
The clicks were added on a Page, not in the experiment. In Optimizely, create a page whose URL match is Landing Page A or Landing Page B. On that page, add one click event per button and put both selectors in the same selector field, separated by a comma. Do not add this page to the experiment. A click on either page then counts for the variation the visitor is already in.
The lead and the scheduler are not clicks, so they could not go on that page. They were added in the project script: Settings, then JavaScript. That is the project settings, not the code editor on the experiment. That script runs when the snippet loads, on Landing Page A and again on Landing Page B after the redirect. A path check keeps the events on these two URLs, so the rest of the site does not fire them. They sit inside the dataLayer wrapper that was already there. A second wrapper was not added.
Then the experiment code editor was emptied. Listeners left in there only run on Landing Page A, which is how the variation stayed at zero.
A new private window, allowed to redirect on its own, then a lead submitted on Landing Page B, put the first conversion on the variation.