Optimizely has been a great website testing and optimization tool in Ladder’s arsenal for the past two years, but the signs are on the wall that it’s time to move on.
It started with issues around the fact that they don’t allow self-serve signup and quote different prices per client, and we were getting frustrated. Then some of our strategists started having problems getting actual help from Optimizely’s support whenever we came upon an error or issue.
So we switched to VWO, and a ton of those support issues were gone. Beyond that VWO turned out to be a quality product for a fraction of the cost, making our on-site optimization and A/B testing tactics easy to execute.
But the problem we ran into with VWO was this: their standard tiered plans allow for just 2 sub-accounts, making it difficult to work on the system with more than two clients. Further, it would be too difficult for us to front an account for every client.
So my own difficulty became this: We already have two clients on VWO, and Optimizely has presented issues in the past. But I needed to start running site optimization tests for Ladder as part of our 2017 marketing plan.
So I went with Optimizely anyways, and here’s what happened:
Ok… that’s weird. So it kept loading and loading and loading. Switching to interactive mode did nothing to fix the problem either.
Then I figured out that I could turn on unsafe scripts (scary!) and it would work:
Except… It still loaded our website strangely, and switching over to interactive mode only made it load infinitely again.
Blah.
But wait… The whole point of the experiment I intended to run was a change of our Hero Section Copy. We recently uploaded a new meta description that we developed in our SEO Audit, and we wanted to test whether we’d get lower bounce rates and higher conversion rates with the new description serving as our Hero Copy.
Since the hero section loaded, I could go ahead and run the test anyways. So I tried. I changed the copy, which worked just fine. The only thing left was to set goals so we could actually track what was going on – one for engagement and one for CTA clicks. Engagement was easy. CTA clicks… that’s where things got complicated again.
Same issue. At this point, I was frustrated and considering just running the test on guesswork. But we don’t do that at Ladder, so I tempered my frustration and remembered I’d actually signed up for and gotten invited into the Google Optimize beta.
“Wait!” I hear you saying. “That’s just a problem with your site not integrating well with Optimizely.”
Sure… Except we’ve had this issue with clients, both past and current, and we don’t see the same problems with VWO or Google Optimize.
So rather than bother with Optimizely’s mediocre customer support, I went for Google Optimize.
And boy was it easy.
Below I’ll detail exactly how easy it was, and why we’re switching our site optimization and A/B testing tool for Ladder’s own marketing efforts away from Optimizely and to Google Optimize.
Let’s get started:
Setup was an absolute breeze. It’s a Google product, after all, and plays extremely well with every other Google product we currently use on the Ladder homepage — Google Tag Manager, Google Analytics, etc… I set up Optimize through Google Tag Manager, simply copying over the Optimize code.
And finally, Optimize asked me to download a Chrome extension that serves as the editor for the page I want to test. Simple, quick install.
And we’re done! Google Optimize is all set up. Easy, right?
Here’s how easy it is to run an experiment in Google Optimize:
(NOTE: GIF too small? Here’s the full-sized version: CLICK HERE >>)
Yeah.
Here’s a step-by-step of what happens in that GIF.
First thing you do once you hit your dashboard is click “Create Experiment.”
This prompts you to name your experiment and name the page you’re going to be editing. You can also choose what type of test you want to run — A/B, Multivariate, or Redirect. We’re currently running A/B tests.
And that’s it. Hit next, and you’ll be asked to…
…set your hypothesis. If you know anything about the way Ladder works, we love writing hypotheses. It’s an integral part of the Ladder Planner.
And being able to just enter the hypothesis we added to the Ladder Planner here helps us keep our tests aligned across platforms.
Next, we’ll have to…
…set up our goals! Here’s the nicest part of Google Optimize. Since it lives in your Google business ecosystem, it connects directly with your Google Analytics account. The implications? You can actually have your experiments immediately connect to the goals you already set up.
This is extremely useful for us, because we set up tons of Google Analytics goals for both ourselves and all our clients.
Now we’ll set up which goals we want to track with this experiment. Since we’re tracking whether or not we get more submitted forms in the hero section, our Hero Form Submission goal is the primary goal here.
But we can select up to 3 goals. An interesting seconday goal might be bounce rate. Does the new CTA make people stay on the site longer and actually click around? And a tertiary goal might be our Contact Form Submission goal, which tracks the submission of our more detailed contact form further down the page.
This one is interesting because we might want to see how the CTA copy at the top affects performance of our other CTAs.
So here’s our final setup:
And now let’s get to…
…who we’re targeting.
Very straightforward stuff. We’ll want 100% of our traffic to get both the original and variant versions of the CTA copy, and we’ll want each to get 50% of the traffic. We’ll have the test show on page load, since we want them to see it immediately. You can also have a test load on a custom event, which lets you do some other fun things like load different forms for different visitors when they click a CTA.
You can also set a boatload of other rules and parameters, which I won’t go into here.
We don’t need any of them for this experiment, but they’re really useful across the board for any test you want to run.
We’re all set with parameters! Now it’s time for the fun part — actually creating your Version B.
Here’s how:
Name your variant whatever you want to. We’ll call ours “CTA Edit” to keep it simple.
Then click on the variant and you’ll be taken to the page you chose to test.
Next, select the element you want to edit. Ours is the Hero Section Call to Action Button.
Edit the element as needed – you can remove it, edit the text, edit the HTML, or insert custom HTML / JavaScript. We’re just editing the text here.
And that’s it! Click done at the bottom-right, hit save then done at the top-right, and you’re ready to…
…hit Start Experiment!
Our experiment is now live, and we’re about to start gathering data from your visitors!
So we’re not actually going to run the experiment above (maybe next month!) but I will show you what’s happening with our most important currently-running experiment — the Hero Copy change I talked about at the start of this post.
Google Optimize has a really nice dashboard that shows performance, but to be perfectly honest, I don’t want to keep going back into Optimize to find performance.
Luckily, this is all connected to Google Analytics, remember?
All our tests are right there! Under Behavior > Experiments, you can see every single experiment you’re running and have run. The only caveat here is that you can’t currently filter out experiments that you’ve stopped running, so it can and will get a bit cluttered. I doubt Google will let that be the case in the future.
So that’s how our hero copy test is performing. Clearly our new copy is killing it so far, to the point where in just 8 days of testing, we’ve reached >80% statistical significance that the new copy is the better choice.
Damn useful tracking, with no extra setup work. That’s a huge advantage over Optimizely.
Google Optimize is still a beta product, so there are naturally some wonky behaviors and limitations that mean we can ONLY use it in-house rather than with all our clients.
Here are the main limitations that make it an imperfect beta product:
So you can only have one container per Google Optimize account. That means we can’t set up multiple clients, because we can’t create multiple Optimize codes in a single account for us to place on clients’ websites. Bummer. That means we’ll have to use Optimizely where it works for clients, and use alternatives like Unbounce to create custom landing pages where Optimizely fails.
You can only run three experiments at any given time in a single container. As an agency that prides itself on marketing tactics and ROI-driven tests, this hurts quite a bit. We love testing anything and everything to constantly optimize our homepage for conversion, so not being able to do more than three tests at a time limits our ability to run experiments that move the needle for our business.
Not truly a major drawback, as generally you’d only want to have two or three goals tracked, but it’d be nice to track many goals at once. It’s not a deal-breaker by any means, but Google should consider removing the limitations, since each metric is tracked independently.
Here’s an annoying one – for some reason, Google Optimize experiments don’t seem to receive real-time data. I’m not yet sure on what interval new data is updated, but it’s definitely not live. Not a deal-breaker either, but Google really needs to make sure this is running real-time just like the Google Analytics real-time overview.
Google Optimize is currently the best possible site optimization and A/B testing tool in our arsenal, at least for use with Ladder’s own campaigns. Since it’s a beta, invite-only product and we don’t have the ability to invite clients, we currently can only realistically use it for ourselves.
What we’re expecting is that Google brings Optimize out of beta soon with some agency-level pricing options so we can completely transition away from VWO and Optimizely in favor of Optimize.
Optimizely and VWO should be shaking in their boots. Google seems to be getting this product right in so many ways, and if they nail down pricing, that could deal a huge blow to any competitor.
As an agency that uses the latest tech to grow businesses, we’re eager to switch to something so effective and easy to use. It should make the work of our strategists a lot more streamlined and optimized ?.
Sign up to our newsletter to stay up to date with all the latest movements in the field.
More from Ladder Blog
The art world is being disrupted by generative AI, and artists aren’t happy. Generative AI models like DALL-E, Midjourney, and Stable Diffusion were trained by scraping millions of images from the internet, without permission. Artists claim this violates copyright, while the companies training these models argue this falls under fair use.
Read More →In the world of digital marketing and data-driven decision-making, creative testing is a pivotal tool in achieving business growth. Gone are the days of relying on gut feelings or guesswork; now, business decisions are powered by data-validated insights, meticulously collected, analyzed, and validated. This transformative process empowers businesses of all sizes, from established enterprises to budding startups, to thrive in an ever-evolving digital market. This article looks at the practical applications, challenges, and innovative transformations associated with creative testing, offering you valuable insights and actionable strategies to implement in your own digital marketing efforts for achieving growth and success.
Read More →
