Optimizely

Um, I wanna tell you a little bit about some of the problems that can arise when you use experiments in online settings as, as we typically do in the tech industry and beyond, um, and also what we can do about it.

Uh, so before I start, just one thing I should highlight, um, the work I'm talking about was actually done, um, under my role as a technical advisor to a company called Optimizely, which is a, a large, uh, AB testing platform.

So they provide essentially a platform that allows other companies to run AB tests on their, on their sites.

So I'll tell you a little bit about, you know, first what exactly is AB testing, just to make sure we're all on the same page.

Um, what's really interesting today is that we're kind of living through a period where the ability to run these experiments is being democratized quickly.

And what I mean by democratized is that now, um, sort of a typical employee at a business has the ability to run these experiments without really a lot of engineering or statistical knowledge.

That's kind of what the point of Optimizely really was.

It was to make it so that all you needed to know was a little bit of, you know, ability to code up a webpage.

I'm so thankful for them.

And then the session I did yesterday was about agentic ABM, what's coming next in terms of ABM and really in marketing in general because of AI.

Why do you think your

session sold out so fast? Like, what do you think is happening in marketing that made a bunch of marketers look at the list of like 20 or so sessions and pick a Gentic ABM?

Yeah.

So I think that my session was particularly interesting because a lot of marketers right now are very excited, but also very unsure about what's happening with AI.

Understandably, any transformation comes with unknown risks and unknown rewards.

So ABM is kind of this new wave of, I would say, marketing in general.

I eventually wanted to, uh, take what I learned and apply that to a much earlier stage venture, so I ended up joining a startup as a first finance hire.

The company name was Optimizely.

When I decided to join Optimizely, the decision was driven a lot by a mentor's advice, which is when you see a rocket ship, you strap yourself on and you go for the ride.

And Optimizely at the time was going through a phase of hypergrowth.

I think the biggest lesson learned from Optimizely is that every business can embrace a spirit of experimentation very early on, and experimentation's going to look different at every stage of the business.

When you're very small, you know, experimentation is really about trying to learn from your users, a lot of qualitative feedback, talking to them, understanding what their pain points are, showing them initial concepts or prototypes, and then being willing to go back and iterate on that.

Companies that embrace experimentation understand that when you ship a product or a feature, that isn't the end.

That is the beginning.

Yeah.

And then I got the first job that brought me to San Francisco, which was, uh, at Optimizely, and they were, they were hiring for a role called customer education manager on a team called customer success.

And I was like, "Customer education sounds like what I do.

Customer success, never heard about this." This is 2013.

And I called my dad and I was like, "Hey, I just got offered this role doing customer education on a customer success team." And he said, "That sounds fake.

Don't take it."

Okay, so Optimizely was, uh, incredibly ahead of their time.

Wow.

But typical players.

Well, and, you know, but then upward from that, if you have more than 50,000 sessions, you're very limited in choices, 'cause you're gonna start, for the Google recommended one, A- VWO, AB testing, Optimizely, you're gonna start at, like, a $30,000 subscription.

But if you are on a Shopify, there are many Shopify partners that allow you.

So we're partnered with a company called Shoplift, which is one of my favorite names.

At the beginning of my career I studied computer science at Stanford, left Stanford to go started a career at Google as an associate product manager, and in November 2007 joined the Obama campaign.

That's inspired me to start my first company, Optimizely, which we grew from zero to $120 million annual recurring revenue, 450 employees.

We sold that company and that's when I started Rewind.

Rewind is a personalized AI powered by everything you've seen, said, or heard.

You know, it's, it's not a new idea.

It's really just the scientific method, but it's applied to the real world.

What ultimately inspired me to start Optimizely, which was trying to build a product I wish we had in the Obama campaign to make it easy for anybody to do A/B testing.

At that time, the tools we used on the campaign and those that were available online were very expensive.

Our next guest here on Future of Marketing loves hot chocolate, so he started actually a hot chocolate brand, and then ended up spending near fifteen years at SAP, starting as chief of staff to the CEO and ending up running two of its biggest businesses.

Today, Alex runs Optimizely, a company that went from a content management system to the AI platform for marketing.

So Alex, welcome to the show.

We're so excited to have you here with us today.

Very nice to see you again.

And Ethan, thanks so much for having me.

So I was looking to your background, and it's very interesting in how you became the CEO of Optimizely.

You were the CEO of Episerver and then took it through the Optimizely acquisition.

And we're like, "How do you test things?" And they're like, "Oh, we just enable, like, a small section of our website, you know, like the cooking section." And we're like, "Okay.

You know, you could get, what was it, Optimizely?" And there's a whole bunch of services out there that build test pools, and they are super expensive, and they slow down your site, and they do a bunch of things, and you still have to think about randomized traffic and what geos are you testing.

Really sophisticated publishers have that stuff, and they've had it for a long time because they had to.

The original version of these, I think you can think of, were like the wrappers, like Demand Manager and Open Wrap, even the Index wrapper from way back in the day.

The downsides, I think, are limited, but you will have to learn a new system.

There is a cost to these platforms, and you'll have to evaluate that against the human cost of what it takes to run those, and the tech costs.

Optimizely was incredibly expensive, and that was just to run test pools, and, like, we'd have to turn them off and on and manage them.

That was super expensive.

But for companies, though, I don't know.

Because, I don't know, you're going to send the email from, hey, it's Bob at Optimizely.

Do people care about Bob at Optimizely? Maybe not.

But me, you, we have more of a... affiliation with the space, more of a personal stamp behind the things that we put out.

So I think it means more.

the three letter acronym.

You know, we talked about last time, I think Optimizely has retired their DXP-

Yeah

... label.