‘The hype is real:’ Lyft Head of Marketing Science Alok Gupta on marketing measurement

Marketing is an art. Marketing is a science. Just maybe, measurement is the Gordian knot that ties them both together.

We’ve certainly seen an increasing understanding of how the science of marketing is critical to unleashing the art of marketing in the last few years. Science — and data — ensure that art has maximum impact. That’s what growth marketers everywhere are learning.

Enabling it all?

Measurement.

“As a scientist, the only thing I believe in is experimentation,” Lyft Head of Marketing Science Alok Gupta said recently at UNIFY, Singular’s experts-only marketing technology conference. “I look at my current system, I perturb it through different creative and different levels of spend, and different bids, and new channels, and new partners, and I look what the effect is on the metric I care about.”

Driving the science is something that many might take for granted: measurement.

“The hype is real: measurement is important,” Gupta added. “Measurement is very important.”

Gupta, like many unique people with interesting careers, didn’t plan to be a marketing technologist or a marketing scientist. His original plan was to work in finance in the City in London, England. The bank-driven financial downturn of a decade ago delayed that plan, but eventually he found a home in high-frequency trading.

The corollaries with today’s programmatic real-time bidding data-driven marketing scene are obvious. Gupta, however, didn’t get deep into marketing and measurement until about four years ago at AirBnB.

And for his scientific mind, marketing measurement was problematic.

Very problematic.

“A lot of people were really excited about all these clever algorithms,” Gupta says. “Personalization, targeting, automated bidding, keyword expansion … and I kept saying to everyone: Yeah, but how do you know if it’s going to work?”

The answer he received — “we’ll just see if it goes up” — was unsatisfactory.

Gupta questioned: if what, exactly, goes up? The answer: conversions, bookings.

And his response: How do you know if it wouldn’t have gone up anyways?

The answer he received: I dunno.

“So I said, step back … let’s first figure out measurement” Gupta told the audience at UNIFY. “Once you have measurement, optimization is easy. Everyone can optimize. But measurement is really really hard. I’ve been devoting my last two or three years to measurement.”

That sounds like a lot of time, and it is.

There’s a reason why we see so little great work on thorny problems like incrementality and multi-touch attribution however. They’re just plain hard to solve, says Gupta, and that’s why so few marketers really use MTA and actually do solid incrementality testing.

“I think a core factor here is just how hard measurement is,” Gupta says. “I use the word ‘measurement’ to more broadly represent attribution, incrementality,  marketing mix modeling, LTV, ROI … it’s all coming back to the same thing. At the highest altitude, a company has many opportunities to invest. Marketing is one of those levers … they can invest in product, they can invest in partnership, they can invest in acquisition, they can invest in many things …. for them to be able invest sensibly, they want to know what their return is.”

Hence the experimentation.

Once Gupta has experimented, he has a ground truth to which he can fit multi-touch attribution, or a market mix model. That enables marketers to scale learning.

Experiments are tough, though. Each is very specific to the platform.

For AirBnB, if the growth marketing team brings on a new customer and she books a room, that’s good … but AirBnB has to consider the possibility that Jim, an existing customer, would have booked it anyways. It’s a supply-constrained market. Amazon, on the other hand, can happily sell the same book to both Jane and Jim — especially on Kindle but also in dead tree — and there’s always another one for the next customer: it’s not a supply-constrained market.

This is where marketing experiments get tough, Gupta says, and that’s why you need scientists and technical marketers.

Driving it all is accurate measurement. But how you do measurement will differ depending on your size and spend.

“If you’re spending $10 million plus, maybe you do want to invest in a data scientist, a data engineer, to get your data in good order from someone like Singular,” Gupta says. “And on the back of that build — marry it to more internal data — and build a more holistic conversion system.”

Over $100 million in annual marketing spend, and you probably need two or three data scientists, four or five engineers, and technical marketers.

In fact, kicking off a project to really get serious about measurement starts with marketers.

“You probably need someone … a marketer who cares deeply about measurement,” Gupta says. “To first have an advocate for doing this … then you bring a data engineer in, and then maybe a data scientist. But first of all, a strong technical marketer.”

How do you start?

Start small, Gupta says. Big projects are tough, expensive, and there’s no guarantee of success. Instead, take one partner who can do incrementality testing and run a small campaign. Generate 1,000 conversions, and compare that number to what your existing technology says. If your existing tech says you only got 500 conversions, then you know you need a better model to reflect ground truth.

Once you’ve started, it’ll be much easier to go to people who control budget and demonstrate the need — and the benefits — to going bigger with measurement.

The final critical factor: incentive alignment on the marketing team.

“Everyone on your team … their only goal is to drive the business metric,” says Gupta. “If that means one channel switches off, great, and another channels ramps up … great! That has to be the starting point. That has to be the mindset shift.”

Setting up the right goals is critical.

“If every 12 months your goal in their performance review is how much did you spend, and how many acquisitions did you bring in … that’s the wrong goaling,” says Gupta. “You have to goal them on how much efficiency did you unlock … how much opportunity did you unlock beyond the baseline?”

Now go deeper: Learn more about scientific marketing measurement