Privacy Sandbox Simulator: get a sneak peek NOW at the data you’ll get

Welcome to Privacy Sandbox Simulator, the way to know now what kind of data you’ll get once Google flips a switch and most Android devices globally start using Privacy Sandbox for attribution and install analytics.

Mobile marketers have a ton of questions about Privacy Sandbox.

And there aren’t many answers.

• What mobile attribution data will Privacy Sandbox give you?
• How many in-app events will you be able to track when using Privacy Sandbox?
• What kinds of campaign, creative, and geo breakdowns will Privacy Sandbox allow that will help you analyze your marketing measurement data? 

Few really know, unless you’ve started testing Privacy Sandbox. But now you can get a sneak peek, thanks to Singular’s new Privacy Sandbox Simulator.

Get access immediately right here.

Or, watch Singular CTO Eran Friedman and I play around with varying models for different verticals, multiple monetization methods, and both high and low scale. Each will reveal the data Privacy Sandbox will return, with an error margin or confidence interval. Watching this will give you a good sense about how to start prepping conversion models for your Android apps that will work under Privacy Sandbox, in much the same way as you’ve had to build conversion models under SKAdNetwork for iOS apps.

Privacy Sandbox Simulator is a noise calculator

The first thing you become aware of when you check out Privacy Sandbox is that it is amazingly customizable. Depending on what you want in terms of granularity, events, and attribution windows, you can customize much more than you can in SKAN or AdAttributionKit.

But that power has a cost.

And there’s a very important choice to be made. 

You can …

1. Track a large number of events (and get limited fidelity)
2. Or, get extremely precise data (about a limited number of events)

The more events you want to track, the more noise Privacy Sandbox will add to your data. At low volumes, that noise will obliterate the signal: meaning you get little useful information out of Privacy Sandbox. You can think of Privacy Sandbox’s noise as similar to SKAdNetwork’s privacy thresholds and AdAttributionKit’s crowd anonymity.

A good way to test how much noise you’re going to get is to use Singular’s Privacy Sandbox Simulator.

How the Simulator works

First, navigate to the Simulator home page and get access. Note: there is a sign-up process.

1. App Volume
   Enter your estimated App Volume: how many installs your app typically gets per day
2. Allocate tracking budget
   Decide where to allocate your event and revenue tracking “budget” between installs, events within 7 days, and revenue within 7 days
   1. Higher percentages result in better measurement accuracy
   2. Note, however, that under Privacy Sandbox, installs are “cheap” to measure (allocating just 20% or even 10% with high volume will often be sufficient to get relatively noise-free data)
3. Pick data breakdowns
   Decide what data breakdowns you want to be able to slice and dice your event and revenue by

 

 

Learning from the Privacy Sandbox Simulator

Play with the Simulator to see how different emphases on events and data breakdowns work under Privacy Sandbox. For example, with …

• Low volume
• Focus on events and revenue
• Breakdowns for source and campaign

… you can see that small campaigns with large campaigns with large ad networks will work pretty well, but smaller campaigns with small ad networks will have a few issues:

• 5% error margin for installs
• 12% error margin for events
• 142% error margin for revenue

 

 

The same is true for Source and Campaign.

Where the confidence interval is tight, the Privacy Sandbox Simulator shows results in green. Where the error margin is greater, results show up in orange. And where the results are nowhere near good enough for useful marketing analytics, results show up in red.

 

 

Boost the daily installs to 10,000, however, and you’ll see what increased volume does. Google is still adding noise, but the impact of the noise as a percentage of the overall signal is significantly reduced.

 

 

What this tells us about Privacy Sandbox (scale matters)

What the Privacy Sandbox Simulator clearly reveals is that Privacy Sandbox is very much like SKAdNetwork and AdAttributionKit, or AAK in at least 1 way.

Scale is key.

The early months of SKAN 3 were chaotic for many. They weren’t getting sufficient attribution signal to guide their campaign and partner optimization.

Clearly, Privacy Sandbox has the potential to be very similar. There is a critical mass of installs that you have to achieve in order to maximize attribution and analytics.

Small networks and small campaigns on large networks will be problematic. Large publishers with large budgets and high daily install counts will be basically fine as long as they concentrate campaign spend sufficiently. Smaller advertisers or large advertisers that are used to running hundreds or thousands of small campaigns will need to adjust what they’re doing in order to maximize data harvesting.

And yes, ad networks will need to adjust as well, especially those who create sub campaigns non-transparently to test targeting and optimize conversions.

 

 

At high scale, like this app that gets 100,000 app installs per day, you get pretty much all the data you want. You can increase the number of events you want to track, and you can add sophisticated data breakdowns to deeply analyze the data you get back. 

Even your smaller campaigns return actually fairly usable data. The error margins are not entirely horrific.

Watch the whole video above for more insight

Check out the entire Growth Masterminds episode above for more insight that might be more relevant to your specific app, monetization model, and vertical.

For example, Friedman and I go through a variety of scenarios:

• High scale
• Low scale
• Subscription apps with free trials
• An indie game with IAPs
• Casual games with ad monetization at varying volumes

Once you’ve done that, play with the simulator right here. It should give you a good sense of what to expect under Privacy Sandbox.
Oh, and if you want to go to the next level and actually start testing Privacy Sandbox, we’ve got your back there too. Check out what’s happening, and talk to your Singular representative about beginning a test yourself.