A brain model, not a survey

fMRI is a brain scan that tracks activity across the whole brain, moment by moment, while a person does something: in our case, while they watched hours of films and television. From those recordings we trained a model that learned the link between what is on screen and in the soundtrack, and how each part of the brain responds.

Once trained, the model is frozen. Give it a new video it has never seen and it predicts, for each of 1,000 small patches of the brain's surface, how strongly that patch would respond, every 1.5 seconds of film. Nobody is recruited, nobody fills in a survey, nobody goes near a scanner: the model reads the video file directly.

What the numbers mean

Every brain region has a normal level of activity during ordinary viewing: think of it as that region's hum. The numbers in a readout measure distance from that hum. 0 means nothing unusual. Positive means the ad is predicted to drive that region harder than normal film watching does; negative means quieter.

The unit (statisticians call it a z score) is that region's own normal variation. So +1 is noticeably above normal, +2 is rare territory, and a few tenths either way means nothing: treat small differences as noise. One honest detail: "normal" here is normal film and TV viewing, because that is what the model was trained on. It is not an average of other ads.

How we know the predictions are any good

We test the model the hard way: on a person it never trained on, watching videos it never saw. That is the same situation your ad is in, so the test tells you what to expect. The accuracy measure is r, the match between what the model predicted and what the scanner actually recorded: 1.0 would be a perfect match, 0 means no relationship at all.

Accuracy is not uniform across the brain. Our single best-predicted region is a small patch of auditory cortex: the model's predicted activity curve for it matches the actually recorded curve at r 0.64, moment by moment, in a person the model never met. That region sits inside our Sound signal, which is defined as the hearing cortex (auditory plus temporoparietal regions) and averages r 0.38, our most accurate signal. Sight follows at 0.27, Story at 0.23 and Attention at 0.20. The same discipline applies inside every signal: Story, for example, uses only the 77 best-predicted regions of the default-mode network; the network's weakly predicted remainder is excluded rather than averaged in.

Some parts of the brain we predict much worse, and here is our rule: if accuracy is below r 0.15, we do not show it at all. Motor and touch cortex (r 0.09: viewers lie still in a scanner, so there is little to predict), the weaker part of the default-mode network, the salience and frontoparietal control networks, and limbic cortex all fail that bar: 574 of the 1,000 regions, drawn permanently gray on the brain map. We report the 426 we can stand behind. At that accuracy a prediction overlaps with reality by less than 2%: showing it would look like information while being mostly noise.

Fast signals and slow signals. Sight, Sound and Attention respond within seconds, so they are read moment by moment. The story system integrates a narrative over minutes: its value at any instant reflects everything that came before, so Story is read over the whole film instead (Hasson et al. 2008, J Neurosci).

Two more honest limits. First, there is a ceiling nobody can beat: the same person watching the same ad twice does not produce the same brain response, and no model or scanner can predict the part that never repeats. Second, the scan data covers the brain's outer surface; deep structures such as the reward circuitry are not measured, so we do not score them.

Photos use a different model

Still images are scored by a separate model that reports four brain regions chosen because science understands them well: one that responds to objects (does the product read as a distinct thing?) and three that respond to places (does the image read as a real environment?). On the place regions it captures 73.9% of the signal that is possible to capture, our most accurate prediction anywhere.

A fair question about any image tool: is it secretly just measuring brightness or clutter? We checked. Simple pixel statistics predict the place-region response at r 0.03, essentially zero. Whatever these regions read in an image, it is not recoverable from pixel math. Video and photo numbers are not on one scale and should not be compared to each other.

Every claim about the brain carries its source

The model produces numbers. Saying what a brain region does is a different kind of claim, so we never improvise it: every such sentence in this product is pulled from a fixed set of published, peer-reviewed brain-imaging studies, and shown with its source (Epstein & Kanwisher 1998, Nature; Corbetta & Shulman 2002, Nat Rev Neurosci; Raichle et al. 2001, PNAS; among others).

And one tempting move we refuse on principle: "this region lit up, so the viewer must have felt X." Reading thoughts or feelings off a brain activation sounds scientific but is known to be unreliable; the field calls it reverse inference (Poldrack 2006, Trends Cogn Sci). When you ask the assistant what someone felt or will remember, it will tell you that is beyond the data.

How to use a readout

Open an asset. For video, press play: the film runs in step with the brain map, and pausing anywhere gives you each signal's predicted level at that exact moment. Below, every signal carries its film average and its arc across four quarters. The written read of those numbers — what carries the film, where it slips, and what to do about it — is checked by a person against the underlying data before we release it, so you request it rather than being handed it automatically.