Advertising

Business Intelligence

TRIBE v2: AI Model That Predicts Your Brain

Table of Contents

Imagine an AI system that watches you watch a movie, listens to what you hear, and then predicts exactly what your brain is doing—in real time, across thousands of cortical regions. That’s no longer science fiction. Meta’s researchers just released TRIBE v2, a tri-modal foundation model trained on over 1,000 hours of fMRI data that can predict human brain activity from video, audio, and language input with stunning accuracy.

This isn’t incremental progress. It’s a paradigm shift that’s opening doors to in-silico neuroscience experiments, better experimental design, and a unified computational model of human cognition. Here’s what you need to know.

What Is TRIBE v2?

TRIBE v2 (Tri-modal Brain Encoder v2) is a transformer-based neural network developed by researchers at Meta FAIR and École Normale Supérieure. It takes multimodal stimuli—videos, audio tracks, and text transcripts—and predicts the corresponding fMRI brain responses across the entire brain.

The numbers are impressive: trained on data from 720 subjects across eight different datasets, TRIBE v2 achieves several-fold improvements over traditional linear encoding models. But the real breakthrough isn’t just raw accuracy—it’s generalization. The model can predict brain responses for completely new subjects without retraining, and it works across naturalistic stimuli (movies, podcasts) and controlled experimental paradigms alike.

The Three Modalities That Make It Work

TRIBE v2 combines three independent feature extractors:

Text embeddings come from Llama-3.2-3B, processing word-by-word context from video transcripts. Audio embeddings use Wav2Vec-Bert-2.0 to extract 50 Hz representations, resampled to 2 Hz to align with visual and textual inputs. Video embeddings leverage Video-JEPA-2-Giant, a self-supervised vision model that processes 64-frame windows spanning 4 seconds of video.

These three streams are then concatenated and fed through a transformer encoder with 8 layers and 8 attention heads. A critical innovation: modality dropout. During training, each modality is randomly masked with 30% probability, forcing the model to work gracefully with incomplete inputs—think silent movies or podcasts without visuals.

Why This Matters: The Death of Task-Specific Models

For decades, neuroscience operated like a fragmented empire. Researchers built separate models for faces, for places, for language, for motion. Each model was specialized, each required meticulous feature engineering, and none of them talked to each other.

TRIBE v2 dismantles this. A single architecture now captures whole-brain responses across dozens of cognitive domains. No hand-crafted features. No task-specific tuning. Just raw AI representations aligned with brain geometry.

Here’s the kicker: TRIBE v2 recovers classical neuroscience findings that took decades to establish through traditional experiments. When tested on the Individual Brain Charting (IBC) dataset’s visual localizers, it correctly identified the Fusiform Face Area (FFA) for faces, the Parahippocampal Place Area (PPA) for places, and the Visual Word-Form Area (VWFA) for text. Same story with language experiments—it isolated the temporo-parietal junction (TPJ) for emotional processing and correctly predicted left-hemisphere lateralization for syntactic complexity.

This is the gold standard of validation: your new model doesn’t just predict novel brain responses—it recovers known neuroscience.

The Real Innovation: In-Silico Experimentation

The most powerful application isn’t prediction—it’s hypothesis testing without subjects. Want to test how the brain responds to a new visual stimulus? Feed it to TRIBE v2. Want to pilot a complex fMRI protocol before scanning expensive 7T machines? Let the model tell you if it’s worth the investment.

This flips neuroscience economics. Instead of recruiting subjects, booking scanner time, and waiting months for data, researchers can now do rapid prototyping in silico. The model becomes a “digital twin” of human brain function.

Meta demonstrated this by replicating seminal experiments from 60+ years of vision and language neuroscience. Every major finding checked out. The model correctly predicted where selectivity for faces, places, bodies, and text emerges in visual cortex. It recovered classical language network organization. It even showed the expected emotional-processing response when contrasting emotional versus physical pain sentences.

Multimodal Integration: The Brain’s Secret Sauce

One section deserves its own deep dive: how the brain integrates information across modalities. TRIBE v2 reveals this via ablation studies. When trained on a single modality alone, performance tanks. But certain brain regions show massive gains from multimodality.

The temporal-parietal-occipital junction (TPJ)—a region implicated in integrating visual, auditory, and semantic information—shows up to 50% improvement when multiple modalities are combined. The prefrontal cortex also benefits. These aren’t arbitrary findings; they align perfectly with decades of neuroscience literature on multisensory integration.

Using RGB color-coding (red for text, green for audio, blue for video), the paper reveals which brain regions are driven by which modalities. Yellow (text + audio) appears in superior temporal regions involved in speech. Cyan (video + audio) shows up in visual and hippocampal areas. Different subjects show subtle but consistent variations in this spatial organization—individual differences that matter.

Performance at Scale

TRIBE v2 achieved first place in the Algonauts 2025 competition (out of 263 teams). But more importantly, it shows log-linear scaling: accuracy improves consistently with more training data, with no plateau in sight. This means the ceiling for brain encoding hasn’t been reached. Scale the data further, and prediction accuracy will keep climbing.

The model was trained in 24 hours on 128 V100 GPUs, with cached embeddings pre-computed to accelerate training. That’s accessible to well-funded labs, not just megacorp research centers.

The Limitations (And Why They Don’t Matter Yet)

TRIBE v2 can’t capture millisecond-level neuronal dynamics (fMRI is too slow). It doesn’t model the brain as an active agent—only as a passive observer. It’s missing whole sensory modalities: olfaction, balance, somatosensation. And it treats brain function as static, not accounting for development, aging, or clinical pathology.

These are real constraints. But they’re not deal-breakers—they’re roadmaps. The architecture is proven. The scaling laws work. The next versions will address these gaps.

What This Means for Developers and Researchers

If you work in neuroscience, this is now your baseline. If you build AI models for understanding human behavior, TRIBE v2 shows what’s possible when you align your representations with brain geometry. If you’re exploring whether your model architecture captures human cognition, you now have a gold-standard benchmark.

The code is open-sourced on GitHub. The weights are on Hugging Face. Meta released an interactive demo where you can feed any video, podcast, or text and watch real-time brain predictions unfold.

Frequently Asked Questions

What’s the difference between TRIBE v2 and previous brain encoding models?

Traditional models use linear algebra (ridge regression) to map AI representations onto brain responses, assuming a straight-line relationship. TRIBE v2 is nonlinear—it uses a deep transformer to learn complex interactions between modalities. It’s also trained on 25 subjects across four different tasks, not just one subject on one task. Result: several-fold accuracy improvements.

Can TRIBE v2 read minds?

No. TRIBE v2 predicts *aggregate* fMRI activity patterns in response to stimuli you’re *consciously experiencing*. It doesn’t decode thoughts, intentions, or mental imagery. It can’t tell what you’re thinking—only what your brain does when exposed to videos, audio, and text it’s been trained on.

Do I need an fMRI scan for TRIBE v2 to work?

No. The model is trained on fMRI data, but you only need that data if you’re *training* a new model or validating predictions. To use TRIBE v2 for in-silico experimentation, you just feed it a video or audio file—no brain scans required.

Can I use TRIBE v2 to improve my own AI model?

Absolutely. TRIBE v2 shows which representational structures in neural networks best align with biological brains. If you’re building models for human-centric tasks (translation, summarization, content recommendation), aligning your representations with TRIBE v2’s brain alignments could improve both accuracy and human-like behavior.

How accurate are the predictions really?

On naturalistic stimuli (movies, podcasts), TRIBE v2 achieves Pearson correlations of 0.2–0.4 between predicted and actual brain responses, depending on the brain region. Visual cortex predictions are stronger (~0.4). Subcortical regions are weaker (~0.1). This vastly exceeds previous linear models, which topped out around 0.15.

What does “in-silico” mean?

Latin for “in silicon”—essentially, run experiments on a computer instead of with real subjects. You propose an experiment, feed the stimulus to TRIBE v2, get back predicted brain responses, and validate your hypothesis without recruiting, scanning, or paying participants.

Is the code and models free to use?

Yes. Both the code (GitHub) and pre-trained weights (Hugging Face) are open-source. You can download, fine-tune, and use TRIBE v2 for research and development at no cost. The interactive demo is free to explore with any video or audio file.

Key Takeaways

TRIBE v2 predicts whole-brain fMRI responses from multimodal stimuli, achieving several-fold accuracy gains over traditional models
A single architecture generalizes across dozens of cognitive domains—no task-specific retuning required
The model recovers classical neuroscience findings, validating its internal representations
In-silico experimentation enables rapid hypothesis testing without recruiting subjects
Multimodal integration reveals how the brain combines information across sensory modalities
Log-linear scaling suggests prediction accuracy will keep improving with more data
The code and weights are open-sourced and publicly available

Start Exploring

The interactive demo at aidemos.atmeta.com/tribev2 lets you upload video segments and see predictions in real time. The full codebase is on GitHub, and pre-trained weights are available on Hugging Face.

If you’re building AI models or working in neuroscience, spend an afternoon with this. It’s a watershed moment for understanding how human brains encode the world.