UniReps Workshop

Unifying Representations in Neural Models

Location: NeurIPS, Vancouver (Canada)

Date: December 14th, 2024


When, how and why do different neural models learn the same representations?


New findings in neuroscience and artificial intelligence reveal a shared pattern: whether in biological brains or artificial models, different learning systems tend to create similar representations when subject to similar stimuli.

The emergence of these similar representations is igniting a growing interest in the fields of neuroscience and artificial intelligence, with both fields offering promising directions for their theoretical understanding. These include analyzing the learning dynamics in neuroscience and studying the problem of identifiability in the functional and parameter space in artificial intelligence.

While the theoretical aspects already demand investigation, the practical applications are equally compelling: aligning representations allows for model merging, stitching and reuse, while also playing a crucial role in multi-modal scenarios. Furthermore, studying the features that are universally highlighted by different learning processes brings us closer to pinpoint the invariances that naturally emerge from learning models, possibly suggesting ways to enforce them.

The objective of the workshop is to discuss theoretical findings, empirical evidence and practical applications of this phenomenon, benefiting from the cross-pollination of different fields (ML, Neuroscience, Cognitive Science) to foster the exchange of ideas and encourage collaborations.

In conclusion, our primary focus is to delve into the underlying reasons, mechanisms, and extent of similarity in internal representations across distinct neural models, with the ultimate goal of unifying them into a single cohesive whole.

Speakers

Avatar

SueYeon Chung

Flatiron Institute / NYU

Avatar

Neel Nanda

Google DeepMind

Organizers

Avatar

Clementine Domine

University College London

Avatar

Donato Crisostomi

Sapienza, University of Rome

Avatar

Marco Fumero

Institute of Science and Technology Austria

Avatar

Zorah Lähner

University of Bonn

Avatar

Kim Stachenfeld

Google DeepMind

Program Chairs

Avatar

Irene Cannistraci

Sapienza, University of Rome

Avatar

Valentino Maiorca

Sapienza, University of Rome

Sponsors

Avatar
Avatar

Acknowledgements

We would like to thank Riccardo Marin for serving as COI chair, and Fabian M. Mager and Andrea Santilli for their support during the event.