UniReps Workshop

Unifying Representations in Neural Models

Location: NeurIPS, Vancouver (Canada)

Date: December 14th, 2024


When, how and why do different neural models learn the same representations?


New findings in neuroscience and artificial intelligence reveal a shared pattern: whether in biological brains or artificial models, different learning systems tend to create similar representations when subject to similar stimuli.

The emergence of these similar representations is igniting a growing interest in the fields of neuroscience and artificial intelligence, with both fields offering promising directions for their theoretical understanding. These include analyzing the learning dynamics in neuroscience and studying the problem of identifiability in the functional and parameter space in artificial intelligence.

While the theoretical aspects already demand investigation, the practical applications are equally compelling: aligning representations allows for model merging, stitching and reuse, while also playing a crucial role in multi-modal scenarios. Furthermore, studying the features that are universally highlighted by different learning processes brings us closer to pinpoint the invariances that naturally emerge from learning models, possibly suggesting ways to enforce them.

The objective of the workshop is to discuss theoretical findings, empirical evidence and practical applications of this phenomenon, benefiting from the cross-pollination of different fields (ML, Neuroscience, Cognitive Science) to foster the exchange of ideas and encourage collaborations.

In conclusion, our primary focus is to delve into the underlying reasons, mechanisms, and extent of similarity in internal representations across distinct neural models, with the ultimate goal of unifying them into a single cohesive whole.

Speakers

Avatar

SueYeon Chung

Flatiron Institute / NYU

Avatar

Neel Nanda

Google DeepMind

Organizers

Avatar

Clementine Domine

University College London

Avatar

Donato Crisostomi

Sapienza, University of Rome

Avatar

Marco Fumero

Institute of Science and Technology Austria

Avatar

Zorah Lähner

University of Bonn

Avatar

Kim Stachenfeld

Google DeepMind

Program Chairs

Avatar

Irene Cannistraci

Sapienza, University of Rome

Avatar

Valentino Maiorca

Sapienza, University of Rome

Sponsors

Avatar
Avatar

Program Committee

Abhi Kamboj

University of Illinois at Urbana-Champaign

Aishwarya Gupta

Indian Institute of Technology, Kanpur

Ajay Subramanian

New York University

Akshata Kishore Moharir

Microsoft

Akshay Malhotra

Interdigital

Albert Manuel Orozco Camacho

Concordia University

Aldo Glielmo

Banca d’Italia

Alejandro Garca-Castellanos

University of Amsterdam

Aleksander Piotr Skorupa

University of Edinburgh

Alex H Williams

New York University

Alexander Huth

The University of Texas at Austin

Alice Bizeul

ETH Zurich

Alish Dipani

Georgia Institute of Technology

Ananya Passi

Johns Hopkins University

Andy T. Liu

ASUS

Anirudh Govil

International Institute of Information Technology Hyderabad

Anugunj Naman

Purdue University

Arif Dnmez

IUF Leibniz Research Institute for Environmental Medicine

Austin Meek

University of Delaware

Avisha Das

University of Houston

Ayyce Begm Bekta

Memorial Sloan Kettering Cancer Centre

Beatrix Miranda Ginn Nielsen

Technical University of Denmark

Berfin Inal

University of Amsterdam

Berivan Isik

Google

Berker Demirel

Institute of Science and Technology

Binxi Xie

Emory University

Biswarup Bhattacharya

Citadel

Bo Zhao

University of California, San Diego

Bogdan-Ionut Cirstea

Tlcom ParisTech

Bridget Leonard

University of Washington

Bruna Junqueira Lopes

École des Ponts ParisTech

Chandrasekhar Karnam

Hudson River Trading

Changqing Fu

Universit Paris-Dauphine (Paris IX)

Charles Camboulin

CY Cergy Paris Université

Chen Bo Calvin Zhang

UC Berkeley

Chenyu Wang

MIT

Chi-Ning Chou

Center of Computational Neuroscience, Flatiron Institute

Ching Fang

Columbia University

David Lipshutz

Flatiron Institute

Davit Soselia

University of Maryland, College Park

Denis A Gudovskiy

Panasonic Corp

Deval Mehta

Monash University

Dimitra Maoutsa

Technische Universität München

Edgar E Robles

University of California, Irvine

EHSAN KIAKOJOURI

Sapienza University of Rome

Elom Amematsro

Columbia University

Emilian Postolache

Ca’ Foscari University of Venice

Emily Cheng

Universitat Pompeu Fabra

Etienne Gay

Sorbonne Universit - Facult des Sciences (Paris VI)

Fabian Mager

Technical University of Denmark

Fahimeh Arab

University of California, Riverside

Gabor Lengyel

University of Rochester

Gasser Elbanna

Harvard University

geraldin nanfack

Concordia University

Hamed Karimi

Boston College

Hamidreza Jamalabadi

Philipps-Universität Marburg

Hanlin Yu

University of Helsinki

Hannah Small

Johns Hopkins University

Haoyu Zhang

The Chinese University of Hong Kong, Shenzhen

Harshay Shah

MIT

Huichi Zhou

Imperial College London

Hyewon Willow Han

University of Western Ontario

Inwoo Hwang

Seoul National University

Irene Tallini

Sapienza University of Rome

Itay Evron

Facebook

Jacob S. Prince

Harvard University

Jan Finkbeiner

Research Center Juelich

Jan Philipp Bauer

University College London, University of London

Jenelle Feather

Flatiron Institute

Jerry Ngo

MIT

Jiajing Chen

New York University

Jiancheng Pan

Tsinghua University

Jianyu Wu

Beijing University of Aeronautics and Astronautics

Jie Mei

IT:U Interdisciplinary Transformation University Austria

Jingxiao Tian

University of California, San Diego

Jinyung Hong

Arizona State University

Joo Abrantes

Sakana AI

Jrg Schltterer

Universität Mannheim

Juan Miguel Navarro Carranza

Stanford University

Juanxi Tian

University of Illinois at Urbana-Champaign

Kasper Vinken

Fujitsu Research of America

Keun Hee Park

Arizona State University

Konrad Karanowski

Technical University of Wroclaw

Konstantin Hemker

University of Cambridge

Krishna Sri Ipsit Mantri

Purdue University

Kusumakumari Vanteru

Westcliff University

Kyle Daruwalla

Cold Spring Harbor Laboratory

Leyla Isik

Johns Hopkins University

Lorenzo Basile

University of Trieste

Lorenzo Giusti

CERN

Luca Zhou

Sapienza University of Rome

Luigi Capogrosso

University of Verona

Luigi Gresele

University of Copenhagen

Luke McDermott

University of California, San Diego

Maciej Zieba

Wroclaw University of Science and Technology

Marco Pegoraro

Sapienza University of Rome

Martin Ester

Simon Fraser University

Maryam Hoseini Behbahani

Sharif University of Technology

Maryam Mirian

University of British Columbia

Maryam Mirian

University of British Columbia

Matteo Alleman

Columbia University

Matteo Ferrante

Università di Roma Tor Vergata

Maxime Di Folco

Helmholtz Zentrum München

Mohamed Shawky Sabae

Faculty of Engineering Cairo University, Cairo University

Mohammadamin Tavakoli

Caltech

Mohammed Adnan

University of Calgary / Vector Institute

Morteza Mahdiani

Université de Montréal

Moshe Eliasof

University of Cambridge

Mycal Tucker

MIT

Nanda H Krishna

Université de Montréal

Nicola Toschi

Università di Roma Tor Vergata

Nicolas Zilberstein

Rice University

Niharika S. D'Souza

International Business Machines

Nikolas McNeal

Georgia Institute of Technology

Omkar Joglekar

Bosch

Oriol Caudevilla

eBay Inc.

Osamu Hirose

Kanazawa University

Paris Giampouras

University of Warwick

Phuong Quynh Le

Philipps-Universität Marburg

Qiang Li

GSU

Qingqing Yang

Ohio State University, Columbus

Raghav Singhal

Mohamed bin Zayed University of Artificial Intelligence

Raja Kumar

CNRS

Riccardo Cadei

Institute of Science and Technology

Riccardo Renzulli

University of Turin

Richard Antonello

University of Texas, Austin

Rohan Jain

University of Calgary

Ruchira Dhar

University of Copenhagen

Rylan Schaeffer

Computer Science Department, Stanford University

Saaketh Medepalli

CMU

Sarah E Harvey

Flatiron Institute

Sayed Soroush Daftarian

Universitätsklinikum Marburg

Shashwat Singh

International Institute of Information Technology Hyderabad

Shiwen Zhang

Alibaba Group

Shreya Kapoor

Friedrich-Alexander-Universität Erlangen-Nürnberg

Shreya Kapoor

Friedrich-Alexander-Universität Erlangen-Nürnberg

Shubham Shukla

Nordstrom

Shuman Peng

Simon Fraser University

Siddhartha Gairola

Saarland Informatics Campus, Max-Planck Institute

Sining Huang

UC Berkeley

Stefan Horoi

Université de Montréal

Sujin Jeon

Seoul National University

Sukanya Moorthy

Credit Karma

Sukanya Moorthy

Credit Karma

Suklav Ghosh

Indian Institute of Technology, Guwahati

Tahmineh A. Koosha

Philipps-Universität Marburg

Tanmoy Mukherjee

Vrije Universiteit Brussel

Teresa Dorszewski

Technical University of Denmark

Thanh-Tung Nguyen

asus

Thibault Malherbe

Inetum

Thomas Edward Yerxa

New York University

Thu Bui

Purdue University

Tom White

Victoria University of Wellington

Udita Patel

Amazon

Valeria Ruscio

Sapienza University of Rome

Vatsala Nema

Indian Institute of Technology, Hyderabad, Dhirubhai Ambani Institute of Information and Communication Technology

Vighnesh Subramaniam

MIT

Vijay Prakash Dwivedi

Computer Science Department, Stanford University

Viktor Schlegel

Imperial College London

Wangjiaxuan Xin

University of North Carolina at Charlotte

Weisi Liu

University of Memphis

Weizhi Zhang

University of Illinois Chicago

William Yang

Flatiron Institute

Xiequn Wang

South University of Science and Technology of China

Xingyu Zheng

Cold Spring Harbor Laboratory

Xinyi Yang

Salesforce Research

Yalda Mohsenzadeh

University of Western Ontario

Yani Ioannou

University of Calgary

Ye Zhang

University of Pittsburgh

Yicheng Fu

Stanford University

Yixiao Kang

Facebook

Yixiao Yuan

Columbia University

Yulong Zhang

Zhejiang University

Yuxin Qiao

Northern Arizona University

Zaigham Abbas Randhawa

West Virginia University

Zhan Zhuang

City University of Hong Kong

Ziao Zhang

School of Engineering and Applied Sciences, Harvard University

Ziqing Yang

CISPA Helmholtz Center for Information Security

Ziyi Zhu

Deloitte Consulting

Acknowledgements

We would like to thank Riccardo Marin for serving as COI chair, and Fabian M. Mager and Andrea Santilli for their support during the event.