V-116
Accuracy Is Not Enough: Divergent Brain–Model Similarities During Learning and Perturbation
Eric Lützow Holm1,2, Diego Fernández Slezak1,3,4, Enzo Tagliazucchi1,2,5
  1. Consejo Nacional de Investigaciones Científicas y Técnicas (CONICET), CABA, Argentina
  2. Instituto de Física Interdisciplinaria y Aplicada, Departamento de Física, UBA, CABA, Argentina
  3. Departamento de Computación, FCEyN, UBA, CABA, Argentina
  4. Instituto de Investigación en Ciencias de la Computación (ICC), CONICET–UBA, CABA, Argentina
  5. Latin American Brain Health (BrainLat), Universidad Adolfo Ibáñez, Santiago, Chile
Presenting Author:
Eric Lützow Holm
elholm90@gmail.com
Deep neural networks (DNNs) are increasingly used as computational models of human visual processing, yet it remains unclear how their similarity to neural activity depends on the processes of learning and perturbation. In this project, we examine the evolving relationship between DNN representations and human brain activity during object recognition. Using EEG recordings of participants viewing briefly presented stimuli, we compare representational dissimilarity matrices (RDMs) derived from EEG with those extracted from networks such as ResNet-50 and AlexNet trained from scratch on ImageNet. We track similarity across training epochs, relating representational alignment to model accuracy. In parallel, we probe the robustness of these similarities by perturbing trained networks with multiplicative Gaussian noise in different layers, thereby altering their internal feature representations while monitoring the consequences for both accuracy and brain–model alignment. A central hypothesis is that the trajectory of similarity with respect to accuracy differs between the learning and perturbation regimes, pointing to distinct mechanisms underlying representational convergence and representational stability. This work aims to clarify whether training-driven and perturbation-driven changes in DNNs offer comparable insights into human vision, thereby informing the use of artificial networks as neuroscience models.