Publications

2026

1. Does Aurora Encode Atmospheric Structure? Latent Regime Analysis and Attribution

   Emma Kasteleyn and Ana Lucic

   ICLR Workshop on Scientific Methods for Understanding Deep Learning, 2026

   Abs PDF

   ML foundation models are able to emulate atmospheric dynamics accurately and efficiently but operate as opaque "black boxes". We investigate the internal representations of the Aurora model using spatially pooled PCA and layer-wise relevance propagation (LRP). We find evidence that Aurora’s latent space is primarily organized by seasonal cycles, whereas extreme storm events do not form a linearly separable cluster. LRP indicates that the model attends to features consistent with the 3D vertical structure of the Great Storm of 1987. Perturbation tests show masking relevant regions degrades forecasts more than random masking. These findings suggest that Aurora learns meteorological coherence and vertical structure without explicit instruction.

2025

1. Group Equivariance Meets Mechanistic Interpretability: Equivariant Sparse Autoencoders

   Ege Erdogan and Ana Lucic

   NeurIPS Workshop on Mechanistic Interpretability, 2025

   Abs PDF

   Sparse autoencoders (SAEs) have proven useful in disentangling the opaque activations of neural networks, primarily large language models, into sets of interpretable features. However, adapting them to domains beyond language, such as scientific data with group symmetries, introduces challenges that can hinder their effectiveness. We show that incorporating such group symmetries into the SAEs yields features more useful in downstream tasks. More specifically, we train autoencoders on synthetic images and find that a single matrix can explain how their activations transform as the images are rotated. Building on this, we develop adaptively equivariant SAEs that can adapt to the base model’s level of equivariance. These adaptive SAEs discover features that lead to superior probing performance compared to regular SAEs, demonstrating the value of incorporating symmetries in mechanistic interpretability tools.

2. Three Desiderata for Faithfulness in Machine Learning Explanations: The Case for Causal Abstraction

   Mette Friis Andersen, Maria Heuss, and Ana Lucic

   NeurIPS Workshop on Mechanistic Interpretability, 2025

   Abs PDF

   Faithfulness is a broadly agreed-upon desideratum for explanations of machine learning model predictions. While many different methods have been adopted by the community, there is no agreed-upon definition of faithfulness. Here, we propose three desiderata for faithfulness beyond the standard intuition of accurately representing the reasoning process of the model, related to (1) enabling reverseengineering of specific behaviors, (2) capturing interventionist causal relations, and (3) achieving an appropriate model decomposition. We argue that causal abstraction satisfies these, and provides a framework for evaluating faithfulness claims in the community.