---
res:
  bibo_abstract:
  - "Recent work has seen the development of general purpose neural architectures\r\nthat
    can be trained to perform tasks across diverse data modalities. General\r\npurpose
    models typically make few assumptions about the underlying\r\ndata-structure and
    are known to perform well in the large-data regime. At the\r\nsame time, there
    has been growing interest in modular neural architectures that\r\nrepresent the
    data using sparsely interacting modules. These models can be more\r\nrobust out-of-distribution,
    computationally efficient, and capable of\r\nsample-efficient adaptation to new
    data. However, they tend to make\r\ndomain-specific assumptions about the data,
    and present challenges in how\r\nmodule behavior (i.e., parameterization) and
    connectivity (i.e., their layout)\r\ncan be jointly learned. In this work, we
    introduce a general purpose, yet\r\nmodular neural architecture called Neural
    Attentive Circuits (NACs) that\r\njointly learns the parameterization and a sparse
    connectivity of neural modules\r\nwithout using domain knowledge. NACs are best
    understood as the combination of\r\ntwo systems that are jointly trained end-to-end:
    one that determines the module\r\nconfiguration and the other that executes it
    on an input. We demonstrate\r\nqualitatively that NACs learn diverse and meaningful
    module configurations on\r\nthe NLVR2 dataset without additional supervision.
    Quantitatively, we show that\r\nby incorporating modularity in this way, NACs
    improve upon a strong non-modular\r\nbaseline in terms of low-shot adaptation
    on CIFAR and CUBs dataset by about\r\n10%, and OOD robustness on Tiny ImageNet-R
    by about 2.5%. Further, we find that\r\nNACs can achieve an 8x speedup at inference
    time while losing less than 3%\r\nperformance. Finally, we find NACs to yield
    competitive results on diverse data\r\nmodalities spanning point-cloud classification,
    symbolic processing and\r\ntext-classification from ASCII bytes, thereby confirming
    its general purpose\r\nnature.@eng"
  bibo_authorlist:
  - foaf_Person:
      foaf_givenName: Nasim
      foaf_name: Rahaman, Nasim
      foaf_surname: Rahaman
  - foaf_Person:
      foaf_givenName: Martin
      foaf_name: Weiss, Martin
      foaf_surname: Weiss
  - foaf_Person:
      foaf_givenName: Francesco
      foaf_name: Locatello, Francesco
      foaf_surname: Locatello
      foaf_workInfoHomepage: http://www.librecat.org/personId=26cfd52f-2483-11ee-8040-88983bcc06d4
    orcid: 0000-0002-4850-0683
  - foaf_Person:
      foaf_givenName: Chris
      foaf_name: Pal, Chris
      foaf_surname: Pal
  - foaf_Person:
      foaf_givenName: Yoshua
      foaf_name: Bengio, Yoshua
      foaf_surname: Bengio
  - foaf_Person:
      foaf_givenName: Bernhard
      foaf_name: Schölkopf, Bernhard
      foaf_surname: Schölkopf
  - foaf_Person:
      foaf_givenName: Li Erran
      foaf_name: Li, Li Erran
      foaf_surname: Li
  - foaf_Person:
      foaf_givenName: Nicolas
      foaf_name: Ballas, Nicolas
      foaf_surname: Ballas
  bibo_volume: 35
  dct_date: 2022^xs_gYear
  dct_language: eng
  dct_title: Neural attentive circuits@
...