---
OA_place: repository
_id: '15039'
abstract:
- lang: eng
  text: 'A crucial property for achieving secure, trustworthy and interpretable deep
    learning systems is their robustness: small changes to a system''s inputs should
    not result in large changes to its outputs. Mathematically, this means one strives
    for networks with a small Lipschitz constant. Several recent works have focused
    on how to construct such Lipschitz networks, typically by imposing constraints
    on the weight matrices. In this work, we study an orthogonal aspect, namely the
    role of the activation function. We show that commonly used activation functions,
    such as MaxMin, as well as all piece-wise linear ones with two segments unnecessarily
    restrict the class of representable functions, even in the simplest one-dimensional
    setting. We furthermore introduce the new N-activation function that is provably
    more expressive than currently popular activation functions. We provide code at
    this https URL.'
article_number: '2311.06103'
article_processing_charge: No
arxiv: 1
author:
- first_name: Bernd
  full_name: Prach, Bernd
  id: 2D561D42-C427-11E9-89B4-9C1AE6697425
  last_name: Prach
- first_name: Christoph
  full_name: Lampert, Christoph
  id: 40C20FD2-F248-11E8-B48F-1D18A9856A87
  last_name: Lampert
  orcid: 0000-0001-8622-7887
citation:
  ama: Prach B, Lampert C. 1-Lipschitz neural networks are more expressive with N-activations.
    <i>arXiv</i>. doi:<a href="https://doi.org/10.48550/ARXIV.2311.06103">10.48550/ARXIV.2311.06103</a>
  apa: Prach, B., &#38; Lampert, C. (n.d.). 1-Lipschitz neural networks are more expressive
    with N-activations. <i>arXiv</i>. <a href="https://doi.org/10.48550/ARXIV.2311.06103">https://doi.org/10.48550/ARXIV.2311.06103</a>
  chicago: Prach, Bernd, and Christoph Lampert. “1-Lipschitz Neural Networks Are More
    Expressive with N-Activations.” <i>ArXiv</i>, n.d. <a href="https://doi.org/10.48550/ARXIV.2311.06103">https://doi.org/10.48550/ARXIV.2311.06103</a>.
  ieee: B. Prach and C. Lampert, “1-Lipschitz neural networks are more expressive
    with N-activations,” <i>arXiv</i>. .
  ista: Prach B, Lampert C. 1-Lipschitz neural networks are more expressive with N-activations.
    arXiv, 2311.06103.
  mla: Prach, Bernd, and Christoph Lampert. “1-Lipschitz Neural Networks Are More
    Expressive with N-Activations.” <i>ArXiv</i>, 2311.06103, doi:<a href="https://doi.org/10.48550/ARXIV.2311.06103">10.48550/ARXIV.2311.06103</a>.
  short: B. Prach, C. Lampert, ArXiv (n.d.).
corr_author: '1'
date_created: 2024-02-28T17:59:32Z
date_published: 2023-11-10T00:00:00Z
date_updated: 2026-04-07T11:49:51Z
day: '10'
department:
- _id: GradSch
- _id: ChLa
doi: 10.48550/ARXIV.2311.06103
external_id:
  arxiv:
  - '2311.06103'
language:
- iso: eng
license: https://creativecommons.org/licenses/by/4.0/
main_file_link:
- open_access: '1'
  url: https://doi.org/10.48550/arXiv.2311.06103
month: '11'
oa: 1
oa_version: Preprint
publication: arXiv
publication_status: draft
related_material:
  record:
  - id: '19759'
    relation: dissertation_contains
    status: public
status: public
title: 1-Lipschitz neural networks are more expressive with N-activations
tmp:
  image: /images/cc_by.png
  legal_code_url: https://creativecommons.org/licenses/by/4.0/legalcode
  name: Creative Commons Attribution 4.0 International Public License (CC-BY 4.0)
  short: CC BY (4.0)
type: preprint
user_id: 8b945eb4-e2f2-11eb-945a-df72226e66a9
year: '2023'
...