---
OA_place: publisher
_id: '19759'
abstract:
- lang: eng
  text: "Despite generating remarkable results in various computer vision tasks, deep
    learning comes\r\nwith some surprising shortcomings. For example, tiny perturbations,
    often imperceptible to\r\nthe human eye, can completely change the predictions
    of image classifiers. Despite a decade\r\nof research, the field has made limited
    progress in developing image classifiers that are both\r\naccurate and robust.
    This thesis aims to address this gap.\r\nAs our first contribution, we aim to
    simplify the process of training certifiably robust image\r\nclassifiers. We do
    this by designing a convolutional layer that does not require executing an\r\niterative
    procedure in every forward pass, but relies on an explicit bound instead. We also\r\npropose
    a loss function that allows optimizing for a particular margin more precisely.\r\nNext,
    we provide an overview and comparison of various methods that create robust image\r\nclassifiers
    by constraining the Lipschitz constant. This is important since generally longer\r\ntraining
    times and more parameters improve the performance of robust classifiers, making
    it\r\nchallenging to determine the most practical and effective methods from existing
    literature.\r\nIn 1-Lipschitz classification, the performance of current methods
    is still much worse than what\r\nwe expect on the simple tasks we consider. Therefore,
    we next investigate potential causes of\r\nthis shortcoming. We first consider
    the role of the activation function. We prove a theoretical\r\nshortcoming of
    the commonly used activation function, and provide an alternative without it.\r\nHowever
    this theoretical improvement does barely translate to the empirical performance
    of\r\nrobust classifiers, suggesting a different bottleneck.\r\nTherefore, in
    the final chapter, we study how the performance depends on the amount of\r\ntraining
    data. We prove that in the worst case, we might require far more data to train
    a\r\nrobust classifier compared to a normal one. We furthermore find that the
    amount of training\r\ndata is a key determinant of the performance current methods
    achieve on popular datasets.\r\nAdditionally, we show that linear subspaces exist
    with tiny data variance, and yet we can\r\nstill train very accurate classifiers
    after projecting into those subspaces. This shows that on\r\nthe datasets considered,
    enforcing robustness in classification makes the task strictly more\r\nchallenging.\r\n\r\n-----------------“In
    reference to IEEE copyrighted material which is used with permission in this thesis,
    the IEEE does not endorse any of [name of university or educational entity]’s
    products or services. Internal or personal use of this material is permitted.
    If interested in reprinting/republishing IEEE copyrighted material for advertising
    or promotional purposes or for creating new collective works for resale or redistribution,
    please go to http://www.ieee.org/publications_standards/publications/rights/rights_link.html
    to learn how to obtain a License from RightsLink. If applicable, University Microfilms
    and/or ProQuest Library, or the Archives of Canada may supply single copies of
    the dissertation.”\r\n"
alternative_title:
- ISTA Thesis
article_processing_charge: No
author:
- first_name: Bernd
  full_name: Prach, Bernd
  id: 2D561D42-C427-11E9-89B4-9C1AE6697425
  last_name: Prach
citation:
  ama: Prach B. Robust image classification with 1-Lipschitz networks. 2025. doi:<a
    href="https://doi.org/10.15479/10.15479/at-ista-19759">10.15479/10.15479/at-ista-19759</a>
  apa: Prach, B. (2025). <i>Robust image classification with 1-Lipschitz networks</i>.
    Institute of Science and Technology Austria. <a href="https://doi.org/10.15479/10.15479/at-ista-19759">https://doi.org/10.15479/10.15479/at-ista-19759</a>
  chicago: Prach, Bernd. “Robust Image Classification with 1-Lipschitz Networks.”
    Institute of Science and Technology Austria, 2025. <a href="https://doi.org/10.15479/10.15479/at-ista-19759">https://doi.org/10.15479/10.15479/at-ista-19759</a>.
  ieee: B. Prach, “Robust image classification with 1-Lipschitz networks,” Institute
    of Science and Technology Austria, 2025.
  ista: Prach B. 2025. Robust image classification with 1-Lipschitz networks. Institute
    of Science and Technology Austria.
  mla: Prach, Bernd. <i>Robust Image Classification with 1-Lipschitz Networks</i>.
    Institute of Science and Technology Austria, 2025, doi:<a href="https://doi.org/10.15479/10.15479/at-ista-19759">10.15479/10.15479/at-ista-19759</a>.
  short: B. Prach, Robust Image Classification with 1-Lipschitz Networks, Institute
    of Science and Technology Austria, 2025.
corr_author: '1'
date_created: 2025-05-28T16:20:48Z
date_published: 2025-05-30T00:00:00Z
date_updated: 2026-04-07T11:49:52Z
day: '30'
ddc:
- '000'
degree_awarded: PhD
department:
- _id: GradSch
- _id: ChLa
doi: 10.15479/10.15479/at-ista-19759
file:
- access_level: open_access
  checksum: e5108e759014e2a9020c973c778fafc9
  content_type: application/pdf
  creator: bprach
  date_created: 2025-06-10T18:11:05Z
  date_updated: 2025-06-10T18:11:05Z
  file_id: '19829'
  file_name: ThesisFinal.pdf
  file_size: 3578077
  relation: main_file
- access_level: closed
  checksum: 51bf6c11fb6d8a9f8010b458c600a83f
  content_type: application/x-zip-compressed
  creator: bprach
  date_created: 2025-06-10T18:14:03Z
  date_updated: 2025-06-10T18:14:03Z
  file_id: '19830'
  file_name: ThesisFinal.zip
  file_size: 74894357
  relation: source_file
file_date_updated: 2025-06-10T18:14:03Z
has_accepted_license: '1'
language:
- iso: eng
month: '05'
oa: 1
oa_version: Published Version
page: '84'
publication_identifier:
  issn:
  - 2663-337X
publication_status: published
publisher: Institute of Science and Technology Austria
related_material:
  record:
  - id: '15039'
    relation: part_of_dissertation
    status: public
  - id: '18874'
    relation: part_of_dissertation
    status: public
  - id: '17426'
    relation: part_of_dissertation
    status: public
  - id: '11839'
    relation: part_of_dissertation
    status: public
status: public
supervisor:
- first_name: Christoph
  full_name: Lampert, Christoph
  id: 40C20FD2-F248-11E8-B48F-1D18A9856A87
  last_name: Lampert
  orcid: 0000-0001-8622-7887
title: Robust image classification with 1-Lipschitz networks
type: dissertation
user_id: ba8df636-2132-11f1-aed0-ed93e2281fdd
year: '2025'
...
---
OA_place: repository
OA_type: green
_id: '20455'
abstract:
- lang: eng
  text: Despite extensive research since the community learned about adversarial examples
    10 years ago, we still do not know how to train high-accuracy classifiers that
    are guaranteed to be robust to small perturbations of their inputs. Previous works
    often argued that this might be because no classifier exists that is robust and
    accurate at the same time. However, in computer vision this assumption does not
    match reality where humans are usually accurate and robust on most tasks of interest.
    We offer an alternative explanation and show that in certain settings robust generalization
    is only possible with unrealistically large amounts of data. Specifically, we
    find a setting where a robust classifier exists, it is easy to learn an accurate
    classifier, yet it requires an exponential amount of data to learn a robust classifier.
    Based on this theoretical result, we evaluate the influence of the amount of training
    data on datasets such as CIFAR10. Our findings indicate that the the amount of
    training data is the main factor determining the robust performance. Furthermore
    we show that that there are low magnitude directions in the data which are useful
    for non-robust generalization but are not available for robust classifiers. This
    implies that robust classification is a strictly harder tasks than normal classification,
    thereby providing an explanation why robust classification requires more data.
article_processing_charge: No
arxiv: 1
author:
- first_name: Bernd
  full_name: Prach, Bernd
  id: 2D561D42-C427-11E9-89B4-9C1AE6697425
  last_name: Prach
- first_name: Christoph
  full_name: Lampert, Christoph
  id: 40C20FD2-F248-11E8-B48F-1D18A9856A87
  last_name: Lampert
  orcid: 0000-0001-8622-7887
citation:
  ama: 'Prach B, Lampert C. Intriguing properties of robust classification. In: <i>2025
    IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops</i>.
    IEEE; 2025:660-669. doi:<a href="https://doi.org/10.1109/CVPRW67362.2025.00071">10.1109/CVPRW67362.2025.00071</a>'
  apa: 'Prach, B., &#38; Lampert, C. (2025). Intriguing properties of robust classification.
    In <i>2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops</i>
    (pp. 660–669). Nashville, TN, United States: IEEE. <a href="https://doi.org/10.1109/CVPRW67362.2025.00071">https://doi.org/10.1109/CVPRW67362.2025.00071</a>'
  chicago: Prach, Bernd, and Christoph Lampert. “Intriguing Properties of Robust Classification.”
    In <i>2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops</i>,
    660–69. IEEE, 2025. <a href="https://doi.org/10.1109/CVPRW67362.2025.00071">https://doi.org/10.1109/CVPRW67362.2025.00071</a>.
  ieee: B. Prach and C. Lampert, “Intriguing properties of robust classification,”
    in <i>2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops</i>,
    Nashville, TN, United States, 2025, pp. 660–669.
  ista: 'Prach B, Lampert C. 2025. Intriguing properties of robust classification.
    2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops.
    CVPR: Conference on Computer Vision and Pattern Recognition, 660–669.'
  mla: Prach, Bernd, and Christoph Lampert. “Intriguing Properties of Robust Classification.”
    <i>2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops</i>,
    IEEE, 2025, pp. 660–69, doi:<a href="https://doi.org/10.1109/CVPRW67362.2025.00071">10.1109/CVPRW67362.2025.00071</a>.
  short: B. Prach, C. Lampert, in:, 2025 IEEE/CVF Conference on Computer Vision and
    Pattern Recognition Workshops, IEEE, 2025, pp. 660–669.
conference:
  end_date: 2025-06-12
  location: Nashville, TN, United States
  name: 'CVPR: Conference on Computer Vision and Pattern Recognition'
  start_date: 2025-06-11
corr_author: '1'
date_created: 2025-10-12T22:01:26Z
date_published: 2025-06-15T00:00:00Z
date_updated: 2025-10-13T07:18:26Z
day: '15'
department:
- _id: ChLa
doi: 10.1109/CVPRW67362.2025.00071
external_id:
  arxiv:
  - '2412.04245'
language:
- iso: eng
main_file_link:
- open_access: '1'
  url: https://doi.org/10.48550/arXiv.2412.04245
month: '06'
oa: 1
oa_version: Preprint
page: 660-669
publication: 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops
publication_identifier:
  eissn:
  - 2160-7516
  isbn:
  - '9798331599942'
  issn:
  - 2160-7508
publication_status: published
publisher: IEEE
quality_controlled: '1'
related_material:
  record:
  - id: '18874'
    relation: earlier_version
    status: public
scopus_import: '1'
status: public
title: Intriguing properties of robust classification
type: conference
user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87
year: '2025'
...
---
OA_place: repository
_id: '18874'
abstract:
- lang: eng
  text: "Despite extensive research since the community learned about adversarial\r\nexamples
    10 years ago, we still do not know how to train high-accuracy\r\nclassifiers that
    are guaranteed to be robust to small perturbations of their\r\ninputs. Previous
    works often argued that this might be because no classifier\r\nexists that is
    robust and accurate at the same time. However, in computer\r\nvision this assumption
    does not match reality where humans are usually accurate\r\nand robust on most
    tasks of interest. We offer an alternative explanation and\r\nshow that in certain
    settings robust generalization is only possible with\r\nunrealistically large
    amounts of data. More precisely we find a setting where a\r\nrobust classifier
    exists, it is easy to learn an accurate classifier, yet it\r\nrequires an exponential
    amount of data to learn a robust classifier. Based on\r\nthis theoretical result,
    we explore how well robust classifiers generalize on\r\ndatasets such as CIFAR-10.
    We come to the conclusion that on this datasets, the\r\nlimitation of current
    robust models also lies in the generalization, and that\r\nthey require a lot
    of data to do well on the test set. We also show that the\r\nproblem is not in
    the expressiveness or generalization capabilities of current\r\narchitectures,
    and that there are low magnitude features in the data which are\r\nuseful for
    non-robust generalization but are not available for robust\r\nclassifiers."
article_number: '2412.04245'
article_processing_charge: No
arxiv: 1
author:
- first_name: Bernd
  full_name: Prach, Bernd
  id: 2D561D42-C427-11E9-89B4-9C1AE6697425
  last_name: Prach
- first_name: Christoph
  full_name: Lampert, Christoph
  id: 40C20FD2-F248-11E8-B48F-1D18A9856A87
  last_name: Lampert
  orcid: 0000-0001-8622-7887
citation:
  ama: Prach B, Lampert C. Intriguing properties of robust classification. <i>arXiv</i>.
    doi:<a href="https://doi.org/10.48550/arXiv.2412.04245">10.48550/arXiv.2412.04245</a>
  apa: Prach, B., &#38; Lampert, C. (n.d.). Intriguing properties of robust classification.
    <i>arXiv</i>. <a href="https://doi.org/10.48550/arXiv.2412.04245">https://doi.org/10.48550/arXiv.2412.04245</a>
  chicago: Prach, Bernd, and Christoph Lampert. “Intriguing Properties of Robust Classification.”
    <i>ArXiv</i>, n.d. <a href="https://doi.org/10.48550/arXiv.2412.04245">https://doi.org/10.48550/arXiv.2412.04245</a>.
  ieee: B. Prach and C. Lampert, “Intriguing properties of robust classification,”
    <i>arXiv</i>. .
  ista: Prach B, Lampert C. Intriguing properties of robust classification. arXiv,
    2412.04245.
  mla: Prach, Bernd, and Christoph Lampert. “Intriguing Properties of Robust Classification.”
    <i>ArXiv</i>, 2412.04245, doi:<a href="https://doi.org/10.48550/arXiv.2412.04245">10.48550/arXiv.2412.04245</a>.
  short: B. Prach, C. Lampert, ArXiv (n.d.).
corr_author: '1'
date_created: 2025-01-24T16:57:29Z
date_published: 2024-12-05T00:00:00Z
date_updated: 2026-04-07T11:49:51Z
day: '05'
department:
- _id: GradSch
- _id: ChLa
doi: 10.48550/arXiv.2412.04245
external_id:
  arxiv:
  - '2412.04245'
language:
- iso: eng
main_file_link:
- open_access: '1'
  url: https://doi.org/10.48550/arXiv.2412.04245
month: '12'
oa: 1
oa_version: Preprint
publication: arXiv
publication_status: draft
related_material:
  record:
  - id: '20455'
    relation: later_version
    status: public
  - id: '19759'
    relation: dissertation_contains
    status: public
status: public
title: Intriguing properties of robust classification
type: preprint
user_id: 8b945eb4-e2f2-11eb-945a-df72226e66a9
year: '2024'
...
---
OA_place: repository
OA_type: green
_id: '17426'
abstract:
- lang: eng
  text: "The robustness of neural networks against input perturbations with bounded\r\nmagnitude
    represents a serious concern in the deployment of deep learning\r\nmodels in safety-critical
    systems. Recently, the scientific community has\r\nfocused on enhancing certifiable
    robustness guarantees by crafting 1-Lipschitz\r\nneural networks that leverage
    Lipschitz bounded dense and convolutional layers.\r\nAlthough different methods
    have been proposed in the literature to achieve this\r\ngoal, understanding the
    performance of such methods is not straightforward,\r\nsince different metrics
    can be relevant (e.g., training time, memory usage,\r\naccuracy, certifiable robustness)
    for different applications. For this reason,\r\nthis work provides a thorough
    theoretical and empirical comparison between\r\nmethods by evaluating them in
    terms of memory usage, speed, and certifiable\r\nrobust accuracy. The paper also
    provides some guidelines and recommendations to\r\nsupport the user in selecting
    the methods that work best depending on the\r\navailable resources. We provide
    code at\r\nhttps://github.com/berndprach/1LipschitzLayersCompared."
acknowledgement: "This work was partially supported by project SERICS (PE00000014)
  under the MUR National Recovery and Resilience Plan funded by the European Union
  - NextGenerationEU.\r\n"
article_processing_charge: No
arxiv: 1
author:
- first_name: Bernd
  full_name: Prach, Bernd
  id: 2D561D42-C427-11E9-89B4-9C1AE6697425
  last_name: Prach
- first_name: Fabio
  full_name: Brau, Fabio
  last_name: Brau
- first_name: Giorgio
  full_name: Buttazzo, Giorgio
  last_name: Buttazzo
- first_name: Christoph
  full_name: Lampert, Christoph
  id: 40C20FD2-F248-11E8-B48F-1D18A9856A87
  last_name: Lampert
  orcid: 0000-0001-8622-7887
citation:
  ama: 'Prach B, Brau F, Buttazzo G, Lampert C. 1-Lipschitz layers compared: Memory,
    speed, and certifiable robustness. In: <i>Proceedings of the IEEE/CVF Conference
    on Computer Vision and Pattern Recognition</i>. Computer Vision Foundation; 2024:24574-24583.
    doi:<a href="https://doi.org/10.1109/CVPR52733.2024.02320">10.1109/CVPR52733.2024.02320</a>'
  apa: 'Prach, B., Brau, F., Buttazzo, G., &#38; Lampert, C. (2024). 1-Lipschitz layers
    compared: Memory, speed, and certifiable robustness. In <i>Proceedings of the
    IEEE/CVF Conference on Computer Vision and Pattern Recognition</i> (pp. 24574–24583).
    Seattle, WA, United States: Computer Vision Foundation. <a href="https://doi.org/10.1109/CVPR52733.2024.02320">https://doi.org/10.1109/CVPR52733.2024.02320</a>'
  chicago: 'Prach, Bernd, Fabio Brau, Giorgio Buttazzo, and Christoph Lampert. “1-Lipschitz
    Layers Compared: Memory, Speed, and Certifiable Robustness.” In <i>Proceedings
    of the IEEE/CVF Conference on Computer Vision and Pattern Recognition</i>, 24574–83.
    Computer Vision Foundation, 2024. <a href="https://doi.org/10.1109/CVPR52733.2024.02320">https://doi.org/10.1109/CVPR52733.2024.02320</a>.'
  ieee: 'B. Prach, F. Brau, G. Buttazzo, and C. Lampert, “1-Lipschitz layers compared:
    Memory, speed, and certifiable robustness,” in <i>Proceedings of the IEEE/CVF
    Conference on Computer Vision and Pattern Recognition</i>, Seattle, WA, United
    States, 2024, pp. 24574–24583.'
  ista: 'Prach B, Brau F, Buttazzo G, Lampert C. 2024. 1-Lipschitz layers compared:
    Memory, speed, and certifiable robustness. Proceedings of the IEEE/CVF Conference
    on Computer Vision and Pattern Recognition. CVPR: Conference on Computer Vision
    and Pattern Recognition, 24574–24583.'
  mla: 'Prach, Bernd, et al. “1-Lipschitz Layers Compared: Memory, Speed, and Certifiable
    Robustness.” <i>Proceedings of the IEEE/CVF Conference on Computer Vision and
    Pattern Recognition</i>, Computer Vision Foundation, 2024, pp. 24574–83, doi:<a
    href="https://doi.org/10.1109/CVPR52733.2024.02320">10.1109/CVPR52733.2024.02320</a>.'
  short: B. Prach, F. Brau, G. Buttazzo, C. Lampert, in:, Proceedings of the IEEE/CVF
    Conference on Computer Vision and Pattern Recognition, Computer Vision Foundation,
    2024, pp. 24574–24583.
conference:
  end_date: 2024-06-22
  location: Seattle, WA, United States
  name: 'CVPR: Conference on Computer Vision and Pattern Recognition'
  start_date: 2024-06-16
corr_author: '1'
date_created: 2024-08-14T08:42:32Z
date_published: 2024-06-01T00:00:00Z
date_updated: 2026-04-07T11:49:51Z
day: '01'
department:
- _id: GradSch
- _id: ChLa
doi: 10.1109/CVPR52733.2024.02320
external_id:
  arxiv:
  - '2311.16833'
  isi:
  - '001344387500055'
has_accepted_license: '1'
isi: 1
language:
- iso: eng
main_file_link:
- open_access: '1'
  url: https://doi.org/10.48550/arXiv.2311.16833
month: '06'
oa: 1
oa_version: Preprint
page: 24574-24583
publication: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern
  Recognition
publication_status: published
publisher: Computer Vision Foundation
quality_controlled: '1'
related_material:
  link:
  - relation: software
    url: https://github.com/berndprach/1LipschitzLayersCompared
  record:
  - id: '19759'
    relation: dissertation_contains
    status: public
status: public
title: '1-Lipschitz layers compared: Memory, speed, and certifiable robustness'
type: conference
user_id: 317138e5-6ab7-11ef-aa6d-ffef3953e345
year: '2024'
...
---
OA_place: repository
_id: '15039'
abstract:
- lang: eng
  text: 'A crucial property for achieving secure, trustworthy and interpretable deep
    learning systems is their robustness: small changes to a system''s inputs should
    not result in large changes to its outputs. Mathematically, this means one strives
    for networks with a small Lipschitz constant. Several recent works have focused
    on how to construct such Lipschitz networks, typically by imposing constraints
    on the weight matrices. In this work, we study an orthogonal aspect, namely the
    role of the activation function. We show that commonly used activation functions,
    such as MaxMin, as well as all piece-wise linear ones with two segments unnecessarily
    restrict the class of representable functions, even in the simplest one-dimensional
    setting. We furthermore introduce the new N-activation function that is provably
    more expressive than currently popular activation functions. We provide code at
    this https URL.'
article_number: '2311.06103'
article_processing_charge: No
arxiv: 1
author:
- first_name: Bernd
  full_name: Prach, Bernd
  id: 2D561D42-C427-11E9-89B4-9C1AE6697425
  last_name: Prach
- first_name: Christoph
  full_name: Lampert, Christoph
  id: 40C20FD2-F248-11E8-B48F-1D18A9856A87
  last_name: Lampert
  orcid: 0000-0001-8622-7887
citation:
  ama: Prach B, Lampert C. 1-Lipschitz neural networks are more expressive with N-activations.
    <i>arXiv</i>. doi:<a href="https://doi.org/10.48550/ARXIV.2311.06103">10.48550/ARXIV.2311.06103</a>
  apa: Prach, B., &#38; Lampert, C. (n.d.). 1-Lipschitz neural networks are more expressive
    with N-activations. <i>arXiv</i>. <a href="https://doi.org/10.48550/ARXIV.2311.06103">https://doi.org/10.48550/ARXIV.2311.06103</a>
  chicago: Prach, Bernd, and Christoph Lampert. “1-Lipschitz Neural Networks Are More
    Expressive with N-Activations.” <i>ArXiv</i>, n.d. <a href="https://doi.org/10.48550/ARXIV.2311.06103">https://doi.org/10.48550/ARXIV.2311.06103</a>.
  ieee: B. Prach and C. Lampert, “1-Lipschitz neural networks are more expressive
    with N-activations,” <i>arXiv</i>. .
  ista: Prach B, Lampert C. 1-Lipschitz neural networks are more expressive with N-activations.
    arXiv, 2311.06103.
  mla: Prach, Bernd, and Christoph Lampert. “1-Lipschitz Neural Networks Are More
    Expressive with N-Activations.” <i>ArXiv</i>, 2311.06103, doi:<a href="https://doi.org/10.48550/ARXIV.2311.06103">10.48550/ARXIV.2311.06103</a>.
  short: B. Prach, C. Lampert, ArXiv (n.d.).
corr_author: '1'
date_created: 2024-02-28T17:59:32Z
date_published: 2023-11-10T00:00:00Z
date_updated: 2026-04-07T11:49:51Z
day: '10'
department:
- _id: GradSch
- _id: ChLa
doi: 10.48550/ARXIV.2311.06103
external_id:
  arxiv:
  - '2311.06103'
language:
- iso: eng
main_file_link:
- open_access: '1'
  url: https://doi.org/10.48550/arXiv.2311.06103
month: '11'
oa: 1
oa_version: Preprint
publication: arXiv
publication_status: draft
related_material:
  record:
  - id: '19759'
    relation: dissertation_contains
    status: public
status: public
title: 1-Lipschitz neural networks are more expressive with N-activations
tmp:
  image: /images/cc_by.png
  legal_code_url: https://creativecommons.org/licenses/by/4.0/legalcode
  name: Creative Commons Attribution 4.0 International Public License (CC-BY 4.0)
  short: CC BY (4.0)
type: preprint
user_id: 8b945eb4-e2f2-11eb-945a-df72226e66a9
year: '2023'
...
---
_id: '11839'
abstract:
- lang: eng
  text: "It is a highly desirable property for deep networks to be robust against\r\nsmall
    input changes. One popular way to achieve this property is by designing\r\nnetworks
    with a small Lipschitz constant. In this work, we propose a new\r\ntechnique for
    constructing such Lipschitz networks that has a number of\r\ndesirable properties:
    it can be applied to any linear network layer\r\n(fully-connected or convolutional),
    it provides formal guarantees on the\r\nLipschitz constant, it is easy to implement
    and efficient to run, and it can be\r\ncombined with any training objective and
    optimization method. In fact, our\r\ntechnique is the first one in the literature
    that achieves all of these\r\nproperties simultaneously. Our main contribution
    is a rescaling-based weight\r\nmatrix parametrization that guarantees each network
    layer to have a Lipschitz\r\nconstant of at most 1 and results in the learned
    weight matrices to be close to\r\northogonal. Hence we call such layers almost-orthogonal
    Lipschitz (AOL).\r\nExperiments and ablation studies in the context of image classification
    with\r\ncertified robust accuracy confirm that AOL layers achieve results that
    are on\r\npar with most existing methods. Yet, they are simpler to implement and
    more\r\nbroadly applicable, because they do not require computationally expensive\r\nmatrix
    orthogonalization or inversion steps as part of the network\r\narchitecture. We
    provide code at https://github.com/berndprach/AOL."
alternative_title:
- LNCS
article_processing_charge: No
arxiv: 1
author:
- first_name: Bernd
  full_name: Prach, Bernd
  id: 2D561D42-C427-11E9-89B4-9C1AE6697425
  last_name: Prach
- first_name: Christoph
  full_name: Lampert, Christoph
  id: 40C20FD2-F248-11E8-B48F-1D18A9856A87
  last_name: Lampert
  orcid: 0000-0001-8622-7887
citation:
  ama: 'Prach B, Lampert C. Almost-orthogonal layers for efficient general-purpose
    Lipschitz networks. In: <i>Computer Vision – ECCV 2022</i>. Vol 13681. Springer
    Nature; 2022:350-365. doi:<a href="https://doi.org/10.1007/978-3-031-19803-8_21">10.1007/978-3-031-19803-8_21</a>'
  apa: 'Prach, B., &#38; Lampert, C. (2022). Almost-orthogonal layers for efficient
    general-purpose Lipschitz networks. In <i>Computer Vision – ECCV 2022</i> (Vol.
    13681, pp. 350–365). Tel Aviv, Israel: Springer Nature. <a href="https://doi.org/10.1007/978-3-031-19803-8_21">https://doi.org/10.1007/978-3-031-19803-8_21</a>'
  chicago: Prach, Bernd, and Christoph Lampert. “Almost-Orthogonal Layers for Efficient
    General-Purpose Lipschitz Networks.” In <i>Computer Vision – ECCV 2022</i>, 13681:350–65.
    Springer Nature, 2022. <a href="https://doi.org/10.1007/978-3-031-19803-8_21">https://doi.org/10.1007/978-3-031-19803-8_21</a>.
  ieee: B. Prach and C. Lampert, “Almost-orthogonal layers for efficient general-purpose
    Lipschitz networks,” in <i>Computer Vision – ECCV 2022</i>, Tel Aviv, Israel,
    2022, vol. 13681, pp. 350–365.
  ista: 'Prach B, Lampert C. 2022. Almost-orthogonal layers for efficient general-purpose
    Lipschitz networks. Computer Vision – ECCV 2022. ECCV: European Conference on
    Computer Vision, LNCS, vol. 13681, 350–365.'
  mla: Prach, Bernd, and Christoph Lampert. “Almost-Orthogonal Layers for Efficient
    General-Purpose Lipschitz Networks.” <i>Computer Vision – ECCV 2022</i>, vol.
    13681, Springer Nature, 2022, pp. 350–65, doi:<a href="https://doi.org/10.1007/978-3-031-19803-8_21">10.1007/978-3-031-19803-8_21</a>.
  short: B. Prach, C. Lampert, in:, Computer Vision – ECCV 2022, Springer Nature,
    2022, pp. 350–365.
conference:
  end_date: 2022-10-27
  location: Tel Aviv, Israel
  name: 'ECCV: European Conference on Computer Vision'
  start_date: 2022-10-23
corr_author: '1'
date_created: 2022-08-12T15:09:47Z
date_published: 2022-10-23T00:00:00Z
date_updated: 2026-04-07T11:49:51Z
day: '23'
department:
- _id: GradSch
- _id: ChLa
doi: 10.1007/978-3-031-19803-8_21
external_id:
  arxiv:
  - '2208.03160'
  isi:
  - '000904104000021'
intvolume: '     13681'
isi: 1
language:
- iso: eng
main_file_link:
- open_access: '1'
  url: ' https://doi.org/10.48550/arXiv.2208.03160'
month: '10'
oa: 1
oa_version: Preprint
page: 350-365
publication: Computer Vision – ECCV 2022
publication_identifier:
  eisbn:
  - '9783031198038'
  isbn:
  - '9783031198021'
publication_status: published
publisher: Springer Nature
quality_controlled: '1'
related_material:
  record:
  - id: '19759'
    relation: dissertation_contains
    status: public
scopus_import: '1'
status: public
title: Almost-orthogonal layers for efficient general-purpose Lipschitz networks
type: conference
user_id: 317138e5-6ab7-11ef-aa6d-ffef3953e345
volume: 13681
year: '2022'
...
