{"intvolume":"     13681","publication_status":"published","oa_version":"Preprint","oa":1,"author":[{"id":"2D561D42-C427-11E9-89B4-9C1AE6697425","first_name":"Bernd","full_name":"Prach, Bernd","last_name":"Prach"},{"orcid":"0000-0001-8622-7887","first_name":"Christoph","id":"40C20FD2-F248-11E8-B48F-1D18A9856A87","last_name":"Lampert","full_name":"Lampert, Christoph"}],"type":"conference","year":"2022","date_updated":"2023-05-03T08:00:46Z","page":"350-365","month":"10","volume":13681,"date_created":"2022-08-12T15:09:47Z","department":[{"_id":"GradSch"},{"_id":"ChLa"}],"title":"Almost-orthogonal layers for efficient general-purpose Lipschitz networks","_id":"11839","user_id":"2DF688A6-F248-11E8-B48F-1D18A9856A87","article_processing_charge":"No","language":[{"iso":"eng"}],"abstract":[{"text":"It is a highly desirable property for deep networks to be robust against\r\nsmall input changes. One popular way to achieve this property is by designing\r\nnetworks with a small Lipschitz constant. In this work, we propose a new\r\ntechnique for constructing such Lipschitz networks that has a number of\r\ndesirable properties: it can be applied to any linear network layer\r\n(fully-connected or convolutional), it provides formal guarantees on the\r\nLipschitz constant, it is easy to implement and efficient to run, and it can be\r\ncombined with any training objective and optimization method. In fact, our\r\ntechnique is the first one in the literature that achieves all of these\r\nproperties simultaneously. Our main contribution is a rescaling-based weight\r\nmatrix parametrization that guarantees each network layer to have a Lipschitz\r\nconstant of at most 1 and results in the learned weight matrices to be close to\r\northogonal. Hence we call such layers almost-orthogonal Lipschitz (AOL).\r\nExperiments and ablation studies in the context of image classification with\r\ncertified robust accuracy confirm that AOL layers achieve results that are on\r\npar with most existing methods. Yet, they are simpler to implement and more\r\nbroadly applicable, because they do not require computationally expensive\r\nmatrix orthogonalization or inversion steps as part of the network\r\narchitecture. We provide code at https://github.com/berndprach/AOL.","lang":"eng"}],"publisher":"Springer Nature","conference":{"location":"Tel Aviv, Israel","start_date":"2022-10-23","end_date":"2022-10-27","name":"ECCV: European Conference on Computer Vision"},"citation":{"ama":"Prach B, Lampert C. Almost-orthogonal layers for efficient general-purpose Lipschitz networks. In: <i>Computer Vision – ECCV 2022</i>. Vol 13681. Springer Nature; 2022:350-365. doi:<a href=\"https://doi.org/10.1007/978-3-031-19803-8_21\">10.1007/978-3-031-19803-8_21</a>","ieee":"B. Prach and C. Lampert, “Almost-orthogonal layers for efficient general-purpose Lipschitz networks,” in <i>Computer Vision – ECCV 2022</i>, Tel Aviv, Israel, 2022, vol. 13681, pp. 350–365.","chicago":"Prach, Bernd, and Christoph Lampert. “Almost-Orthogonal Layers for Efficient General-Purpose Lipschitz Networks.” In <i>Computer Vision – ECCV 2022</i>, 13681:350–65. Springer Nature, 2022. <a href=\"https://doi.org/10.1007/978-3-031-19803-8_21\">https://doi.org/10.1007/978-3-031-19803-8_21</a>.","mla":"Prach, Bernd, and Christoph Lampert. “Almost-Orthogonal Layers for Efficient General-Purpose Lipschitz Networks.” <i>Computer Vision – ECCV 2022</i>, vol. 13681, Springer Nature, 2022, pp. 350–65, doi:<a href=\"https://doi.org/10.1007/978-3-031-19803-8_21\">10.1007/978-3-031-19803-8_21</a>.","ista":"Prach B, Lampert C. 2022. Almost-orthogonal layers for efficient general-purpose Lipschitz networks. Computer Vision – ECCV 2022. ECCV: European Conference on Computer Vision, LNCS, vol. 13681, 350–365.","short":"B. Prach, C. Lampert, in:, Computer Vision – ECCV 2022, Springer Nature, 2022, pp. 350–365.","apa":"Prach, B., &#38; Lampert, C. (2022). Almost-orthogonal layers for efficient general-purpose Lipschitz networks. In <i>Computer Vision – ECCV 2022</i> (Vol. 13681, pp. 350–365). Tel Aviv, Israel: Springer Nature. <a href=\"https://doi.org/10.1007/978-3-031-19803-8_21\">https://doi.org/10.1007/978-3-031-19803-8_21</a>"},"doi":"10.1007/978-3-031-19803-8_21","scopus_import":"1","main_file_link":[{"open_access":"1","url":" https://doi.org/10.48550/arXiv.2208.03160"}],"publication_identifier":{"eisbn":["9783031198038"],"isbn":["9783031198021"]},"date_published":"2022-10-23T00:00:00Z","day":"23","status":"public","alternative_title":["LNCS"],"publication":"Computer Vision – ECCV 2022","external_id":{"arxiv":["2208.03160"]},"quality_controlled":"1"}