--- OA_place: publisher _id: '21854' abstract: - lang: eng text: "As neural-network-based models grow both in size and popularity, interest has grown in making the models smaller and more efficient to train. To that end, many methods have been proposed to prune models by reducing their number of nonzero parameters. Additionally, parameter-efficient fine-tuning, in which a much smaller number of parameters than the total contained in the model is updated during training, has become very popular, especially in the space of Large Language Models. At the same time, the increasingly routine deployment of machine learning in real-world applications has spurred a drive to make them more trustworthy - in the sense of, among other things, being unbiased, interpretable, and editable. In this thesis, we examine the interplay between efficiency and trustworthiness.\r\n\r\nFirst, we analyze the effects of model pruning on bias in computer vision models, demonstrating that increased sparsity leads to greater bias, largely as a function of increased model uncertainty in marginal cases. Based on this observation, we propose several bias mitigation techniques. Then, we demonstrate that example-specific model pruning can improve model interpretation methods while improving pruning efficiency to make example-specific model pruning feasible in real time. Then, we investigate the effectiveness of parameter-efficient and data-efficient model personalization via fine-tuning, demonstrating that it is highly feasible with very small computational and data resources. Finally, we consider efficiency in editing model knowledge using a custom synthetic data framework, demonstrating that parameter-efficient, low-rank fine-tuning frequently outperforms full-rank fine-tuning, and, additionally, that restricting which model blocks are fine-tuned frequently improves results. Together, the results in this thesis provide new insights and techniques for combining trustworthiness and efficiency during neural network inference and training.\r\n\r\n-----------------“In reference to IEEE copyrighted material which is used with permission in this thesis, the IEEE does not endorse any of [name of university or educational entity]’s products or services. Internal or personal use of this material is permitted. If interested in reprinting/republishing IEEE copyrighted material for advertising or promotional purposes or for creating new collective works for resale or redistribution, please go to http://www.ieee.org/publications_standards/publications/rights/rights_link.html to learn how to obtain a License from RightsLink. If applicable, University Microfilms and/or ProQuest Library, or the Archives of Canada may supply single copies of the dissertation.”" acknowledged_ssus: - _id: ScienComp acknowledgement: "The research in this Ph.D. was funded in whole\r\nor in part by the Austrian Science Fund (FWF) W1260-N35 (Vienna Graduate School for\r\nComputational Optimization). For open access purposes the author has applied a CC BY\r\npublic copyright license to any author accepted manuscript version arising from this submission\r\nwherever possible. Additionally, I am grateful to Alois Schlögl, Waleed Khalid, and the rest of\r\nthe ISTA Scientific Computing team for building and maintaining the infrastructure I used\r\nto run experiments. I’m also deeply grateful to the Alistarh group’s administrative assistant,\r\nChristine Francois, who always deals with our nonsense with common sense and a smile.\r\n" alternative_title: - ISTA Thesis article_processing_charge: No author: - first_name: Eugenia B full_name: Iofinova, Eugenia B id: f9a17499-f6e0-11ea-865d-fdf9a3f77117 last_name: Iofinova orcid: 0000-0002-7778-3221 citation: ama: Iofinova EB. On the utility and effects of efficiency in artificial neural networks. 2026. doi:10.15479/AT-ISTA-21854 apa: Iofinova, E. B. (2026). On the utility and effects of efficiency in artificial neural networks. Institute of Science and Technology Austria. https://doi.org/10.15479/AT-ISTA-21854 chicago: Iofinova, Eugenia B. “On the Utility and Effects of Efficiency in Artificial Neural Networks.” Institute of Science and Technology Austria, 2026. https://doi.org/10.15479/AT-ISTA-21854. ieee: E. B. Iofinova, “On the utility and effects of efficiency in artificial neural networks,” Institute of Science and Technology Austria, 2026. ista: Iofinova EB. 2026. On the utility and effects of efficiency in artificial neural networks. Institute of Science and Technology Austria. mla: Iofinova, Eugenia B. On the Utility and Effects of Efficiency in Artificial Neural Networks. Institute of Science and Technology Austria, 2026, doi:10.15479/AT-ISTA-21854. short: E.B. Iofinova, On the Utility and Effects of Efficiency in Artificial Neural Networks, Institute of Science and Technology Austria, 2026. corr_author: '1' date_created: 2026-05-11T08:43:22Z date_published: 2026-05-11T00:00:00Z date_updated: 2026-05-19T11:20:28Z day: '11' ddc: - '000' degree_awarded: PhD department: - _id: GradSch - _id: DaAl doi: 10.15479/AT-ISTA-21854 file: - access_level: closed checksum: 2e148dad920e3f9b7c32796e0ba2e5f7 content_type: application/zip creator: eiofinov date_created: 2026-05-11T08:36:01Z date_updated: 2026-05-11T08:36:01Z file_id: '21856' file_name: EIofinova_thesis_FinalVersion.zip file_size: 28479571 relation: source_file - access_level: open_access checksum: b10c2933f386f532b2dbf28b19c5525c content_type: application/pdf creator: eiofinov date_created: 2026-05-13T13:10:48Z date_updated: 2026-05-13T13:10:48Z file_id: '21877' file_name: 2026_Iofinova_Eugenia_Thesis.pdf file_size: 18137757 relation: main_file success: 1 file_date_updated: 2026-05-13T13:10:48Z has_accepted_license: '1' language: - iso: eng month: '05' oa: 1 oa_version: Published Version page: '237' project: - _id: 9B9290DE-BA93-11EA-9121-9846C619BF3A grant_number: W1260-N35 name: Vienna Graduate School on Computational Optimization publication_identifier: issn: - 2663-337X publication_status: published publisher: Institute of Science and Technology Austria related_material: record: - id: '14771' relation: part_of_dissertation status: public - id: '18121' relation: part_of_dissertation status: public - id: '21858' relation: part_of_dissertation status: public - id: '21859' relation: part_of_dissertation status: public - id: '21857' relation: part_of_dissertation status: public status: public supervisor: - first_name: Dan-Adrian full_name: Alistarh, Dan-Adrian id: 4A899BFC-F248-11E8-B48F-1D18A9856A87 last_name: Alistarh orcid: 0000-0003-3650-940X title: On the utility and effects of efficiency in artificial neural networks type: dissertation user_id: 8b945eb4-e2f2-11eb-945a-df72226e66a9 year: '2026' ... --- OA_place: repository OA_type: green _id: '21859' abstract: - lang: eng text: As artificial neural networks, and specifically large language models, have improved rapidly in capabilities and quality, they have increasingly been deployed in real-world applications, from customer service to Google search, despite the fact that they frequently make factually incorrect or undesirable statements. This trend has inspired practical and academic interest in model editing, that is, in adjusting the weights of the model to modify its likely outputs for queries relating to a specific fact or set of facts. This may be done either to amend a fact or set of facts, for instance, to fix a frequent error in the training data, or to suppress a fact or set of facts entirely, for instance, in case of dangerous knowledge. Multiple methods have been proposed to do such edits. However, at the same time, it has been shown that such model editing can be brittle and incomplete. Moreover the effectiveness of any model editing method necessarily depends on the data on which the model is trained, and, therefore, a good understanding of the interaction of the training data distribution and the way it is stored in the network is necessary and helpful to reliably perform model editing. However, working with large language models trained on real-world data does not allow us to understand this relationship or fully measure the effects of model editing. We therefore propose Behemoth, a fully synthetic data generation framework. To demonstrate the practical insights from the framework, we explore model editing in the context of simple tabular data, demonstrating surprising findings that, in some cases, echo real-world results, for instance, that in some cases restricting the update rank results in a more effective update. acknowledged_ssus: - _id: ScienComp acknowledgement: "EI thanks Weiwei Yang, Janardhan Kulkani, and Kate Lytvynets for their advice and support in\r\ndeveloping an earlier version of the Behemoth library. This research was supported by the Scientific\r\nService Units (SSU) of IST Austria through resources provided by Scientific Computing (SciComp).\r\nEI was supported in part by the FWF DK VGSCO, grant agreement number W1260-N35.\r\n" article_processing_charge: No arxiv: 1 author: - first_name: Eugenia B full_name: Iofinova, Eugenia B id: f9a17499-f6e0-11ea-865d-fdf9a3f77117 last_name: Iofinova orcid: 0000-0002-7778-3221 - first_name: Dan-Adrian full_name: Alistarh, Dan-Adrian id: 4A899BFC-F248-11E8-B48F-1D18A9856A87 last_name: Alistarh orcid: 0000-0003-3650-940X citation: ama: 'Iofinova EB, Alistarh D-A. Behemoth: Benchmarking unlearning in LLMs using fully synthetic data. arXiv. doi:10.48550/arXiv.2601.23153' apa: 'Iofinova, E. B., & Alistarh, D.-A. (n.d.). Behemoth: Benchmarking unlearning in LLMs using fully synthetic data. arXiv. https://doi.org/10.48550/arXiv.2601.23153' chicago: 'Iofinova, Eugenia B, and Dan-Adrian Alistarh. “Behemoth: Benchmarking Unlearning in LLMs Using Fully Synthetic Data.” ArXiv, n.d. https://doi.org/10.48550/arXiv.2601.23153.' ieee: 'E. B. Iofinova and D.-A. Alistarh, “Behemoth: Benchmarking unlearning in LLMs using fully synthetic data,” arXiv. .' ista: 'Iofinova EB, Alistarh D-A. Behemoth: Benchmarking unlearning in LLMs using fully synthetic data. arXiv, 10.48550/arXiv.2601.23153.' mla: 'Iofinova, Eugenia B., and Dan-Adrian Alistarh. “Behemoth: Benchmarking Unlearning in LLMs Using Fully Synthetic Data.” ArXiv, doi:10.48550/arXiv.2601.23153.' short: E.B. Iofinova, D.-A. Alistarh, ArXiv (n.d.). corr_author: '1' date_created: 2026-05-11T08:58:07Z date_published: 2026-01-30T00:00:00Z date_updated: 2026-05-19T11:20:27Z day: '30' department: - _id: GradSch - _id: DaAl doi: 10.48550/arXiv.2601.23153 external_id: arxiv: - '2601.23153' language: - iso: eng main_file_link: - open_access: '1' url: https://doi.org/10.48550/arXiv.2601.23153 month: '01' oa: 1 oa_version: Preprint project: - _id: 9B9290DE-BA93-11EA-9121-9846C619BF3A grant_number: W1260-N35 name: Vienna Graduate School on Computational Optimization publication: arXiv publication_status: draft related_material: record: - id: '21854' relation: dissertation_contains status: public status: public title: 'Behemoth: Benchmarking unlearning in LLMs using fully synthetic data' type: preprint user_id: 8b945eb4-e2f2-11eb-945a-df72226e66a9 year: '2026' ... --- OA_place: publisher _id: '21957' abstract: - lang: eng text: "This thesis investigates algorithmic certification and approximation methods for degenerate semidefinite programs (SDPs) and the singular roots of polynomial systems. In the first part, we present a hybrid symbolic-numeric algorithm for certifying the feasibility of weakly feasible, degenerate SDPs. By reformulating linear matrix inequalities (LMIs) into a structured polynomial system via facial reduction and incidence varieties, we guarantee the existence of an isolated exact solution. This algebraic reduction enables the certification of maximum-rank numerical approximations using methods from algebraic geometry.\r\n\r\nIn the second part, we address the severe ill-conditioning and loss of quadratic convergence that plague standard path-tracking methods near isolated singular roots. To overcome this, we propose tracking algorithms that achieve superlinear convergence without the computational bloat characteristic of classical deflation techniques. By modeling the solution path as a generalized fractional Puiseux series, our approach combines an explicitly derived algebraic predictor with a localized hyperplane desingularization phase during the corrector step. Furthermore, we introduce a continuous path-limit method and an extension of the geometric sequence rule to directly extract exact fractional exponents. This bypasses traditional heuristic trial-and-error methods and explicitly accommodates sparse series expansions. Numerical experiments confirm that our method significantly reduces the cumulative number of matrix inversions while achieving high-accuracy root approximations, even for heavily degenerate systems exhibiting higher coranks." acknowledgement: 'Funding: Vienna Graduate School on Computational Optimization (FWF), grant DOI: 10.55776/W1260.' alternative_title: - ISTA Thesis article_processing_charge: No author: - first_name: Jeferson full_name: Zapata, Jeferson id: 00223538-AF8F-11E9-A4C7-F729E6697425 last_name: Zapata citation: ama: 'Zapata J. Overcoming degeneracy and singularity : Techniques for semidefinite programs and homotopy continuation endgames. 2026. doi:10.15479/AT-ISTA-21957' apa: 'Zapata, J. (2026). Overcoming degeneracy and singularity : Techniques for semidefinite programs and homotopy continuation endgames. Institute of Science and Technology Austria. https://doi.org/10.15479/AT-ISTA-21957' chicago: 'Zapata, Jeferson. “Overcoming Degeneracy and Singularity : Techniques for Semidefinite Programs and Homotopy Continuation Endgames.” Institute of Science and Technology Austria, 2026. https://doi.org/10.15479/AT-ISTA-21957.' ieee: 'J. Zapata, “Overcoming degeneracy and singularity : Techniques for semidefinite programs and homotopy continuation endgames,” Institute of Science and Technology Austria, 2026.' ista: 'Zapata J. 2026. Overcoming degeneracy and singularity : Techniques for semidefinite programs and homotopy continuation endgames. Institute of Science and Technology Austria.' mla: 'Zapata, Jeferson. Overcoming Degeneracy and Singularity : Techniques for Semidefinite Programs and Homotopy Continuation Endgames. Institute of Science and Technology Austria, 2026, doi:10.15479/AT-ISTA-21957.' short: 'J. Zapata, Overcoming Degeneracy and Singularity : Techniques for Semidefinite Programs and Homotopy Continuation Endgames, Institute of Science and Technology Austria, 2026.' corr_author: '1' date_created: 2026-06-08T13:29:52Z date_published: 2026-06-09T00:00:00Z date_updated: 2026-06-12T10:37:00Z day: '09' ddc: - '500' degree_awarded: PhD department: - _id: GradSch - _id: VlKo doi: 10.15479/AT-ISTA-21957 file: - access_level: closed checksum: b11a959e99d3dcf61040282b5c837141 content_type: application/zip creator: jzapata date_created: 2026-06-08T13:20:02Z date_updated: 2026-06-08T13:20:02Z file_id: '21958' file_name: istaustriathesis_JZapata.zip file_size: 40811933 relation: source_file - access_level: open_access checksum: edf1e5899b2e31505cd1aa3fe8bd4b7f content_type: application/pdf creator: jzapata date_created: 2026-06-10T13:33:25Z date_updated: 2026-06-10T13:33:25Z file_id: '21992' file_name: 4_Final_Thesis_JZapata_REX.pdf file_size: 2207892 relation: main_file success: 1 file_date_updated: 2026-06-10T13:33:25Z has_accepted_license: '1' language: - iso: eng month: '06' oa: 1 oa_version: Published Version page: '89' project: - _id: 9B9290DE-BA93-11EA-9121-9846C619BF3A grant_number: W1260-N35 name: Vienna Graduate School on Computational Optimization publication_identifier: isbn: - 978-3-99078-079-4 issn: - 2663-337X publication_status: published publisher: Institute of Science and Technology Austria related_material: record: - id: '21144' relation: part_of_dissertation status: public status: public supervisor: - first_name: Vladimir full_name: Kolmogorov, Vladimir id: 3D50B0BA-F248-11E8-B48F-1D18A9856A87 last_name: Kolmogorov title: 'Overcoming degeneracy and singularity : Techniques for semidefinite programs and homotopy continuation endgames' tmp: image: /images/cc_by.png legal_code_url: https://creativecommons.org/licenses/by/4.0/legalcode name: Creative Commons Attribution 4.0 International Public License (CC-BY 4.0) short: CC BY (4.0) type: dissertation user_id: 8b945eb4-e2f2-11eb-945a-df72226e66a9 year: '2026' ... --- OA_place: repository OA_type: green _id: '21858' abstract: - lang: eng text: "The recent surge in high-quality open-source Generative AI text models (colloquially: LLMs), as well as efficient finetuning techniques, have opened the possibility of creating high-quality personalized models that generate text attuned to a specific individual’s needs and are capable of credibly imitating their writing style by refining an open-source model with that person’s own data. The technology to create such models is accessible to private individuals, and training and running such models can be done cheaply on consumer-grade hardware. While these advancements are a huge gain for usability and privacy, this position paper argues that the practical feasibility of impersonating specific individuals also introduces novel safety risks. For instance, this technology enables the creation of phishing emails\r\nor fraudulent social media accounts, based on small amounts of publicly available text, or by the individuals themselves to escape AI text detection. We further argue that these risks are complementary to—and distinct from—the much-discussed risks of other impersonation attacks such as image, voice, or video deepfakes, and are not adequately addressed by the larger research community, or the current generation of open- and closed-source models." acknowledgement: "This research was supported by the Scientific Service Units (SSU) of IST Austria through resources\r\nprovided by Scientific Computing (SciComp). EI was supported in part by the FWF DK VGSCO,\r\ngrant agreement number W1260-N35. AJ was supported in part by ERC Proof-of-Concept Grant\r\nFastML, grant agreement 101158077." article_processing_charge: No arxiv: 1 author: - first_name: Eugenia B full_name: Iofinova, Eugenia B id: f9a17499-f6e0-11ea-865d-fdf9a3f77117 last_name: Iofinova orcid: 0000-0002-7778-3221 - first_name: Andrej full_name: Jovanovic, Andrej last_name: Jovanovic - first_name: Dan-Adrian full_name: Alistarh, Dan-Adrian id: 4A899BFC-F248-11E8-B48F-1D18A9856A87 last_name: Alistarh orcid: 0000-0003-3650-940X citation: ama: 'Iofinova EB, Jovanovic A, Alistarh D-A. Position: It’s time to act on the risk of efficient personalized text generation. arXiv. doi:10.48550/arXiv.2502.06560' apa: 'Iofinova, E. B., Jovanovic, A., & Alistarh, D.-A. (n.d.). Position: It’s time to act on the risk of efficient personalized text generation. arXiv. https://doi.org/10.48550/arXiv.2502.06560' chicago: 'Iofinova, Eugenia B, Andrej Jovanovic, and Dan-Adrian Alistarh. “Position: It’s Time to Act on the Risk of Efficient Personalized Text Generation.” ArXiv, n.d. https://doi.org/10.48550/arXiv.2502.06560.' ieee: 'E. B. Iofinova, A. Jovanovic, and D.-A. Alistarh, “Position: It’s time to act on the risk of efficient personalized text generation,” arXiv. .' ista: 'Iofinova EB, Jovanovic A, Alistarh D-A. Position: It’s time to act on the risk of efficient personalized text generation. arXiv, 10.48550/arXiv.2502.06560.' mla: 'Iofinova, Eugenia B., et al. “Position: It’s Time to Act on the Risk of Efficient Personalized Text Generation.” ArXiv, doi:10.48550/arXiv.2502.06560.' short: E.B. Iofinova, A. Jovanovic, D.-A. Alistarh, ArXiv (n.d.). corr_author: '1' date_created: 2026-05-11T08:55:23Z date_published: 2025-06-02T00:00:00Z date_updated: 2026-05-19T11:20:27Z day: '02' department: - _id: GradSch - _id: DaAl doi: 10.48550/arXiv.2502.06560 external_id: arxiv: - '2502.06560' language: - iso: eng main_file_link: - open_access: '1' url: https://doi.org/10.48550/arXiv.2502.06560 month: '06' oa: 1 oa_version: Preprint project: - _id: 8e35c14b-16d5-11f0-9cad-a3fc35339161 grant_number: '101158077' name: 'FastML: Efficient and Cost-Effective Distributed Machine Learning' - _id: 9B9290DE-BA93-11EA-9121-9846C619BF3A grant_number: W1260-N35 name: Vienna Graduate School on Computational Optimization publication: arXiv publication_status: draft related_material: record: - id: '21854' relation: dissertation_contains status: public status: public title: 'Position: It''s time to act on the risk of efficient personalized text generation' type: preprint user_id: 8b945eb4-e2f2-11eb-945a-df72226e66a9 year: '2025' ... --- OA_place: publisher OA_type: gold _id: '20006' abstract: - lang: eng text: In numerous fields, dynamic time series data require continuous updates, necessitating efficient data processing techniques for accurate analysis. This paper examines the banana tree data structure, specifically designed to efficiently maintain the multi-scale topological descriptor commonly known as persistent homology for dynamically changing time series data. We implement this data structure and conduct an experimental study to assess its properties and runtime for update operations. Our findings indicate that banana trees are highly effective with unbiased random data, outperforming state-of-the-art static algorithms in these scenarios. Additionally, our results show that real-world time series share structural properties with unbiased random walks, suggesting potential practical utility for our implementation. acknowledgement: "Lara Ost: Supported by the Vienna Graduate School on Computational Optimization\r\n(VGSCO), FWF project no. W1260-N35.\r\nSebastiano Cultrera di Montesano: Supported by the Eric and Wendy Schmidt Center at the Broad Institute of MIT and Harvard.\r\nHerbert Edelsbrunner: Partially supported by the Wittgenstein Prize, FWF grant no. Z 342-N31,\r\nand by the DFG Collaborative Research Center TRR 109, FWF grant no. I 02979-N35." alternative_title: - LIPIcs article_number: '71' article_processing_charge: Yes arxiv: 1 author: - first_name: Lara full_name: Ost, Lara last_name: Ost - first_name: Sebastiano full_name: Cultrera di Montesano, Sebastiano id: 34D2A09C-F248-11E8-B48F-1D18A9856A87 last_name: Cultrera di Montesano orcid: 0000-0001-6249-0832 - first_name: Herbert full_name: Edelsbrunner, Herbert id: 3FB178DA-F248-11E8-B48F-1D18A9856A87 last_name: Edelsbrunner orcid: 0000-0002-9823-6833 citation: ama: 'Ost L, Cultrera di Montesano S, Edelsbrunner H. Banana trees for the persistence in time series experimentally. In: 41st International Symposium on Computational Geometry. Vol 332. Schloss Dagstuhl - Leibniz-Zentrum für Informatik; 2025. doi:10.4230/LIPIcs.SoCG.2025.71' apa: 'Ost, L., Cultrera di Montesano, S., & Edelsbrunner, H. (2025). Banana trees for the persistence in time series experimentally. In 41st International Symposium on Computational Geometry (Vol. 332). Kanazawa, Japan: Schloss Dagstuhl - Leibniz-Zentrum für Informatik. https://doi.org/10.4230/LIPIcs.SoCG.2025.71' chicago: Ost, Lara, Sebastiano Cultrera di Montesano, and Herbert Edelsbrunner. “Banana Trees for the Persistence in Time Series Experimentally.” In 41st International Symposium on Computational Geometry, Vol. 332. Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2025. https://doi.org/10.4230/LIPIcs.SoCG.2025.71. ieee: L. Ost, S. Cultrera di Montesano, and H. Edelsbrunner, “Banana trees for the persistence in time series experimentally,” in 41st International Symposium on Computational Geometry, Kanazawa, Japan, 2025, vol. 332. ista: 'Ost L, Cultrera di Montesano S, Edelsbrunner H. 2025. Banana trees for the persistence in time series experimentally. 41st International Symposium on Computational Geometry. SoCG: Symposium on Computational Geometry, LIPIcs, vol. 332, 71.' mla: Ost, Lara, et al. “Banana Trees for the Persistence in Time Series Experimentally.” 41st International Symposium on Computational Geometry, vol. 332, 71, Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2025, doi:10.4230/LIPIcs.SoCG.2025.71. short: L. Ost, S. Cultrera di Montesano, H. Edelsbrunner, in:, 41st International Symposium on Computational Geometry, Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 2025. conference: end_date: 2025-06-27 location: Kanazawa, Japan name: 'SoCG: Symposium on Computational Geometry' start_date: 2025-06-23 corr_author: '1' date_created: 2025-07-13T22:01:22Z date_published: 2025-06-20T00:00:00Z date_updated: 2025-12-30T11:04:33Z day: '20' ddc: - '000' department: - _id: HeEd doi: 10.4230/LIPIcs.SoCG.2025.71 external_id: arxiv: - '2405.17920' file: - access_level: open_access checksum: 3a4a7a707a56e0cfdf51428782dee55a content_type: application/pdf creator: dernst date_created: 2025-07-14T08:23:38Z date_updated: 2025-07-14T08:23:38Z file_id: '20017' file_name: 2025_LIPIcs.SoCG_Ost.pdf file_size: 834623 relation: main_file success: 1 file_date_updated: 2025-07-14T08:23:38Z has_accepted_license: '1' intvolume: ' 332' language: - iso: eng month: '06' oa: 1 oa_version: Published Version project: - _id: 9B9290DE-BA93-11EA-9121-9846C619BF3A grant_number: W1260-N35 name: Vienna Graduate School on Computational Optimization - _id: 2561EBF4-B435-11E9-9278-68D0E5697425 call_identifier: FWF grant_number: I02979-N35 name: Persistence and stability of geometric complexes - _id: 268116B8-B435-11E9-9278-68D0E5697425 call_identifier: FWF grant_number: Z00342 name: Mathematics, Computer Science publication: 41st International Symposium on Computational Geometry publication_identifier: eissn: - 1868-8969 isbn: - '9783959773706' publication_status: published publisher: Schloss Dagstuhl - Leibniz-Zentrum für Informatik quality_controlled: '1' related_material: link: - relation: software url: https://github.com/laraost/BananaPersist scopus_import: '1' status: public title: Banana trees for the persistence in time series experimentally tmp: image: /images/cc_by.png legal_code_url: https://creativecommons.org/licenses/by/4.0/legalcode name: Creative Commons Attribution 4.0 International Public License (CC-BY 4.0) short: CC BY (4.0) type: conference user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87 volume: 332 year: '2025' ... --- _id: '18121' abstract: - lang: eng text: It is known that sparsity can improve interpretability for deep neural networks. However, existing methods in the area either require networks that are pre-trained with sparsity constraints, or impose sparsity after the fact, altering the network’s general behavior. In this paper, we demonstrate, for the first time, that sparsity can instead be incorporated into the interpretation process itself, as a sample-specific preprocessing step. Unlike previous work, this approach, which we call SPADE, does not place constraints on the trained model and does not affect its behavior during inference on the sample. Given a trained model and a target sample, SPADE uses sample-targeted pruning to provide a "trace" of the network’s execution on the sample, reducing the network to the most important connections prior to computing an interpretation. We demonstrate that preprocessing with SPADE significantly increases the accuracy of image saliency maps across several interpretability methods. Additionally, SPADE improves the usefulness of neuron visualizations, aiding humans in reasoning about network behavior. Our code is available at https://github.com/IST-DASLab/SPADE. acknowledged_ssus: - _id: ScienComp acknowledgement: The authors would like to thank Stephen Casper and Tony Wang for their feedback on this work, and Eldar Kurtic for his advice on aspects of the project. This research was supported by the Scientific Service Units (SSU) of IST Austria through resources provided by Scientific Computing (SciComp). EI was supported in part by the FWF DK VGSCO, grant agreement number W1260-N35. alternative_title: - PMLR article_processing_charge: No arxiv: 1 author: - first_name: Arshia Soltani full_name: Moakhar, Arshia Soltani last_name: Moakhar - first_name: Eugenia B full_name: Iofinova, Eugenia B id: f9a17499-f6e0-11ea-865d-fdf9a3f77117 last_name: Iofinova orcid: 0000-0002-7778-3221 - first_name: Elias full_name: Frantar, Elias id: 09a8f98d-ec99-11ea-ae11-c063a7b7fe5f last_name: Frantar - first_name: Dan-Adrian full_name: Alistarh, Dan-Adrian id: 4A899BFC-F248-11E8-B48F-1D18A9856A87 last_name: Alistarh orcid: 0000-0003-3650-940X citation: ama: 'Moakhar AS, Iofinova EB, Frantar E, Alistarh D-A. SPADE: Sparsity-guided debugging for deep neural networks. In: Proceedings of the 41st International Conference on Machine Learning. Vol 235. ML Research Press; 2024:45955-45987.' apa: 'Moakhar, A. S., Iofinova, E. B., Frantar, E., & Alistarh, D.-A. (2024). SPADE: Sparsity-guided debugging for deep neural networks. In Proceedings of the 41st International Conference on Machine Learning (Vol. 235, pp. 45955–45987). Vienna, Austria: ML Research Press.' chicago: 'Moakhar, Arshia Soltani, Eugenia B Iofinova, Elias Frantar, and Dan-Adrian Alistarh. “SPADE: Sparsity-Guided Debugging for Deep Neural Networks.” In Proceedings of the 41st International Conference on Machine Learning, 235:45955–87. ML Research Press, 2024.' ieee: 'A. S. Moakhar, E. B. Iofinova, E. Frantar, and D.-A. Alistarh, “SPADE: Sparsity-guided debugging for deep neural networks,” in Proceedings of the 41st International Conference on Machine Learning, Vienna, Austria, 2024, vol. 235, pp. 45955–45987.' ista: 'Moakhar AS, Iofinova EB, Frantar E, Alistarh D-A. 2024. SPADE: Sparsity-guided debugging for deep neural networks. Proceedings of the 41st International Conference on Machine Learning. ICML: International Conference on Machine Learning, PMLR, vol. 235, 45955–45987.' mla: 'Moakhar, Arshia Soltani, et al. “SPADE: Sparsity-Guided Debugging for Deep Neural Networks.” Proceedings of the 41st International Conference on Machine Learning, vol. 235, ML Research Press, 2024, pp. 45955–87.' short: A.S. Moakhar, E.B. Iofinova, E. Frantar, D.-A. Alistarh, in:, Proceedings of the 41st International Conference on Machine Learning, ML Research Press, 2024, pp. 45955–45987. conference: end_date: 2024-07-27 location: Vienna, Austria name: 'ICML: International Conference on Machine Learning' start_date: 2024-07-21 corr_author: '1' date_created: 2024-09-22T22:01:46Z date_published: 2024-09-01T00:00:00Z date_updated: 2026-05-19T11:20:27Z day: '01' department: - _id: DaAl external_id: arxiv: - '2310.04519' intvolume: ' 235' language: - iso: eng main_file_link: - open_access: '1' url: https://doi.org/10.48550/arXiv.2310.04519 month: '09' oa: 1 oa_version: Preprint page: 45955-45987 project: - _id: 9B9290DE-BA93-11EA-9121-9846C619BF3A grant_number: W1260-N35 name: Vienna Graduate School on Computational Optimization publication: Proceedings of the 41st International Conference on Machine Learning publication_identifier: eissn: - 2640-3498 publication_status: published publisher: ML Research Press quality_controlled: '1' related_material: link: - relation: software url: https://github.com/IST-DASLab/SPADE record: - id: '21854' relation: dissertation_contains status: public scopus_import: '1' status: public title: 'SPADE: Sparsity-guided debugging for deep neural networks' type: conference user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87 volume: 235 year: '2024' ... --- OA_place: repository _id: '17465' abstract: - lang: eng text: "In the modern age of machine learning, artificial neural networks have become an integral part\r\nof many practical systems. One of the key ingredients of the success of the deep learning\r\napproach is recent computational advances which allowed the training of models with billions\r\nof parameters on large-scale data. Such over-parameterized and data-hungry regimes pose a\r\nchallenge for the theoretical analysis of modern models since “classical” statistical wisdom\r\nis no longer applicable. In this view, it is paramount to extend or develop new machinery\r\nthat will allow tackling the neural network analysis under new challenging asymptotic regimes,\r\nwhich is the focus of this thesis.\r\nLarge neural network systems are usually optimized via “local” search algorithms, such\r\nas stochastic gradient descent (SGD). However, given the high-dimensional nature of the\r\nparameter space, it is a priori not clear why such a crude “local” approach works so remarkably\r\nwell in practice. We take a step towards demystifying this phenomenon by showing that\r\nthe landscape of the SGD training dynamics exhibits a few beneficial properties for the\r\noptimization. First, we show that along the SGD trajectory an over-parameterized network\r\nis dropout stable. The emergence of dropout stability allows to conclude that the minima\r\nfound by SGD are connected via a continuous path of small loss. This in turn means that\r\nthe high-dimensional landscape of the neural network optimization problem is provably not so\r\nunfavourable to gradient-based training, due to mode connectivity. Next, we show that SGD\r\nfor an over-parameterized network tends to find solutions that are functionally more “simple”.\r\nThis in turn means that the SGD minima are more robust, since a less complicated solution\r\nwill less likely overfit the data. More formally, for a prototypical example of a wide two-layer\r\nReLU network on a 1d regression task we show that the SGD algorithm is implicitly selective in\r\nits choice of an interpolating solution. Namely, at convergence the neural network implements\r\na piece-wise linear function with the number of linear regions depending only on the amount\r\nof training data. This is in contrast to a “smooth”-like behaviour which one would expect\r\ngiven such a severe over-parameterization of the model.\r\nDiverging from the generic supervised setting of classification and regression problems, we\r\nanalyze an auto-encoder model that is commonly used for representation learning and data\r\ncompression. Despite the wide applicability of the auto-encoding paradigm, the theoretical\r\nunderstanding of their behaviour is limited even in the simplistic shallow case. The related\r\nwork is restricted to extreme asymptotic regimes in which the auto-encoder is either severely\r\nover-parameterized or under-parameterized. In contrast, we provide a tight characterization\r\nfor the 1-bit compression of Gaussian signals in the challenging proportional regime, i.e., the\r\ninput dimension and the size of the compressed representation obey the same asymptotics.\r\nWe also show that gradient-based methods are able to find a globally optimal solution and\r\nthat the predictions made for Gaussian data extrapolate beyond - to the case of compression\r\nof natural images. Next, we relax the Gaussian assumption and study more structured input\r\nsources. We show that the shallow model is sometimes agnostic to the structure of the data\r\nvii\r\nwhich results in a Gaussian-like behaviour. We prove that making the decoding component\r\nslightly less shallow is already enough to escape the “curse” of Gaussian performance.\r\n" acknowledged_ssus: - _id: ScienComp alternative_title: - ISTA Thesis article_processing_charge: No author: - first_name: Aleksandr full_name: Shevchenko, Aleksandr id: F2B06EC2-C99E-11E9-89F0-752EE6697425 last_name: Shevchenko citation: ama: Shevchenko A. High-dimensional limits in artificial neural networks. 2024. doi:10.15479/at:ista:17465 apa: Shevchenko, A. (2024). High-dimensional limits in artificial neural networks. Institute of Science and Technology Austria. https://doi.org/10.15479/at:ista:17465 chicago: Shevchenko, Alexander. “High-Dimensional Limits in Artificial Neural Networks.” Institute of Science and Technology Austria, 2024. https://doi.org/10.15479/at:ista:17465. ieee: A. Shevchenko, “High-dimensional limits in artificial neural networks,” Institute of Science and Technology Austria, 2024. ista: Shevchenko A. 2024. High-dimensional limits in artificial neural networks. Institute of Science and Technology Austria. mla: Shevchenko, Alexander. High-Dimensional Limits in Artificial Neural Networks. Institute of Science and Technology Austria, 2024, doi:10.15479/at:ista:17465. short: A. Shevchenko, High-Dimensional Limits in Artificial Neural Networks, Institute of Science and Technology Austria, 2024. corr_author: '1' date_created: 2024-08-28T15:14:25Z date_published: 2024-08-29T00:00:00Z date_updated: 2026-06-18T17:55:53Z day: '29' ddc: - '519' degree_awarded: PhD department: - _id: GradSch - _id: DaAl - _id: MaMo doi: 10.15479/at:ista:17465 file: - access_level: open_access checksum: da6dd3166078934577f6af93d27000e2 content_type: application/pdf creator: ashevche date_created: 2024-09-02T09:23:32Z date_updated: 2024-10-05T22:30:05Z embargo: 2024-10-04 file_id: '17482' file_name: thesis_a2b.pdf file_size: 4468610 relation: main_file - access_level: closed checksum: 76a39ef252239560923cdda4ce0a31a4 content_type: application/zip creator: ashevche date_created: 2024-09-02T09:23:46Z date_updated: 2024-10-05T22:30:05Z embargo_to: open_access file_id: '17483' file_name: Thesis Alex - ISTA.zip file_size: 15930999 relation: source_file file_date_updated: 2024-10-05T22:30:05Z has_accepted_license: '1' language: - iso: eng month: '08' oa: 1 oa_version: Published Version page: '232' project: - _id: 059876FA-7A3F-11EA-A408-12923DDC885E name: Prix Lopez-Loretta 2019 - Marco Mondelli - _id: 9B9290DE-BA93-11EA-9121-9846C619BF3A grant_number: W1260-N35 name: Vienna Graduate School on Computational Optimization publication_identifier: issn: - 2663-337X publication_status: published publisher: Institute of Science and Technology Austria related_material: record: - id: '11420' relation: part_of_dissertation status: public - id: '14459' relation: part_of_dissertation status: public - id: '9198' relation: part_of_dissertation status: public - id: '17469' relation: part_of_dissertation status: public status: public supervisor: - first_name: Marco full_name: Mondelli, Marco id: 27EB676C-8706-11E9-9510-7717E6697425 last_name: Mondelli orcid: 0000-0002-3242-7020 - first_name: Dan-Adrian full_name: Alistarh, Dan-Adrian id: 4A899BFC-F248-11E8-B48F-1D18A9856A87 last_name: Alistarh orcid: 0000-0003-3650-940X title: High-dimensional limits in artificial neural networks type: dissertation user_id: 8b945eb4-e2f2-11eb-945a-df72226e66a9 year: '2024' ... --- _id: '14771' abstract: - lang: eng text: Pruning—that is, setting a significant subset of the parameters of a neural network to zero—is one of the most popular methods of model compression. Yet, several recent works have raised the issue that pruning may induce or exacerbate bias in the output of the compressed model. Despite existing evidence for this phenomenon, the relationship between neural network pruning and induced bias is not well-understood. In this work, we systematically investigate and characterize this phenomenon in Convolutional Neural Networks for computer vision. First, we show that it is in fact possible to obtain highly-sparse models, e.g. with less than 10% remaining weights, which do not decrease in accuracy nor substantially increase in bias when compared to dense models. At the same time, we also find that, at higher sparsities, pruned models exhibit higher uncertainty in their outputs, as well as increased correlations, which we directly link to increased bias. We propose easy-to-use criteria which, based only on the uncompressed model, establish whether bias will increase with pruning, and identify the samples most susceptible to biased predictions post-compression. Our code can be found at https://github.com/IST-DASLab/pruned-vision-model-bias. acknowledgement: The authors would like to sincerely thank Sara Hooker for her feedback during the development of this work. EI was supported in part by the FWF DK VGSCO, grant agreement number W1260-N35. AP and DA acknowledge generous ERC support, via Starting Grant 805223 ScaleML. article_processing_charge: No arxiv: 1 author: - first_name: Eugenia B full_name: Iofinova, Eugenia B id: f9a17499-f6e0-11ea-865d-fdf9a3f77117 last_name: Iofinova orcid: 0000-0002-7778-3221 - first_name: Elena-Alexandra full_name: Peste, Elena-Alexandra id: 32D78294-F248-11E8-B48F-1D18A9856A87 last_name: Peste - first_name: Dan-Adrian full_name: Alistarh, Dan-Adrian id: 4A899BFC-F248-11E8-B48F-1D18A9856A87 last_name: Alistarh orcid: 0000-0003-3650-940X citation: ama: 'Iofinova EB, Krumes A, Alistarh D-A. Bias in pruned vision models: In-depth analysis and countermeasures. In: 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE; 2023:24364-24373. doi:10.1109/cvpr52729.2023.02334' apa: 'Iofinova, E. B., Krumes, A., & Alistarh, D.-A. (2023). Bias in pruned vision models: In-depth analysis and countermeasures. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 24364–24373). Vancouver, BC, Canada: IEEE. https://doi.org/10.1109/cvpr52729.2023.02334' chicago: 'Iofinova, Eugenia B, Alexandra Krumes, and Dan-Adrian Alistarh. “Bias in Pruned Vision Models: In-Depth Analysis and Countermeasures.” In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 24364–73. IEEE, 2023. https://doi.org/10.1109/cvpr52729.2023.02334.' ieee: 'E. B. Iofinova, A. Krumes, and D.-A. Alistarh, “Bias in pruned vision models: In-depth analysis and countermeasures,” in 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada, 2023, pp. 24364–24373.' ista: 'Iofinova EB, Krumes A, Alistarh D-A. 2023. Bias in pruned vision models: In-depth analysis and countermeasures. 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. CVPR: Conference on Computer Vision and Pattern Recognition, 24364–24373.' mla: 'Iofinova, Eugenia B., et al. “Bias in Pruned Vision Models: In-Depth Analysis and Countermeasures.” 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE, 2023, pp. 24364–73, doi:10.1109/cvpr52729.2023.02334.' short: E.B. Iofinova, A. Krumes, D.-A. Alistarh, in:, 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE, 2023, pp. 24364–24373. conference: end_date: 2023-06-24 location: Vancouver, BC, Canada name: 'CVPR: Conference on Computer Vision and Pattern Recognition' start_date: 2023-06-17 corr_author: '1' date_created: 2024-01-10T08:42:40Z date_published: 2023-08-22T00:00:00Z date_updated: 2026-05-19T11:20:27Z day: '22' department: - _id: DaAl - _id: ChLa doi: 10.1109/cvpr52729.2023.02334 ec_funded: 1 external_id: arxiv: - '2304.12622' isi: - '001062531308068' isi: 1 language: - iso: eng main_file_link: - open_access: '1' url: https://doi.org/10.48550/arXiv.2304.12622 month: '08' oa: 1 oa_version: Preprint page: 24364-24373 project: - _id: 9B9290DE-BA93-11EA-9121-9846C619BF3A grant_number: W1260-N35 name: Vienna Graduate School on Computational Optimization - _id: 268A44D6-B435-11E9-9278-68D0E5697425 call_identifier: H2020 grant_number: '805223' name: Elastic Coordination for Scalable Machine Learning publication: 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition publication_identifier: eisbn: - '9798350301298' eissn: - 2575-7075 publication_status: published publisher: IEEE quality_controlled: '1' related_material: link: - relation: software url: https://github.com/IST-DASLab/pruned-vision-model-bias record: - id: '21854' relation: dissertation_contains status: public status: public title: 'Bias in pruned vision models: In-depth analysis and countermeasures' type: conference user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87 year: '2023' ... --- _id: '12299' abstract: - lang: eng text: 'Transfer learning is a classic paradigm by which models pretrained on large “upstream” datasets are adapted to yield good results on “downstream” specialized datasets. Generally, more accurate models on the “upstream” dataset tend to provide better transfer accuracy “downstream”. In this work, we perform an in-depth investigation of this phenomenon in the context of convolutional neural networks (CNNs) trained on the ImageNet dataset, which have been pruned-that is, compressed by sparsifiying their connections. We consider transfer using unstructured pruned models obtained by applying several state-of-the-art pruning methods, including magnitude-based, second-order, regrowth, lottery-ticket, and regularization approaches, in the context of twelve standard transfer tasks. In a nutshell, our study shows that sparse models can match or even outperform the transfer performance of dense models, even at high sparsities, and, while doing so, can lead to significant inference and even training speedups. At the same time, we observe and analyze significant differences in the behaviour of different pruning methods. The code is available at: https://github.com/IST-DASLab/sparse-imagenet-transfer.' acknowledgement: he authors would like to sincerely thank Christoph Lampert and Nir Shavit for fruitful discussions during the development of this work, and Eldar Kurtic for experimental support. EI was supported in part by the FWF DK VGSCO, grant agreement number W1260-N35, while AP and DA acknowledge generous support by the ERC, via Starting Grant 805223 ScaleML. article_processing_charge: No arxiv: 1 author: - first_name: Eugenia B full_name: Iofinova, Eugenia B id: f9a17499-f6e0-11ea-865d-fdf9a3f77117 last_name: Iofinova orcid: 0000-0002-7778-3221 - first_name: Elena-Alexandra full_name: Peste, Elena-Alexandra id: 32D78294-F248-11E8-B48F-1D18A9856A87 last_name: Peste - first_name: Mark full_name: Kurtz, Mark last_name: Kurtz - first_name: Dan-Adrian full_name: Alistarh, Dan-Adrian id: 4A899BFC-F248-11E8-B48F-1D18A9856A87 last_name: Alistarh orcid: 0000-0003-3650-940X citation: ama: 'Iofinova EB, Krumes A, Kurtz M, Alistarh D-A. How well do sparse ImageNet models transfer? In: 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Institute of Electrical and Electronics Engineers; 2022:12256-12266. doi:10.1109/cvpr52688.2022.01195' apa: 'Iofinova, E. B., Krumes, A., Kurtz, M., & Alistarh, D.-A. (2022). How well do sparse ImageNet models transfer? In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 12256–12266). New Orleans, LA, United States: Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/cvpr52688.2022.01195' chicago: Iofinova, Eugenia B, Alexandra Krumes, Mark Kurtz, and Dan-Adrian Alistarh. “How Well Do Sparse ImageNet Models Transfer?” In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 12256–66. Institute of Electrical and Electronics Engineers, 2022. https://doi.org/10.1109/cvpr52688.2022.01195. ieee: E. B. Iofinova, A. Krumes, M. Kurtz, and D.-A. Alistarh, “How well do sparse ImageNet models transfer?,” in 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, United States, 2022, pp. 12256–12266. ista: 'Iofinova EB, Krumes A, Kurtz M, Alistarh D-A. 2022. How well do sparse ImageNet models transfer? 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. CVPR: Computer Vision and Pattern Recognition, 12256–12266.' mla: Iofinova, Eugenia B., et al. “How Well Do Sparse ImageNet Models Transfer?” 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Institute of Electrical and Electronics Engineers, 2022, pp. 12256–66, doi:10.1109/cvpr52688.2022.01195. short: E.B. Iofinova, A. Krumes, M. Kurtz, D.-A. Alistarh, in:, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Institute of Electrical and Electronics Engineers, 2022, pp. 12256–12266. conference: end_date: 2022-06-24 location: New Orleans, LA, United States name: 'CVPR: Computer Vision and Pattern Recognition' start_date: 2022-06-18 corr_author: '1' date_created: 2023-01-16T10:06:00Z date_published: 2022-09-27T00:00:00Z date_updated: 2026-04-07T13:30:19Z day: '27' department: - _id: DaAl - _id: ChLa doi: 10.1109/cvpr52688.2022.01195 ec_funded: 1 external_id: arxiv: - '2111.13445' isi: - '000870759105034' isi: 1 language: - iso: eng main_file_link: - open_access: '1' url: https://doi.org/10.48550/arXiv.2111.13445 month: '09' oa: 1 oa_version: Preprint page: 12256-12266 project: - _id: 9B9290DE-BA93-11EA-9121-9846C619BF3A grant_number: W1260-N35 name: Vienna Graduate School on Computational Optimization - _id: 268A44D6-B435-11E9-9278-68D0E5697425 call_identifier: H2020 grant_number: '805223' name: Elastic Coordination for Scalable Machine Learning publication: 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition publication_identifier: eissn: - 2575-7075 publication_status: published publisher: Institute of Electrical and Electronics Engineers quality_controlled: '1' related_material: record: - id: '13074' relation: dissertation_contains status: public scopus_import: '1' status: public title: How well do sparse ImageNet models transfer? type: conference user_id: 4359f0d1-fa6c-11eb-b949-802e58b17ae8 year: '2022' ... --- _id: '12495' abstract: - lang: eng text: "Fairness-aware learning aims at constructing classifiers that not only make accurate predictions, but also do not discriminate against specific groups. It is a fast-growing area of\r\nmachine learning with far-reaching societal impact. However, existing fair learning methods\r\nare vulnerable to accidental or malicious artifacts in the training data, which can cause\r\nthem to unknowingly produce unfair classifiers. In this work we address the problem of\r\nfair learning from unreliable training data in the robust multisource setting, where the\r\navailable training data comes from multiple sources, a fraction of which might not be representative of the true data distribution. We introduce FLEA, a filtering-based algorithm\r\nthat identifies and suppresses those data sources that would have a negative impact on\r\nfairness or accuracy if they were used for training. As such, FLEA is not a replacement of\r\nprior fairness-aware learning methods but rather an augmentation that makes any of them\r\nrobust against unreliable training data. We show the effectiveness of our approach by a\r\ndiverse range of experiments on multiple datasets. Additionally, we prove formally that\r\n–given enough data– FLEA protects the learner against corruptions as long as the fraction of\r\naffected data sources is less than half. Our source code and documentation are available at\r\nhttps://github.com/ISTAustria-CVML/FLEA." acknowledged_ssus: - _id: ScienComp acknowledgement: 'The authors would like to thank Bernd Prach, Elias Frantar, Alexandra Peste, Mahdi Nikdan, and Peter Súkeník for their helpful feedback. This research was supported by the Scientific Service Units (SSU) of IST Austria through resources provided by Scientific Computing (SciComp). This publication was made possible by an ETH AI Center postdoctoral fellowship granted to Nikola Konstantinov. Eugenia Iofinova was supported in part by the FWF DK VGSCO, grant agreement number W1260-N35. ' article_processing_charge: No article_type: original arxiv: 1 author: - first_name: Eugenia B full_name: Iofinova, Eugenia B id: f9a17499-f6e0-11ea-865d-fdf9a3f77117 last_name: Iofinova orcid: 0000-0002-7778-3221 - first_name: Nikola H full_name: Konstantinov, Nikola H id: 4B9D76E4-F248-11E8-B48F-1D18A9856A87 last_name: Konstantinov orcid: 0009-0009-5204-7621 - first_name: Christoph full_name: Lampert, Christoph id: 40C20FD2-F248-11E8-B48F-1D18A9856A87 last_name: Lampert orcid: 0000-0001-8622-7887 citation: ama: 'Iofinova EB, Konstantinov NH, Lampert C. FLEA: Provably robust fair multisource learning from unreliable training data. Transactions on Machine Learning Research. 2022.' apa: 'Iofinova, E. B., Konstantinov, N. H., & Lampert, C. (2022). FLEA: Provably robust fair multisource learning from unreliable training data. Transactions on Machine Learning Research. ML Research Press.' chicago: 'Iofinova, Eugenia B, Nikola H Konstantinov, and Christoph Lampert. “FLEA: Provably Robust Fair Multisource Learning from Unreliable Training Data.” Transactions on Machine Learning Research. ML Research Press, 2022.' ieee: 'E. B. Iofinova, N. H. Konstantinov, and C. Lampert, “FLEA: Provably robust fair multisource learning from unreliable training data,” Transactions on Machine Learning Research. ML Research Press, 2022.' ista: 'Iofinova EB, Konstantinov NH, Lampert C. 2022. FLEA: Provably robust fair multisource learning from unreliable training data. Transactions on Machine Learning Research.' mla: 'Iofinova, Eugenia B., et al. “FLEA: Provably Robust Fair Multisource Learning from Unreliable Training Data.” Transactions on Machine Learning Research, ML Research Press, 2022.' short: E.B. Iofinova, N.H. Konstantinov, C. Lampert, Transactions on Machine Learning Research (2022). corr_author: '1' date_created: 2023-02-02T20:29:57Z date_published: 2022-12-22T00:00:00Z date_updated: 2025-12-30T11:04:31Z day: '22' ddc: - '000' department: - _id: ChLa external_id: arxiv: - '2106.11732' file: - access_level: open_access checksum: 97c8a8470759cab597abb973ca137a3b content_type: application/pdf creator: dernst date_created: 2023-02-23T10:30:04Z date_updated: 2023-02-23T10:30:04Z file_id: '12673' file_name: 2022_TMLR_Iofinova.pdf file_size: 1948063 relation: main_file success: 1 file_date_updated: 2023-02-23T10:30:04Z has_accepted_license: '1' language: - iso: eng main_file_link: - open_access: '1' url: https://openreview.net/forum?id=XsPopigZXV month: '12' oa: 1 oa_version: Published Version project: - _id: 9B9290DE-BA93-11EA-9121-9846C619BF3A grant_number: W1260-N35 name: Vienna Graduate School on Computational Optimization publication: Transactions on Machine Learning Research publication_identifier: issn: - 2835-8856 publication_status: published publisher: ML Research Press quality_controlled: '1' related_material: link: - description: source code relation: software url: https://github.com/ISTAustria-CVML/FLEA status: public title: 'FLEA: Provably robust fair multisource learning from unreliable training data' tmp: image: /images/cc_by.png legal_code_url: https://creativecommons.org/licenses/by/4.0/legalcode name: Creative Commons Attribution 4.0 International Public License (CC-BY 4.0) short: CC BY (4.0) type: journal_article user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87 year: '2022' ...