{"language":[{"iso":"eng"}],"ddc":["000"],"article_processing_charge":"No","month":"10","related_material":{"record":[{"status":"public","id":"9418","relation":"dissertation_contains"}]},"conference":{"start_date":"2019-10-27","location":"Seoul, Korea","end_date":"2019-11-02","name":"ICCV: International Conference on Computer Vision"},"publication_identifier":{"issn":["15505499"],"isbn":["9781728148038"]},"ec_funded":1,"type":"conference","citation":{"apa":"Phuong, M., & Lampert, C. (2019). Distillation-based training for multi-exit architectures. In IEEE International Conference on Computer Vision (Vol. 2019–October, pp. 1355–1364). Seoul, Korea: IEEE. https://doi.org/10.1109/ICCV.2019.00144","chicago":"Phuong, Mary, and Christoph Lampert. “Distillation-Based Training for Multi-Exit Architectures.” In IEEE International Conference on Computer Vision, 2019–October:1355–64. IEEE, 2019. https://doi.org/10.1109/ICCV.2019.00144.","ieee":"M. Phuong and C. Lampert, “Distillation-based training for multi-exit architectures,” in IEEE International Conference on Computer Vision, Seoul, Korea, 2019, vol. 2019–October, pp. 1355–1364.","ista":"Phuong M, Lampert C. 2019. Distillation-based training for multi-exit architectures. IEEE International Conference on Computer Vision. ICCV: International Conference on Computer Vision vol. 2019–October, 1355–1364.","mla":"Phuong, Mary, and Christoph Lampert. “Distillation-Based Training for Multi-Exit Architectures.” IEEE International Conference on Computer Vision, vol. 2019–October, IEEE, 2019, pp. 1355–64, doi:10.1109/ICCV.2019.00144.","short":"M. Phuong, C. Lampert, in:, IEEE International Conference on Computer Vision, IEEE, 2019, pp. 1355–1364.","ama":"Phuong M, Lampert C. Distillation-based training for multi-exit architectures. In: IEEE International Conference on Computer Vision. Vol 2019-October. IEEE; 2019:1355-1364. doi:10.1109/ICCV.2019.00144"},"file":[{"checksum":"7b77fb5c2d27c4c37a7612ba46a66117","date_created":"2020-02-11T09:06:39Z","file_id":"7480","content_type":"application/pdf","creator":"bphuong","relation":"main_file","access_level":"open_access","date_updated":"2020-07-14T12:47:59Z","file_size":735768,"file_name":"main.pdf"}],"abstract":[{"lang":"eng","text":"Multi-exit architectures, in which a stack of processing layers is interleaved with early output layers, allow the processing of a test example to stop early and thus save computation time and/or energy. In this work, we propose a new training procedure for multi-exit architectures based on the principle of knowledge distillation. The method encourage searly exits to mimic later, more accurate exits, by matching their output probabilities.\r\nExperiments on CIFAR100 and ImageNet show that distillation-based training significantly improves the accuracy of early exits while maintaining state-of-the-art accuracy for late ones. The method is particularly beneficial when training data is limited and it allows a straightforward extension to semi-supervised learning,i.e. making use of unlabeled data at training time. Moreover, it takes only afew lines to implement and incurs almost no computational overhead at training time, and none at all at test time."}],"date_created":"2020-02-11T09:06:57Z","year":"2019","publication":"IEEE International Conference on Computer Vision","date_updated":"2023-09-08T11:11:12Z","project":[{"name":"Lifelong Learning of Visual Scene Understanding","_id":"2532554C-B435-11E9-9278-68D0E5697425","grant_number":"308036","call_identifier":"FP7"}],"oa":1,"isi":1,"title":"Distillation-based training for multi-exit architectures","user_id":"c635000d-4b10-11ee-a964-aac5a93f6ac1","publication_status":"published","external_id":{"isi":["000531438101047"]},"publisher":"IEEE","page":"1355-1364","volume":"2019-October","file_date_updated":"2020-07-14T12:47:59Z","_id":"7479","quality_controlled":"1","department":[{"_id":"ChLa"}],"doi":"10.1109/ICCV.2019.00144","oa_version":"Submitted Version","author":[{"full_name":"Bui Thi Mai, Phuong","last_name":"Bui Thi Mai","id":"3EC6EE64-F248-11E8-B48F-1D18A9856A87","first_name":"Phuong"},{"full_name":"Lampert, Christoph","last_name":"Lampert","orcid":"0000-0001-8622-7887","id":"40C20FD2-F248-11E8-B48F-1D18A9856A87","first_name":"Christoph"}],"status":"public","has_accepted_license":"1","day":"01","date_published":"2019-10-01T00:00:00Z","scopus_import":"1"}