{"type":"preprint","extern":"1","oa_version":"Preprint","abstract":[{"lang":"eng","text":"In this paper, we show that recent advances in video representation learning\r\nand pre-trained vision-language models allow for substantial improvements in\r\nself-supervised video object localization. We propose a method that first\r\nlocalizes objects in videos via a slot attention approach and then assigns text\r\nto the obtained slots. The latter is achieved by an unsupervised way to read\r\nlocalized semantic information from the pre-trained CLIP model. The resulting\r\nvideo object localization is entirely unsupervised apart from the implicit\r\nannotation contained in CLIP, and it is effectively the first unsupervised\r\napproach that yields good results on regular video benchmarks."}],"publication_status":"submitted","date_published":"2023-09-18T00:00:00Z","status":"public","main_file_link":[{"url":"https://doi.org/10.48550/arXiv.2309.09858","open_access":"1"}],"year":"2023","article_processing_charge":"No","_id":"14962","doi":"10.48550/arXiv.2309.09858","external_id":{"arxiv":["2309.09858"]},"title":"Unsupervised open-vocabulary object localization in videos","user_id":"2DF688A6-F248-11E8-B48F-1D18A9856A87","department":[{"_id":"FrLo"}],"article_number":"2309.09858","date_created":"2024-02-08T15:33:39Z","citation":{"ieee":"K. Fan <i>et al.</i>, “Unsupervised open-vocabulary object localization in videos,” <i>arXiv</i>. .","mla":"Fan, Ke, et al. “Unsupervised Open-Vocabulary Object Localization in Videos.” <i>ArXiv</i>, 2309.09858, doi:<a href=\"https://doi.org/10.48550/arXiv.2309.09858\">10.48550/arXiv.2309.09858</a>.","chicago":"Fan, Ke, Zechen Bai, Tianjun Xiao, Dominik Zietlow, Max Horn, Zixu Zhao, Carl-Johann Simon-Gabriel Carl-Johann Simon-Gabriel, et al. “Unsupervised Open-Vocabulary Object Localization in Videos.” <i>ArXiv</i>, n.d. <a href=\"https://doi.org/10.48550/arXiv.2309.09858\">https://doi.org/10.48550/arXiv.2309.09858</a>.","apa":"Fan, K., Bai, Z., Xiao, T., Zietlow, D., Horn, M., Zhao, Z., … He, T. (n.d.). Unsupervised open-vocabulary object localization in videos. <i>arXiv</i>. <a href=\"https://doi.org/10.48550/arXiv.2309.09858\">https://doi.org/10.48550/arXiv.2309.09858</a>","short":"K. Fan, Z. Bai, T. Xiao, D. Zietlow, M. Horn, Z. Zhao, C.-J.S.-G. Carl-Johann Simon-Gabriel, M.Z. Shou, F. Locatello, B. Schiele, T. Brox, Z. Zhang, Y. Fu, T. He, ArXiv (n.d.).","ista":"Fan K, Bai Z, Xiao T, Zietlow D, Horn M, Zhao Z, Carl-Johann Simon-Gabriel C-JS-G, Shou MZ, Locatello F, Schiele B, Brox T, Zhang Z, Fu Y, He T. Unsupervised open-vocabulary object localization in videos. arXiv, 2309.09858.","ama":"Fan K, Bai Z, Xiao T, et al. Unsupervised open-vocabulary object localization in videos. <i>arXiv</i>. doi:<a href=\"https://doi.org/10.48550/arXiv.2309.09858\">10.48550/arXiv.2309.09858</a>"},"oa":1,"language":[{"iso":"eng"}],"day":"18","date_updated":"2024-02-12T10:12:22Z","month":"09","author":[{"first_name":"Ke","full_name":"Fan, Ke","last_name":"Fan"},{"last_name":"Bai","full_name":"Bai, Zechen","first_name":"Zechen"},{"last_name":"Xiao","full_name":"Xiao, Tianjun","first_name":"Tianjun"},{"full_name":"Zietlow, Dominik","last_name":"Zietlow","first_name":"Dominik"},{"first_name":"Max","full_name":"Horn, Max","last_name":"Horn"},{"first_name":"Zixu","last_name":"Zhao","full_name":"Zhao, Zixu"},{"last_name":"Carl-Johann Simon-Gabriel","full_name":"Carl-Johann Simon-Gabriel, Carl-Johann Simon-Gabriel","first_name":"Carl-Johann Simon-Gabriel"},{"full_name":"Shou, Mike Zheng","last_name":"Shou","first_name":"Mike Zheng"},{"orcid":"0000-0002-4850-0683","last_name":"Locatello","full_name":"Locatello, Francesco","first_name":"Francesco","id":"26cfd52f-2483-11ee-8040-88983bcc06d4"},{"first_name":"Bernt","last_name":"Schiele","full_name":"Schiele, Bernt"},{"last_name":"Brox","full_name":"Brox, Thomas","first_name":"Thomas"},{"last_name":"Zhang","full_name":"Zhang, Zheng","first_name":"Zheng"},{"last_name":"Fu","full_name":"Fu, Yanwei","first_name":"Yanwei"},{"first_name":"Tong","full_name":"He, Tong","last_name":"He"}],"publication":"arXiv"}