{"author":[{"full_name":"Baykan, Eda","first_name":"Eda","last_name":"Baykan"},{"full_name":"Weber, Ingmar","first_name":"Ingmar","last_name":"Weber"},{"first_name":"Monika H","last_name":"Henzinger","orcid":"0000-0002-5008-6530","full_name":"Henzinger, Monika H","id":"540c9bbd-f2de-11ec-812d-d04a5be85630"}],"language":[{"iso":"eng"}],"day":"01","year":"2013","abstract":[{"text":"Given only the URL of a Web page, can we identify its language? In this article we examine this question. URL-based language classification is useful when the content of the Web page is not available or downloading the content is a waste of bandwidth and time.\r\nWe built URL-based language classifiers for English, German, French, Spanish, and Italian by applying a variety of algorithms and features. As algorithms we used machine learning algorithms which are widely applied for text classification and state-of-art algorithms for language identification of text. As features we used words, various sized n-grams, and custom-made features (our novel feature set). We compared our approaches with two baseline methods, namely classification by country code top-level domains and classification by IP addresses of the hosting Web servers.\r\n\r\nWe trained and tested our classifiers in a 10-fold cross-validation setup on a dataset obtained from the Open Directory Project and from querying a commercial search engine. We obtained the lowest F1-measure for English (94) and the highest F1-measure for German (98) with the best performing classifiers.\r\n\r\nWe also evaluated the performance of our methods: (i) on a set of Web pages written in Adobe Flash and (ii) as part of a language-focused crawler. In the first case, the content of the Web page is hard to extract and in the second page downloading pages of the “wrong” language constitutes a waste of bandwidth. In both settings the best classifiers have a high accuracy with an F1-measure between 95 (for English) and 98 (for Italian) for the Adobe Flash pages and a precision between 90 (for Italian) and 97 (for French) for the language-focused crawler.","lang":"eng"}],"scopus_import":"1","date_published":"2013-03-01T00:00:00Z","volume":7,"quality_controlled":"1","article_number":"3","publication_identifier":{"eissn":["1559-114X"],"issn":["1559-1131"]},"intvolume":"         7","citation":{"mla":"Baykan, Eda, et al. “A Comprehensive Study of Techniques for URL-Based Web Page Language Classification.” <i>ACM Transactions on the Web</i>, vol. 7, no. 1, 3, Association for Computing Machinery, 2013, doi:<a href=\"https://doi.org/10.1145/2435215.2435218\">10.1145/2435215.2435218</a>.","ista":"Baykan E, Weber I, Henzinger MH. 2013. A comprehensive study of techniques for URL-based web page language classification. ACM Transactions on the Web. 7(1), 3.","ieee":"E. Baykan, I. Weber, and M. H. Henzinger, “A comprehensive study of techniques for URL-based web page language classification,” <i>ACM Transactions on the Web</i>, vol. 7, no. 1. Association for Computing Machinery, 2013.","short":"E. Baykan, I. Weber, M.H. Henzinger, ACM Transactions on the Web 7 (2013).","apa":"Baykan, E., Weber, I., &#38; Henzinger, M. H. (2013). A comprehensive study of techniques for URL-based web page language classification. <i>ACM Transactions on the Web</i>. Association for Computing Machinery. <a href=\"https://doi.org/10.1145/2435215.2435218\">https://doi.org/10.1145/2435215.2435218</a>","ama":"Baykan E, Weber I, Henzinger MH. A comprehensive study of techniques for URL-based web page language classification. <i>ACM Transactions on the Web</i>. 2013;7(1). doi:<a href=\"https://doi.org/10.1145/2435215.2435218\">10.1145/2435215.2435218</a>","chicago":"Baykan, Eda, Ingmar Weber, and Monika H Henzinger. “A Comprehensive Study of Techniques for URL-Based Web Page Language Classification.” <i>ACM Transactions on the Web</i>. Association for Computing Machinery, 2013. <a href=\"https://doi.org/10.1145/2435215.2435218\">https://doi.org/10.1145/2435215.2435218</a>."},"title":"A comprehensive study of techniques for URL-based web page language classification","article_processing_charge":"No","issue":"1","publication":"ACM Transactions on the Web","publisher":"Association for Computing Machinery","publication_status":"published","date_created":"2022-07-27T12:50:18Z","month":"03","_id":"11671","extern":"1","article_type":"original","status":"public","keyword":["Computer Networks and Communications"],"type":"journal_article","date_updated":"2022-09-12T08:51:57Z","user_id":"2DF688A6-F248-11E8-B48F-1D18A9856A87","oa_version":"None","doi":"10.1145/2435215.2435218"}