{"publication":"ACM Transactions on the Web","month":"03","_id":"11671","type":"journal_article","date_updated":"2022-09-12T08:51:57Z","article_type":"original","quality_controlled":"1","doi":"10.1145/2435215.2435218","volume":7,"article_number":"3","article_processing_charge":"No","abstract":[{"lang":"eng","text":"Given only the URL of a Web page, can we identify its language? In this article we examine this question. URL-based language classification is useful when the content of the Web page is not available or downloading the content is a waste of bandwidth and time.\r\nWe built URL-based language classifiers for English, German, French, Spanish, and Italian by applying a variety of algorithms and features. As algorithms we used machine learning algorithms which are widely applied for text classification and state-of-art algorithms for language identification of text. As features we used words, various sized n-grams, and custom-made features (our novel feature set). We compared our approaches with two baseline methods, namely classification by country code top-level domains and classification by IP addresses of the hosting Web servers.\r\n\r\nWe trained and tested our classifiers in a 10-fold cross-validation setup on a dataset obtained from the Open Directory Project and from querying a commercial search engine. We obtained the lowest F1-measure for English (94) and the highest F1-measure for German (98) with the best performing classifiers.\r\n\r\nWe also evaluated the performance of our methods: (i) on a set of Web pages written in Adobe Flash and (ii) as part of a language-focused crawler. In the first case, the content of the Web page is hard to extract and in the second page downloading pages of the “wrong” language constitutes a waste of bandwidth. In both settings the best classifiers have a high accuracy with an F1-measure between 95 (for English) and 98 (for Italian) for the Adobe Flash pages and a precision between 90 (for Italian) and 97 (for French) for the language-focused crawler."}],"extern":"1","day":"01","publication_status":"published","issue":"1","oa_version":"None","publication_identifier":{"eissn":["1559-114X"],"issn":["1559-1131"]},"user_id":"2DF688A6-F248-11E8-B48F-1D18A9856A87","publisher":"Association for Computing Machinery","citation":{"ieee":"E. Baykan, I. Weber, and M. H. Henzinger, “A comprehensive study of techniques for URL-based web page language classification,” ACM Transactions on the Web, vol. 7, no. 1. Association for Computing Machinery, 2013.","ama":"Baykan E, Weber I, Henzinger MH. A comprehensive study of techniques for URL-based web page language classification. ACM Transactions on the Web. 2013;7(1). doi:10.1145/2435215.2435218","mla":"Baykan, Eda, et al. “A Comprehensive Study of Techniques for URL-Based Web Page Language Classification.” ACM Transactions on the Web, vol. 7, no. 1, 3, Association for Computing Machinery, 2013, doi:10.1145/2435215.2435218.","chicago":"Baykan, Eda, Ingmar Weber, and Monika H Henzinger. “A Comprehensive Study of Techniques for URL-Based Web Page Language Classification.” ACM Transactions on the Web. Association for Computing Machinery, 2013. https://doi.org/10.1145/2435215.2435218.","ista":"Baykan E, Weber I, Henzinger MH. 2013. A comprehensive study of techniques for URL-based web page language classification. ACM Transactions on the Web. 7(1), 3.","short":"E. Baykan, I. Weber, M.H. Henzinger, ACM Transactions on the Web 7 (2013).","apa":"Baykan, E., Weber, I., & Henzinger, M. H. (2013). A comprehensive study of techniques for URL-based web page language classification. ACM Transactions on the Web. Association for Computing Machinery. https://doi.org/10.1145/2435215.2435218"},"keyword":["Computer Networks and Communications"],"scopus_import":"1","date_published":"2013-03-01T00:00:00Z","intvolume":" 7","title":"A comprehensive study of techniques for URL-based web page language classification","status":"public","language":[{"iso":"eng"}],"date_created":"2022-07-27T12:50:18Z","author":[{"first_name":"Eda","full_name":"Baykan, Eda","last_name":"Baykan"},{"full_name":"Weber, Ingmar","first_name":"Ingmar","last_name":"Weber"},{"last_name":"Henzinger","orcid":"0000-0002-5008-6530","id":"540c9bbd-f2de-11ec-812d-d04a5be85630","first_name":"Monika H","full_name":"Henzinger, Monika H"}],"year":"2013"}