---
_id: '11671'
abstract:
- lang: eng
  text: "Given only the URL of a Web page, can we identify its language? In this article
    we examine this question. URL-based language classification is useful when the
    content of the Web page is not available or downloading the content is a waste
    of bandwidth and time.\r\nWe built URL-based language classifiers for English,
    German, French, Spanish, and Italian by applying a variety of algorithms and features.
    As algorithms we used machine learning algorithms which are widely applied for
    text classification and state-of-art algorithms for language identification of
    text. As features we used words, various sized n-grams, and custom-made features
    (our novel feature set). We compared our approaches with two baseline methods,
    namely classification by country code top-level domains and classification by
    IP addresses of the hosting Web servers.\r\n\r\nWe trained and tested our classifiers
    in a 10-fold cross-validation setup on a dataset obtained from the Open Directory
    Project and from querying a commercial search engine. We obtained the lowest F1-measure
    for English (94) and the highest F1-measure for German (98) with the best performing
    classifiers.\r\n\r\nWe also evaluated the performance of our methods: (i) on a
    set of Web pages written in Adobe Flash and (ii) as part of a language-focused
    crawler. In the first case, the content of the Web page is hard to extract and
    in the second page downloading pages of the “wrong” language constitutes a waste
    of bandwidth. In both settings the best classifiers have a high accuracy with
    an F1-measure between 95 (for English) and 98 (for Italian) for the Adobe Flash
    pages and a precision between 90 (for Italian) and 97 (for French) for the language-focused
    crawler."
article_number: '3'
article_processing_charge: No
article_type: original
author:
- first_name: Eda
  full_name: Baykan, Eda
  last_name: Baykan
- first_name: Ingmar
  full_name: Weber, Ingmar
  last_name: Weber
- first_name: Monika H
  full_name: Henzinger, Monika H
  id: 540c9bbd-f2de-11ec-812d-d04a5be85630
  last_name: Henzinger
  orcid: 0000-0002-5008-6530
citation:
  ama: Baykan E, Weber I, Henzinger M. A comprehensive study of techniques for URL-based
    web page language classification. <i>ACM Transactions on the Web</i>. 2013;7(1).
    doi:<a href="https://doi.org/10.1145/2435215.2435218">10.1145/2435215.2435218</a>
  apa: Baykan, E., Weber, I., &#38; Henzinger, M. (2013). A comprehensive study of
    techniques for URL-based web page language classification. <i>ACM Transactions
    on the Web</i>. Association for Computing Machinery. <a href="https://doi.org/10.1145/2435215.2435218">https://doi.org/10.1145/2435215.2435218</a>
  chicago: Baykan, Eda, Ingmar Weber, and Monika Henzinger. “A Comprehensive Study
    of Techniques for URL-Based Web Page Language Classification.” <i>ACM Transactions
    on the Web</i>. Association for Computing Machinery, 2013. <a href="https://doi.org/10.1145/2435215.2435218">https://doi.org/10.1145/2435215.2435218</a>.
  ieee: E. Baykan, I. Weber, and M. Henzinger, “A comprehensive study of techniques
    for URL-based web page language classification,” <i>ACM Transactions on the Web</i>,
    vol. 7, no. 1. Association for Computing Machinery, 2013.
  ista: Baykan E, Weber I, Henzinger M. 2013. A comprehensive study of techniques
    for URL-based web page language classification. ACM Transactions on the Web. 7(1),
    3.
  mla: Baykan, Eda, et al. “A Comprehensive Study of Techniques for URL-Based Web
    Page Language Classification.” <i>ACM Transactions on the Web</i>, vol. 7, no.
    1, 3, Association for Computing Machinery, 2013, doi:<a href="https://doi.org/10.1145/2435215.2435218">10.1145/2435215.2435218</a>.
  short: E. Baykan, I. Weber, M. Henzinger, ACM Transactions on the Web 7 (2013).
date_created: 2022-07-27T12:50:18Z
date_published: 2013-03-01T00:00:00Z
date_updated: 2024-11-06T12:07:18Z
day: '01'
doi: 10.1145/2435215.2435218
extern: '1'
intvolume: '         7'
issue: '1'
keyword:
- Computer Networks and Communications
language:
- iso: eng
month: '03'
oa_version: None
publication: ACM Transactions on the Web
publication_identifier:
  eissn:
  - 1559-114X
  issn:
  - 1559-1131
publication_status: published
publisher: Association for Computing Machinery
quality_controlled: '1'
scopus_import: '1'
status: public
title: A comprehensive study of techniques for URL-based web page language classification
type: journal_article
user_id: 2DF688A6-F248-11E8-B48F-1D18A9856A87
volume: 7
year: '2013'
...