{"year":"2006","conference":{"name":"SIGIR: International Conference on Research and Development in Information Retrieval","start_date":"2006-08-06","location":"Seattle, WA, United States","end_date":"2006-08-11"},"citation":{"short":"M.H. Henzinger, in:, 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Association for Computing Machinery, 2006, pp. 284–291.","apa":"Henzinger, M. H. (2006). Finding near-duplicate web pages: A large-scale evaluation of algorithms. In <i>29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval</i> (pp. 284–291). Seattle, WA, United States: Association for Computing Machinery. <a href=\"https://doi.org/10.1145/1148170.1148222\">https://doi.org/10.1145/1148170.1148222</a>","chicago":"Henzinger, Monika H. “Finding Near-Duplicate Web Pages: A Large-Scale Evaluation of Algorithms.” In <i>29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval</i>, 284–91. Association for Computing Machinery, 2006. <a href=\"https://doi.org/10.1145/1148170.1148222\">https://doi.org/10.1145/1148170.1148222</a>.","ista":"Henzinger MH. 2006. Finding near-duplicate web pages: A large-scale evaluation of algorithms. 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. SIGIR: International Conference on Research and Development in Information Retrieval, 284–291.","mla":"Henzinger, Monika H. “Finding Near-Duplicate Web Pages: A Large-Scale Evaluation of Algorithms.” <i>29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval</i>, Association for Computing Machinery, 2006, pp. 284–91, doi:<a href=\"https://doi.org/10.1145/1148170.1148222\">10.1145/1148170.1148222</a>.","ieee":"M. H. Henzinger, “Finding near-duplicate web pages: A large-scale evaluation of algorithms,” in <i>29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval</i>, Seattle, WA, United States, 2006, pp. 284–291.","ama":"Henzinger MH. Finding near-duplicate web pages: A large-scale evaluation of algorithms. In: <i>29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval</i>. Association for Computing Machinery; 2006:284-291. doi:<a href=\"https://doi.org/10.1145/1148170.1148222\">10.1145/1148170.1148222</a>"},"type":"conference","_id":"11929","date_created":"2022-08-19T07:20:31Z","extern":"1","status":"public","article_processing_charge":"No","language":[{"iso":"eng"}],"author":[{"first_name":"Monika H","id":"540c9bbd-f2de-11ec-812d-d04a5be85630","full_name":"Henzinger, Monika H","orcid":"0000-0002-5008-6530","last_name":"Henzinger"}],"publication_status":"published","month":"08","title":"Finding near-duplicate web pages: A large-scale evaluation of algorithms","date_published":"2006-08-01T00:00:00Z","abstract":[{"text":"Broder et al.'s [3] shingling algorithm and Charikar's [4] random projection based approach are considered \"state-of-the-art\" algorithms for finding near-duplicate web pages. Both algorithms were either developed at or used by popular web search engines. We compare the two algorithms on a very large scale, namely on a set of 1.6B distinct web pages. The results show that neither of the algorithms works well for finding near-duplicate pairs on the same site, while both achieve high precision for near-duplicate pairs on different sites. Since Charikar's algorithm finds more near-duplicate pairs on different sites, it achieves a better precision overall, namely 0.50 versus 0.38 for Broder et al.'s algorithm. We present a combined algorithm which achieves precision 0.79 with 79% of the recall of the other algorithms.","lang":"eng"}],"page":"284-291","doi":"10.1145/1148170.1148222","publication":"29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval","scopus_import":"1","date_updated":"2023-02-17T13:49:03Z","quality_controlled":"1","publisher":"Association for Computing Machinery","oa_version":"None","user_id":"2DF688A6-F248-11E8-B48F-1D18A9856A87","day":"01"}