Dictionary-Based Bilingual Web Page Classification

Abstract

Web page classification poses new research challenges because of the noisy nature of the pages. For the bilingual Chinese-English web pages, it also needs to be considered that how to extract the terms of different languages exactly. A new dictionary-based multilingual text categorization approach is proposed in this paper to try to classify the Chinese-English web pages in specific domain into a hierarchical topic structure more accurately. The approach can properly recognize and integrate the web page encodings by using an automatic encoding detection and integration method. This makes the feature extraction more precise for the multilingual pages. The approach can also intensify the domain concepts in the web pages based on a domain dictionary. From the results of the experiments, it can be found that the proposed approach get the better performance than the traditional classification method when classifying the bilingual web pages.