Using Continuous Integration to organize and monitor the annotation process of domain specific corpora
- 1 April 2014
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
Abstract
Applications in the World Wide Web aggregate vast amounts of information from different data sources. The aggregation process is often implemented with Extract, Transform and Load (ETL) processes. Usually ETL processes require information for aggregation available in structured formats, e. g. XML or JSON. In many cases the information is provided in natural language text which makes the application of ETL processes impractical. Due to the fact that information is provided in natural language, Information Extraction (IE) systems have been evolved. They make use of Natural Language Processing (NLP) tools to derive meaning from natural language text. State-of-the-art NLP tools apply Machine Learning methods. These NLP tools perform on newspapers with good quality, but they drop accuracy in other domains. However, to improve the quality for IE systems in specific domains often NLP tools are trained on domain specific text which is a time consuming process. This paper introduces an approach using a Continuous Integration pipeline for organizing and monitoring the annotation process on domain specific corpora.Keywords
This publication has 4 references indexed in Scilit:
- BioLemmatizer: a lemmatization tool for morphological processing of biomedical textJournal of Biomedical Semantics, 2012
- Building a biomedical tokenizer using the token lattice design pattern and the adapted Viterbi algorithmBMC Bioinformatics, 2011
- Comparing and combining chunkers of biomedical textJournal of Biomedical Informatics, 2011
- Ontology-based information extraction: An introduction and a survey of current approachesJournal of Information Science, 2010