Automated Software Quality Monitoring in Research Collaboration Projects
- 27 June 2020
- conference paper
- conference paper
- Published by Association for Computing Machinery (ACM) in Proceedings of the IEEE/ACM 42nd International Conference on Software Engineering Workshops
Abstract
In collaborative research projects, both researchers and practitioners work together solving business-critical challenges. These projects often deal with ETL processes, in which humans extract information from non-machine-readable documents by hand. AI-based machine learning models can help to solve this problem. Since machine learning approaches are not deterministic, their quality of output may decrease over time. This fact leads to an overall quality loss of the application which embeds machine learning models. Hence, the software qualities in development and production may differ. Machine learning models are black boxes. That makes practitioners skeptical and increases the inhibition threshold for early productive use of research prototypes. Continuous monitoring of software quality in production offers an early response capability on quality loss and encourages the use of machine learning approaches. Furthermore, experts have to ensure that they integrate possible new inputs into the model training as quickly as possible. In this paper, we introduce an architecture pattern with a reference implementation that extends the concept of Metrics Driven Research Collaboration with an automated software quality monitoring in productive use and a possibility to auto-generate new test data coming from processed documents in production. Through automated monitoring of the software quality and auto-generated test data, this approach ensures that the software quality meets and keeps requested thresholds in productive use, even during further continuous deployment and changing input data.Keywords
This publication has 5 references indexed in Scilit:
- Data-Driven Recognition and Extraction of PDF Document ElementsTechnologies, 2019
- A New Process Model for the Comprehensive Management of Machine Learning ModelsPublished by INSTICC ,2019
- Schema Matching with Frequent Changes on Semi-Structured Input Files: A Machine Learning Approach on Biological Product DataPublished by INSTICC ,2019
- Continuously evaluated research projects in collaborative decoupled environmentsPublished by Association for Computing Machinery (ACM) ,2018
- Metrics Driven Research Collaboration: Focusing on Common Project Goals ContinuouslyPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2017