Reflections on conducting an international survey of software engineering

Abstract

Component-based software engineering (CBSE) with commercial-off-the-shelf (COTS) or open source software (OSS) components are more and more frequently being used in industrial software development. We therefore need to issue experience-based guidelines for the evaluation, selection and integration of such components. We have performed a survey on industrial COTS/OSS development in three countries - Norway, Italy and Germany. Concrete survey results, e.g. on risk management policies and process tailoring, are not being described here, but in other papers. This is a method paper, reporting on the challenges, approaches and experiences gained by conducting the main survey. The main contributions are as follows: At best we can achieve a stratified-random sample of ICT companies, followed by a convenience sample of relevant projects. This is probably the ftirst software engineering survey using census type data, and has revealed that the entire sampling and contact process can be unexpectedly expensive. It is also hard to avoid national variations in the total process, possibly leading to uncontrollable method biases