The Bootstrap Discovery Behaviour (BDB): a new outlook on usability evaluation
- 3 November 2010
- journal article
- Published by Springer Science and Business Media LLC in Cognitive Processing
- Vol. 12 (1), 23-31
- https://doi.org/10.1007/s10339-010-0376-6
Abstract
The value of λ is one of the main issues debated in international usability studies. The debate is centred on the deficiencies of the mathematical return on investment model (ROI model) of Nielsen and Landauer (1993). The ROI model is discussed in order to identify the base of another model that, respecting Nielsen and Landauer’s one, tries to consider a large number of variables for the estimation of the number of evaluators needed for an interface. Using the bootstrap model (Efron 1979), we can take into account: (a) the interface properties, as the properties at zero condition of evaluation and (b) the probability that the population discovery behaviour is represented by all the possible discovery behaviours of a sample. Our alternative model, named Bootstrap Discovery Behaviour (BDB), provides an alternative estimation of the number of experts and users needed for a usability evaluation. Two experimental groups of users and experts are involved in the evaluation of a website (http://www.serviziocivile.it). Applying the BDB model to the problems identified by the two groups, we found that 13 experts and 20 users are needed to identify 80% of usability problems, instead of 6 experts and 7 users required according to the estimation of the discovery likelihood provided by the ROI model. The consequence of the difference between the results of those models is that in following the BDB the costs of usability evaluation increase, although this is justified considering that the results obtained have the best probability of representing the entire population of experts and users.Keywords
This publication has 23 references indexed in Scilit:
- Usability evaluation with screen reader users: a video presentation of the PCTA’s experimental setting and rulesCognitive Processing, 2010
- Web usability evaluation with screen reader users: implementation of the partial concurrent thinking aloud techniqueCognitive Processing, 2009
- The Evaluator Effect: A Chilling Fact About Usability Evaluation MethodsInternational Journal of Human–Computer Interaction, 2003
- Relaxing the homogeneity assumption in usability testingBehaviour & Information Technology, 2001
- The Effects of Concurrent Verbalization on Choice ProcessingJournal of Marketing Research, 1989
- Consumers' Use of Memory and External Information in Choice: Macro and Micro PerspectivesJournal of Consumer Research, 1986
- Information-Presentation Format and Learning Goals as Determinants of Consumers' Memory Retrieval and Choice ProcessesJournal of Consumer Research, 1982
- Experiences with the Bettman-Park Verbal-Protocol Coding SchemeJournal of Consumer Research, 1982
- Effects of Prior Knowledge and Experience and Phase of the Choice Process on Consumer Decision Processes: A Protocol AnalysisJournal of Consumer Research, 1980
- Bootstrap Methods: Another Look at the JackknifeThe Annals of Statistics, 1979