Empirical Software Engineering Experimentation with Human Computation

Reka Marta Sabou, Dietmar Winkler, Stefan Biffl

Publication: Chapter in book/Conference proceedingChapter in edited volume


Empirical software engineering (ESE) focuses on gathering evidence through measurements and experiments involving humans and software systems (software products, processes, and resources). While empirical studies often include considerable human effort for study planning, execution, and data analysis, human computation (HC) methods, such as crowdsourcing, are increasingly used to address human input intensive tasks in software engineering and beyond. Therefore, in this chapter, we explore the use of HC techniques to support ESE experiments. We address researchers from both research communities and provide (1) introductory notions into both fields, (2) an analysis of ESE experiment requirements and HC capabilities that could match those, and (3) a concrete example of an ESE experiment that compares the effects of using HC in software inspection with respect to a traditional inspection process preformed using pen and paper. Our focus is on software inspection for detecting defects in software engineering models (namely, extended entity relationship models). This chapter will enable ESE researchers to apply HC in their work and HC researchers to explore ESE as a new application area to further improve their methods and tools.
Original languageEnglish
Title of host publicationContemporary Empirical Methods in Software Engineering
Editors Felderer, M., Travassos, G.
Place of PublicationCham, Switzerland
Pages173 - 215
Publication statusPublished - 2020

Austrian Classification of Fields of Science and Technology (ÖFOS)

  • 102
  • 102001 Artificial intelligence
  • 102015 Information systems
  • 102022 Software development

Cite this