AB Testing for Process Versions with Contextual Multi-armed Bandit Algorithms

Suhrid Satyal, Ingo Weber, Hye-young Paik, Claudio Di Ciccio, Jan Mendling

Publication: Chapter in book/Conference proceedingContribution to conference proceedings


Business process improvement ideas can be validated through sequential experiment techniques like AB Testing. Such approaches have the inherent risk of exposing customers to an inferior process version, which is why the inferior version should be discarded as quickly as possible. In this paper, we propose a contextual multi-armed bandit algorithm that can observe the performance of process versions and dynamically adjust the routing policy so that the customers are directed to the version that can best serve them. Our algorithm learns the best routing policy in the presence of complications such as multiple process performance indicators, delays in indicator observation, incomplete or partial observations, and contextual factors. We also propose a pluggable architecture that supports such routing algorithms. We evaluate our approach with a case study. Furthermore, we demonstrate that our approach identifies the best routing policy given the process performance and that it scales horizontally.
Original languageEnglish
Title of host publicationAdvanced Information Systems Engineering - 30th International Conference, CAiSE 2018, June 11-15, 2018, Proceedings
Editors John Krogstie and Hajo A. Reijers
Place of PublicationTallinn, Estonia
Pages19 - 34
ISBN (Print)978-3-319-91563-0
Publication statusPublished - 2018

Austrian Classification of Fields of Science and Technology (ÖFOS)

  • 102022 Software development
  • 102
  • 102001 Artificial intelligence
  • 502
  • 502050 Business informatics

Cite this