QRFA: A Data-Driven Model of Information-Seeking Dialogues

Svitlana Vakulenko; Kate Revoredo; Claudio Di Ciccio; Maarten de Rijke

doi:10.1007/978-3-030-15712-8_35

QRFA: A Data-Driven Model of Information-Seeking Dialogues

Svitlana Vakulenko, Kate Revoredo, Claudio Di Ciccio, Maarten de Rijke

Institute for Data, Process and Knowledge Management (AE Sabou)

Publication: Chapter in book/Conference proceeding › Contribution to conference proceedings

Abstract

Understanding the structure of interaction processes helps us to improve information-seeking dialogue systems. Analyzing an interaction process boils down to discovering patterns in sequences of alternating utterances exchanged between a user and an agent. Process mining techniques have been successfully applied to analyze structured event logs, discovering the underlying process models or evaluating whether the observed behavior is in conformance with the known process. In this paper, we apply process mining techniques to discover patterns in conversational transcripts and extract a new model of information-seeking dialogues, QRFA, for Query, Request, Feedback, Answer. Our results are grounded in an empirical evaluation across multiple conversational datasets from different domains, which was never attempted before. We show that the QRFA model better reflects conversation flows observed in real information-seeking conversations than models proposed previously. Moreover, QRFA allows us to identify malfunctioning in dialogue system transcripts as deviations from the expected conversation flow described by the model via conformance analysis.

Original language	English
Title of host publication	41st European Conference on IR Research, ECIR 2019, Cologne, Germany, April 14–18, 2019, Proceedings, Part I
Editors	Leif Azzopardi, Benno Stein, Norbert Fuhr, Philipp Mayr, Claudia Hauff, Djoerd Hiemstra
Place of Publication	Cologne, Germany
Publisher	Springer
Pages	541 - 557
DOIs	https://doi.org/10.1007/978-3-030-15712-8_35
Publication status	Published - 2019

Austrian Classification of Fields of Science and Technology (ÖFOS)

102
102001 Artificial intelligence
102022 Software development
102013 Human-computer interaction
502
502050 Business informatics

Access to Document

10.1007/978-3-030-15712-8_35Licence: Unspecified

Check availability

University Library Catalogue

Cyber-Physical Social Systems for City-wide Infrastructures
Cecconi, A., Di Ciccio, C., Fernandez Garcia, J. D., Mendling, J. & Polleres, A.
1/10/17 → 31/03/20
Project: Research funding
Open Data for Local Communities
Polleres, A., Taudes, A., Neumaier, S., Savenkov, V. & Vakulenko, S.
1/11/16 → 30/09/19
Project: Research funding
RISE_BPM
Di Ciccio, C. & Mendling, J.
1/05/15 → 30/04/19
Project: Research funding

Cite this

Vakulenko, S., Revoredo, K., Di Ciccio, C., & de Rijke, M. (2019). QRFA: A Data-Driven Model of Information-Seeking Dialogues. In Leif Azzopardi, Benno Stein, Norbert Fuhr, Philipp Mayr, Claudia Hauff, Djoerd Hiemstra (Ed.), 41st European Conference on IR Research, ECIR 2019, Cologne, Germany, April 14–18, 2019, Proceedings, Part I (pp. 541 - 557). Springer. https://doi.org/10.1007/978-3-030-15712-8_35

Vakulenko, Svitlana ; Revoredo, Kate ; Di Ciccio, Claudio et al. / QRFA: A Data-Driven Model of Information-Seeking Dialogues. 41st European Conference on IR Research, ECIR 2019, Cologne, Germany, April 14–18, 2019, Proceedings, Part I. editor / Leif Azzopardi, Benno Stein, Norbert Fuhr, Philipp Mayr, Claudia Hauff, Djoerd Hiemstra. Cologne, Germany : Springer, 2019. pp. 541 - 557

@inproceedings{b59f00ed2d794fdc85ffde653a497e63,

title = "QRFA: A Data-Driven Model of Information-Seeking Dialogues",

abstract = "Understanding the structure of interaction processes helps us to improve information-seeking dialogue systems. Analyzing an interaction process boils down to discovering patterns in sequences of alternating utterances exchanged between a user and an agent. Process mining techniques have been successfully applied to analyze structured event logs, discovering the underlying process models or evaluating whether the observed behavior is in conformance with the known process. In this paper, we apply process mining techniques to discover patterns in conversational transcripts and extract a new model of information-seeking dialogues, QRFA, for Query, Request, Feedback, Answer. Our results are grounded in an empirical evaluation across multiple conversational datasets from different domains, which was never attempted before. We show that the QRFA model better reflects conversation flows observed in real information-seeking conversations than models proposed previously. Moreover, QRFA allows us to identify malfunctioning in dialogue system transcripts as deviations from the expected conversation flow described by the model via conformance analysis.",

author = "Svitlana Vakulenko and Kate Revoredo and {Di Ciccio}, Claudio and {de Rijke}, Maarten",

year = "2019",

doi = "10.1007/978-3-030-15712-8_35",

language = "English",

pages = "541 -- 557",

editor = "{Leif Azzopardi, Benno Stein, Norbert Fuhr, Philipp Mayr, Claudia Hauff, Djoerd Hiemstra}",

booktitle = "41st European Conference on IR Research, ECIR 2019, Cologne, Germany, April 14–18, 2019, Proceedings, Part I",

publisher = "Springer",

address = "Germany",

}

Vakulenko, S, Revoredo, K, Di Ciccio, C & de Rijke, M 2019, QRFA: A Data-Driven Model of Information-Seeking Dialogues. in Leif Azzopardi, Benno Stein, Norbert Fuhr, Philipp Mayr, Claudia Hauff, Djoerd Hiemstra (ed.), 41st European Conference on IR Research, ECIR 2019, Cologne, Germany, April 14–18, 2019, Proceedings, Part I. Springer, Cologne, Germany, pp. 541 - 557. https://doi.org/10.1007/978-3-030-15712-8_35

QRFA: A Data-Driven Model of Information-Seeking Dialogues. / Vakulenko, Svitlana; Revoredo, Kate; Di Ciccio, Claudio et al.
41st European Conference on IR Research, ECIR 2019, Cologne, Germany, April 14–18, 2019, Proceedings, Part I. ed. / Leif Azzopardi, Benno Stein, Norbert Fuhr, Philipp Mayr, Claudia Hauff, Djoerd Hiemstra. Cologne, Germany: Springer, 2019. p. 541 - 557.

Publication: Chapter in book/Conference proceeding › Contribution to conference proceedings

TY - GEN

T1 - QRFA: A Data-Driven Model of Information-Seeking Dialogues

AU - Vakulenko, Svitlana

AU - Revoredo, Kate

AU - Di Ciccio, Claudio

AU - de Rijke, Maarten

PY - 2019

Y1 - 2019

N2 - Understanding the structure of interaction processes helps us to improve information-seeking dialogue systems. Analyzing an interaction process boils down to discovering patterns in sequences of alternating utterances exchanged between a user and an agent. Process mining techniques have been successfully applied to analyze structured event logs, discovering the underlying process models or evaluating whether the observed behavior is in conformance with the known process. In this paper, we apply process mining techniques to discover patterns in conversational transcripts and extract a new model of information-seeking dialogues, QRFA, for Query, Request, Feedback, Answer. Our results are grounded in an empirical evaluation across multiple conversational datasets from different domains, which was never attempted before. We show that the QRFA model better reflects conversation flows observed in real information-seeking conversations than models proposed previously. Moreover, QRFA allows us to identify malfunctioning in dialogue system transcripts as deviations from the expected conversation flow described by the model via conformance analysis.

AB - Understanding the structure of interaction processes helps us to improve information-seeking dialogue systems. Analyzing an interaction process boils down to discovering patterns in sequences of alternating utterances exchanged between a user and an agent. Process mining techniques have been successfully applied to analyze structured event logs, discovering the underlying process models or evaluating whether the observed behavior is in conformance with the known process. In this paper, we apply process mining techniques to discover patterns in conversational transcripts and extract a new model of information-seeking dialogues, QRFA, for Query, Request, Feedback, Answer. Our results are grounded in an empirical evaluation across multiple conversational datasets from different domains, which was never attempted before. We show that the QRFA model better reflects conversation flows observed in real information-seeking conversations than models proposed previously. Moreover, QRFA allows us to identify malfunctioning in dialogue system transcripts as deviations from the expected conversation flow described by the model via conformance analysis.

U2 - 10.1007/978-3-030-15712-8_35

DO - 10.1007/978-3-030-15712-8_35

M3 - Contribution to conference proceedings

SP - 541

EP - 557

BT - 41st European Conference on IR Research, ECIR 2019, Cologne, Germany, April 14–18, 2019, Proceedings, Part I

A2 - Leif Azzopardi, Benno Stein, Norbert Fuhr, Philipp Mayr, Claudia Hauff, Djoerd Hiemstra, null

PB - Springer

CY - Cologne, Germany

ER -

Vakulenko S, Revoredo K, Di Ciccio C, de Rijke M. QRFA: A Data-Driven Model of Information-Seeking Dialogues. In Leif Azzopardi, Benno Stein, Norbert Fuhr, Philipp Mayr, Claudia Hauff, Djoerd Hiemstra, editor, 41st European Conference on IR Research, ECIR 2019, Cologne, Germany, April 14–18, 2019, Proceedings, Part I. Cologne, Germany: Springer. 2019. p. 541 - 557 doi: 10.1007/978-3-030-15712-8_35

QRFA: A Data-Driven Model of Information-Seeking Dialogues

Abstract

Austrian Classification of Fields of Science and Technology (ÖFOS)

Access to Document

Check availability

Projects

Cyber-Physical Social Systems for City-wide Infrastructures

Open Data for Local Communities

RISE_BPM

Cite this