Concordance analysis for CADS: key interpretability issues and their implications

Activity: Talk or presentationScience to science


Concordance analysis is widely recognised as one of the main techniques within a corpus linguist's toolkit. However, despite a growing body of work critically exploring previously unquestioned mainstays of corpus methods (Mautner, 2015; Taylor & Marchi, 2018), this has not yet been applied to concordance analysis specifically. In this methodological investigation, we explore some of the issues that researchers may encounter when interpreting concordances. To identify potential interpretability issues, we take a corpus of academic journal articles: every paper and book review published in Administrative Science Quarterly from its first publication in 1956 to the end of 2018 (19,470, 470 tokens). We then search for the lemma LEADER and use the following 3 research questions as a guiding principle when reviewing concordance lines: (i) What do leaders do?; (ii) What is done to leaders?; and (iii) What are leaders like? Provided at least one of those questions could be answered, we considered the concordance line interpretable. If those research questions could not be answered, we made a note of exactly why the concordance line was not useful.

We begin in Step One by eyeballing 800 concordance lines and subsequently identify 8 potential issues. Then, in Step Two, we assess the distribution of those issues in a reduced sample of 200. The 8 key issues are thus: noise in the corpus, non-standard syntax, unclear referring expressions, unclear quotation source attribution, technical terms and jargon, acronyms and initialisms, unspecific co-text, and finally lines that are unrelated to the research question. After reflecting on the practical challenges, we move onto the epistemological implications of removing those concordance lines uncritically, and offer recommendations for future CADS work.
Period9 Sept 2022
Event title15th International American Association for Corpus Linguistics Conference (AACL 2022)
Event typeConference
Conference number15
LocationFlagstaff, United States, ArizonaShow on map
Degree of RecognitionInternational