Usefulness and usability of heuristic walkthroughs for evaluating domain-specific developer tools in industry: Evidence from four field simulations

Olaf Leßenich, Stefan Sobernig

Publikation: Wissenschaftliche FachzeitschriftOriginalbeitrag in FachzeitschriftBegutachtung


Context: The usage of domain-specific languages (DSLs) is an approach to reduce complexity in software development by expert developers for selected application domains. To support expert developers, a DSL is often combined with a tailored, domain-specific developer tool, offering similar functionality as general-purpose programming environments (IDEs like Eclipse for Java) while integrating with a domain-specific toolchain (e.g., code or documentation generators, simulators). General-purpose development environments have been successfully evaluated for the programmer experience (PX) and for anomalies using heuristic walkthroughs as a mixed review technique combining cognitive walkthroughs and a heuristic evaluation. Objective: In this paper, we report on the usefulness and acceptance of heuristic walkthroughs as an PX evaluation technique applied to domain-specific languages and IDEs in an industry context. Methods: Heuristic walkthroughs are used in four interventions (field simulations) to assess the programming experience and usability of domain-specific, Eclipse-based IDEs in a concrete industry setting. Data on the usefulness and acceptance (perceived satisfaction) of the walkthroughs themselves are collected and analysed. Results: Our studies show that, in practice, the instrument of walkthroughs is useful for revealing practically relevant PX anomalies, while maintaining acceptance by practitioners participating in the walkthroughs. Conclusion: The documented variant of heuristic walkthroughs is eligible to become adopted for future field studies in academic research and for evaluation projects in industry, in support of developing domain-specific developer tooling in an evidence-driven manner.
FachzeitschriftInformation and Software Technology
PublikationsstatusVeröffentlicht - 2023