A Human Computation Approach for Ontology Restrictions Verification

Publication: Contribution to conferenceConference paper


Ontologies are semantic resources essential for systems re- quiring real-world knowledge. As such, their correctness and quality are of high importance, and in some cases can only be achieved with human intervention. In this paper, we pro- pose a Human Computation (HC) solution for the verification of ontology restrictions by means of universal and existen- tial quantifiers and report on a controlled experiment to study two core task design aspects: (i) the formalism to represent ontology axioms in the HC task and (2) participant qualifi- cation testing. We find that visual axiom representation and prior knowledge of ontology restriction models lead to best results while prior modeling knowledge reduces the evalua- tion times. Our findings are of interest to researchers aiming to use HC for knowledge engineering tasks related to ontolo- gies or other conceptual structures (e.g., EER diagrams).
Original languageEnglish
Publication statusPublished - 2021
Externally publishedYes
Event9th AAAI Conference on Human Computation and Crowdsourcing -
Duration: 14 Nov 202118 Nov 2021


Conference9th AAAI Conference on Human Computation and Crowdsourcing
Abbreviated titleHCOMP 2021
Internet address


  • human computation
  • crowdsourcing
  • ontology evaluation

Cite this