A Human Computation Approach for Ontology Restrictions Verification

Ontologies are semantic resources essential for systems re- quiring real-world knowledge. As such, their correctness and quality are of high importance, and in some cases can only be achieved with human intervention. In this paper, we pro- pose a Human Computation (HC) solution for the verification of ontology restrictions by means of universal and existen- tial quantifiers and report on a controlled experiment to study two core task design aspects: (i) the formalism to represent ontology axioms in the HC task and (2) participant qualifi- cation testing. We find that visual axiom representation and prior knowledge of ontology restriction models lead to best results while prior modeling knowledge reduces the evalua- tion times. Our findings are of interest to researchers aiming to use HC for knowledge engineering tasks related to ontolo- gies or other conceptual structures (e.g., EER diagrams).