Managing Uncertainty in Crowdsourcing with Interval-Valued Labels

Chenyi Hu, Victor S. Sheng, Ningning Wu, Xintao Wu

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

Crowdsourcing has been an emerging machine learning paradigm. It collects labels from human crowds as inputs typically through the Internet. Due to limitations on knowledge, social-economic status, and other factors, participants may often have ambiguity in labeling some instances in practice. In this work, we propose interval-valued labels (IVLs), instead of commonly used binary-valued ones, to manage such kind of uncertainty in crowdsourcing. IVLs possess interval specific statistic and probabilistic properties. With them, this work presents an algorithm that is able to make an inference with a favorable matching probability as a main result. The algorithm also implies an index, which measures the overall uncertainty of collected IVLs quantitatively. Reported computational experiments further evidence that we may better manage uncertainty in crowdsourcing with IVLs than without.

Original languageEnglish
Title of host publicationExplainable AI and Other Applications of Fuzzy Techniques - Proceedings of the 2021 Annual Conference of the North American Fuzzy Information Processing Society, NAFIPS 2021
EditorsJulia Rayz, Victor Raskin, Scott Dick, Vladik Kreinovich
PublisherSpringer Science and Business Media Deutschland GmbH
Pages166-178
Number of pages13
ISBN (Print)9783030820985
DOIs
StatePublished - 2022
EventAnnual Conference of the North American Fuzzy Information Processing Society, NAFIPS 2021 - West Lafayette, United States
Duration: Jun 7 2021Jun 9 2021

Publication series

NameLecture Notes in Networks and Systems
Volume258
ISSN (Print)2367-3370
ISSN (Electronic)2367-3389

Conference

ConferenceAnnual Conference of the North American Fuzzy Information Processing Society, NAFIPS 2021
Country/TerritoryUnited States
CityWest Lafayette
Period06/7/2106/9/21

Fingerprint

Dive into the research topics of 'Managing Uncertainty in Crowdsourcing with Interval-Valued Labels'. Together they form a unique fingerprint.

Cite this