TY - GEN
T1 - Managing Uncertainty in Crowdsourcing with Interval-Valued Labels
AU - Hu, Chenyi
AU - Sheng, Victor S.
AU - Wu, Ningning
AU - Wu, Xintao
N1 - Funding Information:
This work is partially supported by the US National Science Foundation through the grant award NSF/OIA-1946391.
Publisher Copyright:
© 2022, The Author(s), under exclusive license to Springer Nature Switzerland AG.
PY - 2022
Y1 - 2022
N2 - Crowdsourcing has been an emerging machine learning paradigm. It collects labels from human crowds as inputs typically through the Internet. Due to limitations on knowledge, social-economic status, and other factors, participants may often have ambiguity in labeling some instances in practice. In this work, we propose interval-valued labels (IVLs), instead of commonly used binary-valued ones, to manage such kind of uncertainty in crowdsourcing. IVLs possess interval specific statistic and probabilistic properties. With them, this work presents an algorithm that is able to make an inference with a favorable matching probability as a main result. The algorithm also implies an index, which measures the overall uncertainty of collected IVLs quantitatively. Reported computational experiments further evidence that we may better manage uncertainty in crowdsourcing with IVLs than without.
AB - Crowdsourcing has been an emerging machine learning paradigm. It collects labels from human crowds as inputs typically through the Internet. Due to limitations on knowledge, social-economic status, and other factors, participants may often have ambiguity in labeling some instances in practice. In this work, we propose interval-valued labels (IVLs), instead of commonly used binary-valued ones, to manage such kind of uncertainty in crowdsourcing. IVLs possess interval specific statistic and probabilistic properties. With them, this work presents an algorithm that is able to make an inference with a favorable matching probability as a main result. The algorithm also implies an index, which measures the overall uncertainty of collected IVLs quantitatively. Reported computational experiments further evidence that we may better manage uncertainty in crowdsourcing with IVLs than without.
UR - http://www.scopus.com/inward/record.url?scp=85113396971&partnerID=8YFLogxK
U2 - 10.1007/978-3-030-82099-2_15
DO - 10.1007/978-3-030-82099-2_15
M3 - Conference contribution
AN - SCOPUS:85113396971
SN - 9783030820985
T3 - Lecture Notes in Networks and Systems
SP - 166
EP - 178
BT - Explainable AI and Other Applications of Fuzzy Techniques - Proceedings of the 2021 Annual Conference of the North American Fuzzy Information Processing Society, NAFIPS 2021
A2 - Rayz, Julia
A2 - Raskin, Victor
A2 - Dick, Scott
A2 - Kreinovich, Vladik
PB - Springer Science and Business Media Deutschland GmbH
Y2 - 7 June 2021 through 9 June 2021
ER -