TY - GEN
T1 - Learning from the crowd with neural network
AU - Li, Jingjing
AU - Sheng, Victor S.
AU - Shu, Zhenyu
AU - Cheng, Yanxia
AU - Jin, Yuqin
AU - Yan, Yuan Feng
N1 - Funding Information:
We thank the anonymous reviewers for the valuable comments. The work was supported by the National Science Foundation (IIS-1115417)
Publisher Copyright:
© 2015 IEEE.
PY - 2016/3/2
Y1 - 2016/3/2
N2 - In general, the first step for supervised learning from crowdsourced data is integration. To obtain training data as traditional machine learning, the ground truth for each example in the crowdsourcing dataset must be integrated with consensus algorithms. However, some information and correlations among labels in the crowdsourcing dataset have discarded after integration. In order to study whether the information and correlations are useful for learning, we proposed three types of neural networks. Experimental results show that i) all the three types of neural networks have abilities to predict labels for future unseen examples, ii) when labelers have lower qualities, the information and correlations in crowdsourcing datasets, which are discarded by integration, does improve the performance of neural networks significantly, iii) when labelers have higher label qualities, the information and correlations have little impact on improving accuracy of neural networks.
AB - In general, the first step for supervised learning from crowdsourced data is integration. To obtain training data as traditional machine learning, the ground truth for each example in the crowdsourcing dataset must be integrated with consensus algorithms. However, some information and correlations among labels in the crowdsourcing dataset have discarded after integration. In order to study whether the information and correlations are useful for learning, we proposed three types of neural networks. Experimental results show that i) all the three types of neural networks have abilities to predict labels for future unseen examples, ii) when labelers have lower qualities, the information and correlations in crowdsourcing datasets, which are discarded by integration, does improve the performance of neural networks significantly, iii) when labelers have higher label qualities, the information and correlations have little impact on improving accuracy of neural networks.
KW - Crowdsourcing
KW - Machine learning
KW - Neural network
UR - http://www.scopus.com/inward/record.url?scp=84969674109&partnerID=8YFLogxK
U2 - 10.1109/ICMLA.2015.14
DO - 10.1109/ICMLA.2015.14
M3 - Conference contribution
AN - SCOPUS:84969674109
T3 - Proceedings - 2015 IEEE 14th International Conference on Machine Learning and Applications, ICMLA 2015
SP - 693
EP - 698
BT - Proceedings - 2015 IEEE 14th International Conference on Machine Learning and Applications, ICMLA 2015
PB - Institute of Electrical and Electronics Engineers Inc.
Y2 - 9 December 2015 through 11 December 2015
ER -