Incremental support vector learning for ordinal regression

Bin Gu, Victor S. Sheng, Keng Yeow Tay, Walter Romano, Shuo Li

Research output: Contribution to journalArticlepeer-review

619 Scopus citations

Abstract

Support vector ordinal regression (SVOR) is a popular method to tackle ordinal regression problems. However, until now there were no effective algorithms proposed to address incremental SVOR learning due to the complicated formulations of SVOR. Recently, an interesting accurate on-line algorithm was proposed for training \nu -support vector classification ( \nu -SVC), which can handle a quadratic formulation with a pair of equality constraints. In this paper, we first present a modified SVOR formulation based on a sum-of-margins strategy. The formulation has multiple constraints, and each constraint includes a mixture of an equality and an inequality. Then, we extend the accurate on-line \nu -SVC algorithm to the modified formulation, and propose an effective incremental SVOR algorithm. The algorithm can handle a quadratic formulation with multiple constraints, where each constraint is constituted of an equality and an inequality. More importantly, it tackles the conflicts between the equality and inequality constraints. We also provide the finite convergence analysis for the algorithm. Numerical experiments on the several benchmark and real-world data sets show that the incremental algorithm can converge to the optimal solution in a finite number of steps, and is faster than the existing batch and incremental SVOR algorithms. Meanwhile, the modified formulation has better accuracy than the existing incremental SVOR algorithm, and is as accurate as the sum-of-margins based formulation of Shashua and Levin.

Original languageEnglish
Article number6876186
Pages (from-to)1403-1416
Number of pages14
JournalIEEE Transactions on Neural Networks and Learning Systems
Volume26
Issue number7
DOIs
StatePublished - Jul 1 2015

Keywords

  • Incremental learning
  • online learning
  • ordinal regression (OR)
  • support vector machine (SVM)

Fingerprint Dive into the research topics of 'Incremental support vector learning for ordinal regression'. Together they form a unique fingerprint.

Cite this