Adaptive normalization in streaming data

Vibhuti Gupta, Rattikorn Hewett

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

1 Scopus citations

Abstract

In today's digital era, data are everywhere from Internet of Things to health care or financial applications. This leads to potentially unbounded ever-growing Big data streams and it needs to be utilized effectively. Data normalization is an important preprocessing technique for data analytics. It helps prevent mismodeling and reduce the complexity inherent in the data especially for data integrated from multiple sources and contexts. Normalization of Big Data stream is challenging because of evolving inconsistencies, time and memory constraints, and nonavailability of whole data beforehand. This paper proposes a distributed approach to adaptive normalization for Big data stream. Using sliding windows of fixed size, it provides a simple mechanism to adapt the statistics for normalizing changing data in each window. Implemented on Apache Storm, a distributed realtime stream data framework, our approach exploits distributed data processing for efficient normalization. Unlike other existing adaptive approaches that normalize data for a specific use (e.g., classification), ours does not. Moreover, our adaptive mechanism allows flexible controls, via user-specified thresholds, for normalization tradeoffs between time and precision. The paper illustrates our proposed approach along with a few other techniques and experiments on both synthesized and real-world data. The normalized data obtained from our proposed approach, on 160,000 instances of data stream, improves over the baseline by 89% with 0.0041 root-mean-square error compared with the actual data.

Original languageEnglish
Title of host publicationICBDR 2019 - Proceedings of the 2019 3rd International Conference on Big Data Research
PublisherICST
Pages12-17
Number of pages6
ISBN (Electronic)9781450372015
DOIs
StatePublished - Nov 20 2019
Event3rd International Conference on Big Data Research, ICBDR 2019 - Cergy-Pontoise, France
Duration: Nov 20 2019Nov 21 2019

Publication series

NamePervasiveHealth: Pervasive Computing Technologies for Healthcare
ISSN (Print)2153-1633

Conference

Conference3rd International Conference on Big Data Research, ICBDR 2019
Country/TerritoryFrance
CityCergy-Pontoise
Period11/20/1911/21/19

Keywords

  • Big data stream
  • Normalization
  • Preprocessing

Fingerprint

Dive into the research topics of 'Adaptive normalization in streaming data'. Together they form a unique fingerprint.

Cite this