Health Data Analytics with an Opportunistic Big Data Algorithm

In data-driven society, health data can lead to profound impacts on public safety policies, epidemic modeling, and advancement of health science and medicine. This paper presents an approach to automatically elucidating useful information from "Big" health data. In particular, we analyze manufactured cosmetic products containing chemicals that are known or suspected to cause cancer, birth defects, or developmental and reproductive harm. Our analysis is based on the Apriori algorithm, the heart of the popular Association Rule Mining to discover associations among sets of influencing factors. However, with rapid growth of huge amount of data, including ours, existing data analytics algorithms designed for in-memory data are not adequate. Most Big data analytics algorithms are implemented on MapReduce framework for execution in parallel and distributed environments. Unlike traditional implementation, our approach employs an opportunistic MapReduce-based Apriori algorithm to fully exploit parallelism. The paper describes the algorithm and presents our findings, from 113, 179 data instances, both in terms of the execution times and the discovered associations among product profiles. For a support threshold of 10% (5%,), 20 (53) association rules are obtained with an improved execution time over that of the traditional MapReduce-based algorithm by 14.6% (40.3%) on the average over three machines.