TY - GEN
T1 - Application of Edge-to-Cloud Methods Toward Deep Learning
AU - Choudhary, Khushi
AU - Nersisyan, Nona
AU - Lin, Edward
AU - Chandrasekaran, Shobana
AU - Mayani, Rajiv
AU - Pottier, Loic
AU - Murillo, Angela P.
AU - Virdone, Nicole K.
AU - Kee, Kerk
AU - Deelman, Ewa
N1 - Publisher Copyright:
© 2022 IEEE.
PY - 2022
Y1 - 2022
N2 - Scientific workflows are important in modern computational science and are a convenient way to represent complex computations, which are often geographically distributed among several computers. In many scientific domains, scientists use sensors (e.g., edge devices) to gather data such as CO2 level or temperature, that are usually sent to a central processing facility (e.g., a cloud). However, these edge devices are often not powerful enough to perform basic computations or machine learning inference computations and thus applications need the power of cloud platforms to generate scientific results. This work explores the execution and deployment of a complex workflow on an edge-to-cloud architecture in a use case of the detection and classification of plankton. In the original application, images were captured by cameras attached to buoys floating in Lake Greifensee (Switzerland). We developed a workflow based on that application. The workflow aims to pre-process images locally on the edge devices (i.e., buoys) then transfer data from each edge device to a cloud platform. Here, we developed a Pegasus workflow that runs using HTCondor and leveraged the Chameleon cloud platform and its recent CHI@Edge feature to mimic such deployment and study its feasibility in terms of performance and deployment.
AB - Scientific workflows are important in modern computational science and are a convenient way to represent complex computations, which are often geographically distributed among several computers. In many scientific domains, scientists use sensors (e.g., edge devices) to gather data such as CO2 level or temperature, that are usually sent to a central processing facility (e.g., a cloud). However, these edge devices are often not powerful enough to perform basic computations or machine learning inference computations and thus applications need the power of cloud platforms to generate scientific results. This work explores the execution and deployment of a complex workflow on an edge-to-cloud architecture in a use case of the detection and classification of plankton. In the original application, images were captured by cameras attached to buoys floating in Lake Greifensee (Switzerland). We developed a workflow based on that application. The workflow aims to pre-process images locally on the edge devices (i.e., buoys) then transfer data from each edge device to a cloud platform. Here, we developed a Pegasus workflow that runs using HTCondor and leveraged the Chameleon cloud platform and its recent CHI@Edge feature to mimic such deployment and study its feasibility in terms of performance and deployment.
KW - Edge Computing
KW - Machine Learning
KW - Pegasus
KW - Scientific Workflows
KW - Workflow Management Systems
KW - Zooplankton
UR - http://www.scopus.com/inward/record.url?scp=85145440146&partnerID=8YFLogxK
U2 - 10.1109/eScience55777.2022.00065
DO - 10.1109/eScience55777.2022.00065
M3 - Conference contribution
AN - SCOPUS:85145440146
T3 - Proceedings - 2022 IEEE 18th International Conference on e-Science, eScience 2022
SP - 415
EP - 416
BT - Proceedings - 2022 IEEE 18th International Conference on e-Science, eScience 2022
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 18th IEEE International Conference on e-Science, eScience 2022
Y2 - 10 October 2022 through 14 October 2022
ER -