Execution time reduction in function oriented scientific workflows

Al-Haboobi Ali and Kecskeméti Gábor: Execution time reduction in function oriented scientific workflows. In: Acta cybernetica, (25) 2. pp. 131-150. (2021)

Cikk, tanulmány, mű cybernetica_025_numb_002_131-150.pdf Download (292kB)

Abstract

Scientific workflows have been an increasingly important research area of distributed systems (such as cloud computing). Researchers have shown an increased interest in the automated processing scientific applications such as workflows. Recently, Function as a Service (FaaS) has emerged as a novel distributed systems platform for processing non-interactive applications. FaaS has limitations in resource use (e.g., CPU and RAM) as well as state management. In spite of these, initial studies have already demonstrated using FaaS for processing scientific workflows. DEWE v3 executes workflows in this fashion, but it often suffers from duplicate data transfers while using FaaS. This behaviour is due to the handling of intermediate data dependencies after and before each function invocation. These data dependencies could fill the temporary storage of the function environment. Our approach alters the job dispatch algorithm of DEWE v3 to reduce data dependency transfers. The proposed algorithm schedules jobs with precedence requirements to primarily run in the same function invocation. We evaluate our proposed algorithm and the original algorithm with small- and large-scale Montage workflows. Our results show that the improved system can reduce the total workflow execution time of scientific workflows over DEWE v3 by about 10\% when using AWS Lambda.

Item Type:	Article
Journal or Publication Title:	Acta cybernetica
Date:	2021
Volume:	25
Number:	2
ISSN:	0324-721X
Page Range:	pp. 131-150
Language:	English
Publisher:	University of Szeged, Institute of Informatics
Place of Publication:	Szeged
Event Title:	Conference of PhD Students in Computer Science (12.) (2020) (Szeged)
Related URLs:	http://acta.bibl.u-szeged.hu/75565/
DOI:	10.14232/actacyb.288489
Uncontrolled Keywords:	Algoritmus, Programozás
Additional Information:	Bibliogr.: p. 148-150. ; ill. ; összefoglalás angol nyelven
Subjects:	01. Natural sciences 01. Natural sciences > 01.02. Computer and information sciences
Date Deposited:	2022. May. 12. 13:27
Last Modified:	2022. May. 12. 13:27
URI:	http://acta.bibl.u-szeged.hu/id/eprint/75602

Actions (login required)

View Item