Execution time reduction in function oriented scientific workflows

Al-Haboobi Ali; Kecskeméti Gábor: Execution time reduction in function oriented scientific workflows. In: Acta cybernetica, (25) 2. pp. 131-150. (2021)

[thumbnail of cybernetica_025_numb_002_131-150.pdf] Cikk, tanulmány, mű
cybernetica_025_numb_002_131-150.pdf

Letöltés (292kB)

Absztrakt (kivonat)

Scientific workflows have been an increasingly important research area of distributed systems (such as cloud computing). Researchers have shown an increased interest in the automated processing scientific applications such as workflows. Recently, Function as a Service (FaaS) has emerged as a novel distributed systems platform for processing non-interactive applications. FaaS has limitations in resource use (e.g., CPU and RAM) as well as state management. In spite of these, initial studies have already demonstrated using FaaS for processing scientific workflows. DEWE v3 executes workflows in this fashion, but it often suffers from duplicate data transfers while using FaaS. This behaviour is due to the handling of intermediate data dependencies after and before each function invocation. These data dependencies could fill the temporary storage of the function environment. Our approach alters the job dispatch algorithm of DEWE v3 to reduce data dependency transfers. The proposed algorithm schedules jobs with precedence requirements to primarily run in the same function invocation. We evaluate our proposed algorithm and the original algorithm with small- and large-scale Montage workflows. Our results show that the improved system can reduce the total workflow execution time of scientific workflows over DEWE v3 by about 10\% when using AWS Lambda.

Mű típusa: Cikk, tanulmány, mű
Befoglaló folyóirat/kiadvány címe: Acta cybernetica
Dátum: 2021
Kötet: 25
Szám: 2
ISSN: 0324-721X
Oldalak: pp. 131-150
Nyelv: angol
Kiadó: University of Szeged, Institute of Informatics
Kiadás helye: Szeged
Konferencia neve: Conference of PhD Students in Computer Science (12.) (2020) (Szeged)
Befoglaló mű URL: http://acta.bibl.u-szeged.hu/75565/
DOI: 10.14232/actacyb.288489
Kulcsszavak: Algoritmus, Programozás
Megjegyzések: Bibliogr.: p. 148-150. ; ill. ; összefoglalás angol nyelven
Szakterület: 01. Természettudományok
01. Természettudományok > 01.02. Számítás- és információtudomány
Feltöltés dátuma: 2022. máj. 12. 13:27
Utolsó módosítás: 2022. máj. 12. 13:27
URI: http://acta.bibl.u-szeged.hu/id/eprint/75602
Bővebben:
Tétel nézet Tétel nézet