Next Article in Journal
Design and Realization of a Frequency Reconfigurable Antenna with Wide, Dual, and Single-Band Operations for Compact Sized Wireless Applications
Previous Article in Journal
Beamforming Evaluation of 5G User Equipment through Novel Key Performance Indicators
Previous Article in Special Issue
Effective Cloud Resource Utilisation in Cloud ERP Decision-Making Process for Industry 4.0 in the United States
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Multi-Dependency and Time Based Resource Scheduling Algorithm for Scientific Applications in Cloud Computing

1
Department of Computer Science & Engineering, Thapar Institute of Engineering & Technology, Patiala 147001, India
2
Computer Information Systems, Faculty of Information & Communication Technology, University of Malta, MSD 2080 Msida, Malta
3
Computer Science Department, University of Liverpool, Liverpool 14 3PE, UK
*
Author to whom correspondence should be addressed.
Electronics 2021, 10(11), 1320; https://doi.org/10.3390/electronics10111320
Submission received: 22 April 2021 / Revised: 25 May 2021 / Accepted: 26 May 2021 / Published: 31 May 2021
(This article belongs to the Special Issue Resource Management in Cloud/Edge Computing)

Abstract

:
Workflow scheduling is one of the significant issues for scientific applications among virtual machine migration, database management, security, performance, fault tolerance, server consolidation, etc. In this paper, existing time-based scheduling algorithms, such as first come first serve (FCFS), min–min, max–min, and minimum completion time (MCT), along with dependency-based scheduling algorithm MaxChild have been considered. These time-based scheduling algorithms only compare the burst time of tasks. Based on the burst time, these schedulers, schedule the sub-tasks of the application on suitable virtual machines according to the scheduling criteria. During this process, not much attention was given to the proper utilization of the resources. A novel dependency and time-based scheduling algorithm is proposed that considers the parent to child (P2C) node dependencies, child to parent node dependencies, and the time of different tasks in the workflows. The proposed P2C algorithm emphasizes proper utilization of the resources and overcomes the limitations of these time-based schedulers. The scientific applications, such as CyberShake, Montage, Epigenomics, Inspiral, and SIPHT, are represented in terms of the workflow. The tasks can be represented as the nodes, and relationships between the tasks can be represented as the dependencies in the workflows. All the results have been validated by using the simulation-based environment created with the help of the WorkflowSim simulator for the cloud environment. It has been observed that the proposed approach outperforms the mentioned time and dependency-based scheduling algorithms in terms of the total execution time by efficiently utilizing the resources.

1. Introduction

Most of the business processes [1] can be represented in terms of a workflow. Workflow can be defined as a non-directed acyclic graph (DAG) [2] based structure having a group of connected tasks in a parent to child relationship. There is no parent to parent or child to child relationship in a workflow. It means that the tasks on the same level are not connected to each other, and the connectivity of tasks can be done from higher levels to lower levels only. The task invocation, synchronization, and information flow between the different tasks can be represented in a specific order described by the workflow management [3]. A scientific workflow management system (WMS) [4] is used to specify and execute the processing of the complex data. The biggest problem for WMS is scheduling because it is very difficult to identify the resource availability in the central pool of resources at the time of execution. Workflow scheduling is a challenging job as a proper sequence of the workflow tasks for execution needs to be created. The mapping and management of a workflow’s tasks on shared resources is done with the help of scheduling [5,6].
Workflow management systems [3,7] are basically used for appropriate management and execution of the workflow tasks. The WMS mainly consists of five major entities, including workflow design, information retrieval techniques, scheduling of workflow tasks, fault tolerance, and data movement as depicted in Figure 1.
A workflow structure [8] mainly describes the connection between different tasks of a workflow. It can be of two types such as DAG and Non-DAG based structures. DAG-based workflow structure can further be classified into three main categories: sequence, parallelism, and choice.
Non-DAG structures are superset of DAG and include the iteration pattern [9] in which the tasks can be executed iteratively. The overall design of a workflow can be explained with the help of workflow structure, its model and composition system as shown in Figure 2. The workflow model [10] is used to define the workflow structure and is of two types; abstract and concrete [11]. Workflow composition system helps user to add different components to the workflow [2,12,13].

1.1. Existing Challenges

Scheduling workflow’s tasks to appropriate cloud resources is a challenging job as it depends on the QoS requirements of the cloud applications. In cloud computing, due to heterogeneity; uncertainty and resource mobilization; resource scheduling is a current area of research in demand. Different criteria for scheduling various resources and the parameters, requires different categories of resource scheduling techniques [14]. Further, major challenges in the workflow scheduling are: (1) how to assign appropriate cloud services to each workflow’s task; (2) how to deal with cloud infrastructure variability; (3) how to consider the limits of concurrency in case of multiple tasks running in parallel; (4) how to solve the problem of data transfer between different workflow’s task, etc. [15]. According to the authors [11] the challenges are: (5) resource management, economic barriers, such as costs for migrating workflow’s task from one cloud service provider (CSP) to another; (6) legal issues such as data location or destination can be defined in advance; (7) security issues such as single sign-on authentication method in the inter-cloud environments, monitoring cloud resources, portability to move them from one cloud to another, and service-level agreement (SLA) having global SLAs between a federation and its customers are common issues.

1.2. Research Contribution

The major contribution toward resource scheduling in cloud computing is the proposed P2C algorithm. The proposed P2C algorithm is used to overcome the limitations of the existing time and dependency-based schedulers for scientific applications. The proposed algorithm considers the following parameters at a time:
1.
Parent to child node dependencies (which parent has the maximum number of child nodes);
2.
Child to parent node dependencies (which child has the minimum number of parent odes);
3.
Time of different tasks (in case of a tie in the second condition, as mentioned above) present at different levels in the workflow. The task having maximum time in the ready queue will be scheduled in case of the same dependency ratio.
The key objectives of the proposed algorithm are: (1) to reduce the overall execution time and (2) to utilize the resources efficiently. The proposed P2C algorithm has been validated by using the simulation-based environment created with the help of the WorkflowSim simulator. It has been observed that the proposed P2C algorithm outperforms the existing time and dependency based schedulers in terms of total execution time by utilizing the resources efficiently for the existing scientific applications.

1.3. Paper Organization

The rest of the paper is divided into six sections. In Section 2,the existing literature review and all the components of the workflow scheduling along with their execution environment are discussed. Section 3 describes in detail various scientific applications and their architectures. In Section 4, the problem formulation is discussed. A detailed description of the methodology used to solve the problem, along with the working of proposed P2C algorithm is given in Section 5. Section 6 describes the experimental setup and the workflows in detail with the statistical result analysis. Finally, Section 7 concludes the overall work with the future scope of the paper.

2. Workflow Scheduling

One of the main issues of cloud computing is resource scheduling. Users can use the cloud services anytime from anywhere with the help of a stable internet connection. However, users will not have direct access to the cloud resources; instead, special application programming interfaces (API) should assist the resources on-demand. Allocating resources for cloud computing to a significantly changing request for the resources based on end-user application’s usage pattern is a big challenge. The primary purpose is not only to optimize the resource allocation applications but also to improve resource utilization. There are numerous resource allocation algorithms, models, frameworks, and policies. These assist in allocating or transferring the resources that have proven to be helpful for both cloud service consumers (CSC) and cloud service providers (CSP). Although there are some conditions for resource scheduling, which are not appropriate for resource allocation to customers, these conditions can be highlighted in terms of factors such as: (1) if the cloud has a limited number of resources, then resource shortage may occur; (2) if CSPs has fewer resources for the end-user based on the policies and procedures; (3) allocation of additional resources based on customer’s ongoing applications may violate processing strategies; (4) if more than two end-users try to get the same resource simultaneously, then resource congestion may occur; (5) if substantial resources are available within the cloud, but the cloud applications does not assign them to the relevant CSCs request, then resources may be lost [16].
Workflow scheduling [17] finds a correct sequence of task execution by following the scheduling criteria. The components of workflow scheduling are shown in Figure 3. Scheduling architecture is very important when it comes to the quality, efficiency, and effectiveness of a project. The layout of scheduling architecture is organized into three categories: centralized, decentralized, and hierarchical. In the centralized work environment, one central scheduler takes all scheduling decisions for all the workflow based activities.Whereas in a decentralized approach, there are multiple schedulers but no central controller for these different schedulers to help them to communicate with each other. However, in the case of hierarchical scheduling, there is a central controller which is used to control not only the workflow execution but also the sub-workflows assigned to the lower-level schedulers [18,19].
Scheduling decisions [21] are divided into two types, such as local and global. Decisions made based on the work or sub-employment are known as local decisions, whereas the decisions based on all employment are called global decisions. Global decision-making processes provide better overall results because only one job or fewer tasks in the workflow are considered in a local decision-making process [22].
Abstract workflow models can be transformed into concrete models by using two schemes: static and dynamic. In the static scheme, the dynamic changes in resources are not considered. Whereas in the case of dynamic schemes, concrete models are generated before execution of workflow with the help of static information, such as the execution environment [23]. Static schemes can further be categorized into two types: user-directed and simulation-based [24]. The dynamic scheme can further be divided into prediction based as well as just-in-time. This scheme considers both static and dynamic information about resources used for making scheduling decisions at run-time. Prediction-based planning scheme considers dynamic information along with some results based on prediction and just-in-time scheme makes a decision at the execution time only [8,25,26].
Performance driven scheduling strategies [27] achieve the highest performance for the user-defined QoS parameters as the workflow’s tasks mapped with the resources that give the optimal performance [28]. Most operational scheduling strategies focus on increasing the scope of the workflow. Market-driven scheduling strategies focus on resource availability, cost allocation, quality budget, and time frames. The market model used to organize the workflow has become an available resource that leads to lower costs. The trustees’ strategy focuses on security and reputation for the resources [29,30].
Effective resource scheduling techniques are used to reduce the execution cost and the execution time; power consumption and deal with other QoS requirements [31]. The QoS requirements [32] may consist of quality attributes, such as reliability, security, availability, and scalability, etc. Resources are seen to be more challenging because both the CSCs and CSPs are not ready to share information. The first objective of resource scheduling is to identify adequate resources requirements for appropriate and effective utilization of resources. The second objective of resource scheduling is to identify the sufficient and proper workload and resources to support scheduling multiple workloads to meet many QoS requirements. Therefore, resource scheduling considers the execution time of different workloads. The overall performance depends on the different types of workloads (heterogeneous) and QoS requirements with similar workload (homogeneous) [14].
The authors [33] considered the task scheduling on clouds as a three queue (TQ) process based on three queues and dynamic preferences. First of all, scheduling has been decided on all the tasks in the available queue based on the priority of tasks. Secondly, subsequent tasks are divided separately into different groups based on input data and output data volume, the number of nodes currently running, tasks completion time, disk I/O rate, etc. Finally, the tasks are rearranged into the ready queue by considering all the parameters defined in the second queue.
The authors [34] proposed a solution to maintain a random cloud computing network. The goal is to satisfy QoS parameters by maximizing resource utilization. The proposed scheme increases the resource utilization, as well as reduces the resource consumption and execution time of the applications. However, there is a need for a mapping mechanism between the already allocated resources and the minimum requirement of resources to complete the execution process.
The authors [35] proposed a game-theoretical framework for real-time task scheduling in the cloud computing. In the proposed model, the task acts as a player, and the VM acts as a tactic, and the player’s payoff indicates the player’s completion time and waiting time. The proposed model is very effective in reducing the total time and waiting time of all the tasks in the workflow.
A new convenient and dynamic scheduling algorithm is required in the cloud environment for efficient resource utilization. Some of the existing static algorithms include first in first out (FIFO) [36], shortest job first (SJF) [37], round robin (RR) [38], etc. In contrast, dynamic algorithms do not require any advanced information about VMs and tasks; however, the VM requires constant monitoring. These algorithms are more accurate, efficient, and appropriate in cloud environments. When any VM is overloaded, so the work being done on this VM can be intentionally transferred to an under-loaded VM [39]. Dynamic RR [40], heterogeneous early finish time (HEFT) [41], etc., are a few examples of dynamic scheduling algorithms widely applied in cloud environments.
A novel multi-objective workflow scheduling approach in IaaS clouds [42] is proposed to optimize the cost and workflow’s makespan for real-world scientific applications. The authors [43] proposed a multi-purpose workflow-scheduling algorithm based on decomposition. The proposed model uses Pareto front solutions to achieve at least as good as scheduling results instead of repeatedly implementing the single-objective scheduling algorithm with multiple constraints. Furthermore, the authors [44] proposed a dynamic fault-tolerant workflow scheduling (DFTWS) approach with hybrid spatial and temporary re-implementation schemes. Firstly, DFTWS calculates the time characteristics of each task and predicts the critical path of the workflows. Secondly, DFTWS identifies the appropriate virtual machine (VM) for each task according to the task requirement and budget quota in the initial resource allocation phase. Finally, DFTWS manages online scheduling, making real-time-error-tolerant decisions based on the failure type and task critique during the workflow execution.
The authors [45] proposed a new, budget deadline aware scheduling (BDAS) algorithm that addresses the scheduling of workflows under the budget and deadline constraints for scientific workflows in the IaaS cloud. The proposed heuristic satisfies the budget and time constraints, while the cost–time trade-off is introduced over heterogeneous cases.
The authors [46] proposed a novel resource prediction-based scheduling technique that automates the allocation of resources for scientific application in a virtualized cloud environment. Firstly, the proposed predictive model is trained on datasets that simultaneously generate the functions of a scientific application in the cloud. Secondly, resources are determined based on the production of the proposed estimation model. The main goal of the resource prediction-based scheduling technique is to efficiently manage resources for virtual machines, reduce execution time, and improve error rates and accuracy. Additionally, to manage fluctuating demand for resources, resources need to be managed efficiently. Furthermore, the authors [47] focuses on the design of a prediction-based scheduling approach that maps the function of a scientific application with appropriate VMs by combining the features of swarm intelligence and a multi-objective decision-making process. The proposed approach is used to improve accuracy rate, execution time, cost, and SLA breach rate.
The initial idea of Spring scheduling in terms of computer architecture has been proposed by the authors [48,49]. According to the authors, Spring scheduling co-processor or multiprocessor have been considered as very large-scale integration (VLSI) accelerator for real-time system. Co-processors can be used for both static and dynamic scheduling. Many different approaches and their combinations can be used with the help of Spring scheduling, such as highest value first, earliest deadline first, and earliest available time first, etc. The Spring scheduler works on parallel structure for standard scheduling of several tasks, number of resources, and the internal criteria of scheduling. Trakadas et al. [50] defined the basic building parts and levels of a decentralized hybrid cloud MEC architecture that results in a platform-as-a-service (PaaS). The stakeholder ecosystem is also examined in order to provide a wide view on the business prospects of the platform.

3. Scientific Applications

Scientific applications are used to simulate real-world activities using mathematics. Real-world objects are turned into mathematical models, and their actions are simulated by executing the formulas. In this section, the various scientific applications used for the evaluation purpose are described in detail. These scientific applications are represented in terms of the workflows.

3.1. CyberShake

CyberShake workflow [51] is used to identify earthquake hazards by identifying the earthquake ruptures having a moment magnitude value greater than 6. CyberShake workflow is parallel and can be represented in 5 levels as shown in Figure 4.

3.2. Montage

The montage application [52,53] combines together many input images to create sky mosaics using input images in the flexible image transport system (FITS) format. The montage application has pipeline structure and comprising of nine levels, as shown in Figure 5.

3.3. Epigenomics

The epigenomics [52,53] workflow is used to automate various operations in genome sequence processing. Epigenomics workflow also has a pipeline structure and has eight levels, as shown in Figure 6. The overall input to the workflow are sequential data obtained for multiple “lanes” from the genetic analysis process.

3.4. Inspiral

Inspiral workflow [52] is used to generate and analyze dynamic gravitational wave-formats from data collected during the integration of integrated binary systems. Inspiral workflow has a parallel and pipeline structure and has six levels, as shown in Figure 7.

3.5. SIPHT

The SIPHT workflow [52,53] is used to automatically perform searches for uninterrupted RNA (sRNA) viruses in the National Center for Biotechnology Information (NCBI) database. All SIPHT workflows have almost identical structures, and larger workflows can be made by combining smaller independent workflows. The only difference is in the structure of any two events in the number of Patser’s works. The SIPHT workflow has a pipeline structure, as shown in Figure 8.

4. Problem Formulation

In this section, the research problem has been formulated using the following notations:
Let us consider that W denotes the list of workflows where individual ith workflow is denoted by w i . Each workflow consists of a group of jobs where ith job at kth level is denoted by j i k .
Let us consider that the list of virtual machines (resources) be denoted by VM where individual ith virtual machine is denoted by v m i .
Let ‘C’ be the category list of workflows and jobs, where c i denotes the individual category.
Definition 1.
Let us consider that f 1 denotes one to one mapping function between a job and its burst time.
f 1 : { j i k Δ t b j i k w k }
where Δ t b is the burst time of job j i . There exists dependencies between different jobs present in a workflow.
Definition 2.
Let us consider that f 2 be a one to many mapped function between jobs at different levels.
f 2 : { j i k 1 j l k 2 j i k 1 , j l k 2 w m k 1 < k 2 }
Definition 3.
Let us consider that f 3 be a one to one mapped function between workflow and its execution time.
f 3 : { w i Δ t e w i w i W }
where Δ t e w is the total execution time of the workflow w i .
Definition 4.
Let us consider that f 4 be a one to one mapped function between job and its execution time.
f 4 : { j i Δ t e j i j i w k }
where Δ t e j is the total execution time of the job j i .
Definition 5.
Let us consider that f 5 be a one to one mapped function between a job and its execution status at time ‘t’
f 5 : { j i t b 1 j i t w k b 1 { 0 , 1 , 2 } }
Definition 6.
Similarly, f 6 is a one to one mapped function between workflow and its execution status at time ‘t’
f 6 : { w i t b 2 w i t W b 2 { 0 , 1 , 2 } }
where ‘0’ indicates not started yet, ‘1’ indicates pending and ‘2’ indicates executed successfully.
Definition 7.
Let us consider that f 7 be a one to one mapped function between ith virtual machine and its computational power
f 7 : { v m i c k v m i V M }
Definition 8.
Let us consider that f 8 be a one to one mapped function between workflow and its category ‘C’
f 8 : { w i c j w i W c j C }
Definition 9.
Similarly, let f 9 be a one to one mapped function between job and its category ‘C’
f 9 : { j i c k j i w c k C }
Definition 10.
Let us consider that f 10 be a one to one mapped function between virtual machine and its occupancy status
f 10 : { v m i b 3 v m i V M b 3 { 0 , 1 } }
where ‘0’ indicates not occupied, ‘1’ indicates occupied.
Definition 11.
Let us consider that f 11 be a one to many mapped function between workflow and its jobs
f 11 : { w i j k w i W j k w i n u m ( j k ) 1 }
Definition 12.
Similarly, f 12 is a one to one mapped function between workflow and its job, where job is the starting job
f 12 : { w i j k w i W j k w i }
Definition 13.
Similarly, f 13 is a one to one mapped function between a job and a virtual machine, where the job is assigned to a virtual machine for its execution
f 13 : { j k v m i j k w l v m i V M }
Definition 14.
Similarly, f 14 is a one to many mapped function between workflow jobs
f 14 : { j i k 1 j l k 2 j i k 1 , j l k 2 w m k 1 > k 2 }
The objective function of the research work is given in Equation (15)
k m i n i = 1 n u m ( w k ) Δ t e j i
subject to constraints
c 1 : i , j f 7 ( v m i ) = f 7 ( v m j )
c 2 : i , m f 9 ( j i k ) = f 9 ( j l k )
c 3 : ( f 5 ( j i t ) = = 2 ) f 5 ( f 2 ( j i t ) ) = 2
c 4 : ( f 5 ( j i t ) = = 1 ) f 5 ( f 2 ( j i t ) ) = 2
c 5 : j i w , w i W , c i C , v m i V M
In this section, the problem formulation has been described with the main objectives of the paper, i.e., (1) to reduce the overall execution time of any scientific application, (2) to utilize the resources efficiently, with some constraints such as computational power of each virtual machine is considered to be the same as represented in Equation (16). Furthermore, any job represented as a node in the workflow can only be scheduled for the execution whenever all the predecessors of that node have been executed successfully represented with the help of Equations (18) and (19).

5. Proposed Solution

In this paper, mainly time based scheduling algorithms, such as FCFS, min–min, max–min [54], and (MCT) [55], along with dependency-based scheduler, such as MaxChild [56], have been considered. The MaxChild [56] algorithm is another one-way dependency-based scheduling algorithm that considers the dependency between the parent to child node only. However, there was no consideration given to the dependency between a child to a parent node. To overcome the limitation of these time-based schedulers, multi-dependency, and time-based scheduling algorithm is proposed and compared with the existing approaches. The proposed P2C approach considers the parent to child node relationship (dependencies), child to parent node relationship (dependencies), and the burst time of different tasks present at different levels in the workflow. The main objective of the proposed approach is to reduce the overall execution time of scientific applications, such as CyberShake [51], montage, epigenomics, inspiral, and SIPHT [52], etc., by efficiently utilizing the resources.
The proposed P2C algorithm is used to overcome the limitations of the existing time and dependency-based schedulers for scientific applications. The proposed approach considers the following parameters at a time:
1.
Parent to child node dependencies (which parent has the maximum number of child nodes);
2.
Child to parent node dependencies (which child has the minimum number of parent nodes);
3.
Time of different tasks (in case of a tie in the second condition, as mentioned above) present at different levels in the workflow. The task having maximum time will be scheduled in case of the same dependency ratio.
The proposed P2C algorithm initially checks whether the tasks in the ready queue are less than available resources, or are very much greater than available resources. In cases such as this, the proposed approach will execute the task in the decreasing order of time. Then the P2C algorithm will schedule those child nodes whose parents have already been successfully completed. Maximum dependencies are considered from parent to child nodes, whereas the minimum number of dependencies are considered from child to parent nodes.

Working of the Proposed P2C Algorithm

The step by step working of the proposed P2C algorithm is explained with the help of Figure 9 and according to the steps given in Algorithm 1. Further, to check the status of the ready queue, the procedure READYQUEUESTATUS is described in Algorithm 2.
Montage workflow has been considered to have 25 nodes from the dataset file Montage_25. Further, an execution overhead of 0.21 ms has been added to execute the root node represented by R. The tasks numbered from 22 to 25 have been combined into a single task due to the montage workflow pipeline structure in Figure 5. The step by step description of the P2C algorithm for the execution process is given as follows:
Execution 1:
Step 1: Creation of the parent–child table to find the child nodes of each task in the workflow at level 1.
First of all, the proposed P2C algorithm will check the parent to child dependencies from the first level. In level 1, there are five tasks numbered from 1 to 5. Therefore, the parent–child table, Table 1 has been created to represent the parent–child dependencies.
Step 2: Sorting the ready queue based on child to parent dependencies.
For M o n t a g e _ 25 Dataset file, there are 5 VMs available at a time to execute the tasks of a workflow. In level 1, there are only five tasks to be scheduled on 5 VMs. Further, there is a need to sort the tasks according to child–parent node dependencies. Whereas, in the case of more than one task, there is the need to sort the tasks based on the burst time in decreasing order, as depicted in Table 2.
Correct sequence of tasks in the ready queue will become in the order {2, 4, 1, 3, 5} after sorting the tasks based on the number of parents, as well as decreasing order of time.
Step 3: Scheduling of tasks to appropriate VM.
With the help of Step 1 and Step 2, the ready queue of tasks have been found for the scheduling. At the same time the final list of VMs have also been finalized. The next step is to schedule task 2 V M 1 , task 4 V M 2 , task 1 V M 3 , task 3 V M 4 , task 5 V M 5 . After the first run, tasks numbered from 1 to 5 have been executed and tasks 6–14 are in the available queue. Now, execute all the steps of the algorithm until all the tasks have finished their execution successfully.
Execution 2:
Step 1–2: Creation of the parent–child table to find the child nodes of each task in the workflow at level 2
Each task has only one child, i.e., task 15. Therefore, there is a need to sort the tasks numbered from 6 to 14 in decreasing order of time. Hence, the correct sequence of the tasks in the ready queue will become in the order {8, 9, 13, 6, 7, 12, 10, 14, 11}.
Step 3: Scheduling of tasks to appropriate VM
Schedule task 8 V M 5 , task 9 V M 4 , task 13 V M 3 , task 6 V M 2 , task 7 V M 1 . In this step, we have reversed the VM allocation sequence due to the availability of both free VM and un-allocated tasks in the same order. After the second run, tasks numbered from 1 to 9 and task 13 have been executed successfully and tasks numbered from 10 to 12 and 14 are in the available queue. Hence, the correct sequence for the task in the ready queue will appear in the order {12, 10, 14, 11}.
Execution 3:
Step 1–3: Schedule task 12 VM 1 , task 10 VM 2 , task 14 VM 3 and task 11 VM 2
After the third execution, tasks numbered from 1 to 14 have been completed successfully, and tasks 15 is available in available as well as ready queue. There is a need to schedule task 15 on VM1.
Execution 4:
Step 1: Creation of the parent-child table to find the child nodes of each task in the workflow at level 4
After the fourth execution, tasks numbered from 1–15 have been executed, and tasks 16 is available in available and ready queue. The initial level parent–child table for level 4 can be shown in Table 3.
Step 2: Sorting the ready queue based on child to parent dependencies
After sorting the tasks based on the number of parents and decreasing order of time, updated table can be represented in Table 4.
Execution 5:
After the fifth execution, tasks numbered from 1–16 have been executed, and tasks numbered from 17–21 are available to execution. Hence, the correct sequence of the tasks in the ready queue will become in the order {20, 19, 21, 17, 18}.
Step 1–3: Scheduling of tasks to appropriate VM.
Schedule task 20 V M 1 , task 19 V M 2 , task 21 V M 3 , task 17 V M 4 , task 18 V M 5 . After the sixth execution, tasks numbered 1–21 have been executed, and task 22 is available in available queue and ready queue. Further, there is need to schedule the task 22 on any of the free VM.
After seventh execution, tasks numbered from 1–22 have been executed and the P2C algorithm has also been stopped due to execution of all the tasks. Hence, overall execution time will be the sum of execution time consumed at each step of the algorithm.
E T _ T o t a l = E T 1 + E T 2 + E T 3 + E T 4 + + E T 5 + E T 6 + E T 7 + R o o t n o d e o v e r h e a d
E T _ T o t a l = 13.83 + 10.88 + 10.51 + 1.42 + 10.39 + 10.93 + 8.73 + 0.21
E T _ T o t a l = 63.9 ms
The simplified step by step description of the algorithm is represented with the help of flowchart as shown in Figure 10.

6. Experimental Setup and Results

Cloud applications have different requirements for configuration and deployment [57]. All the experiments have been conducted on a workstation having a 64 bit Windows 10 operating system, 6 GB memory, and Intel (R) Core (TM) i5-3367U @ 1.8GHz CPU. This section describes the details related to the cloud resources, simulation environment, workflows dataset, and results analysis.
Algorithm 1 Proposed P2C Algorithm.
Input : Workflow ( w i ) and Virtual Machines (VMs)
Output : Execution Time ( Δ t e )
1:
for i 1 1 to n u m ( W ) do
2:
for i 2 1 to n u m ( w i ) do
3:
   f 5 ( j i 2 = 0 )
4:
end for
5:
end for
6:
for i 1 1 to n u m ( V M ) do
7:
f 10 ( v m i 1 = 0 )
8:
end for
9:
TL={}
10:
RQ={}
11:
AQ={}
12:
for i 1 1 to n u m ( W ) do
13:
 RQ = RQ + f 12 ( w i )
14:
v m 1 r q 1
15:
f 5 ( r q 1 ) = 1
16:
 AQ = AQ+ {rq1}
17:
f 10 ( v m 1 ) = 1
18:
 RQ = RQ-{rq1}
19:
while f 5 ( f 11 ( w i ) ) ! = 2 do
20:
  if (num(RQ) - k ( f 10 ( v m k ) ) ≤ 0 ) (num(RQ) 2 × k ( f 10 ( v m k ) ) ) then
21:
   RQ = READYQUEUESTATUS()
22:
   if num(RQ) ≠ 0 then
23:
     R Q = S o r t ( R Q , n u m ( f 1 ( r q ) ) , 1 )
24:
    for i 1 1 to n u m ( V M ) do
25:
     if f 10 ( v m i 1 ) = = 0 then
26:
       v m i 1 r q 1
27:
       f 5 ( r q 1 ) = 1
28:
       R Q = R Q { r q 1 }
29:
       f 10 ( v m i 1 ) = 1
30:
       A Q = A Q + { r q 1 }
31:
     end if
32:
    end for
33:
   end if
34:
  else
35:
   RQ = READYQUEUESTATUS()
36:
   if num(RQ) ≠ 0 then
37:
     R Q = S o r t ( R Q , n u m ( f 2 ( r q ) ) , 1 )
38:
    for i 1 1 to n u m ( V M ) do
39:
     if f 10 ( v m i 1 ) = = 0 then
40:
       v m i 1 r q 1
41:
       f 5 ( r q 1 ) = 1
42:
       R Q = R Q { r q 1 }
43:
       f 10 ( v m i 1 ) = 1
44:
       A Q = A Q + { r q 1 }
45:
     end if
46:
    end for
47:
   end if
48:
  end if
49:
end while
50:
end for
Algorithm 2 Ready queue check condition.
1:
procedurereadyQueueStatus
2:
for i 1 1 to n u m ( A Q ) do
3:
  if a q i 1 = job’s execution completed then
4:
    f 5 ( a q i 1 ) = 2
5:
    f 10 ( f 13 ( a q i 1 ) ) = 0
6:
    A Q = A Q a q i 1
7:
    T Q = T Q + f 2 ( a q i 1 )
8:
  end if
9:
   end for
10:
for i 1 1 to T Q do
11:
   R Q = R Q + [ f 5 ( f 14 ( t q i 1 ) ) = = 2 ? t q i 1 : { } ]
12:
end for
13:
end procedure

6.1. Cloud Resources

A resource can be defined as a physical or logical component that is connected to a computer system. Cloud resources are mainly classified as fast computing, storage, communication, power, and security, etc. Every device which is connected to a computer system is considered to be a resource. Figure 11 represents the detailed view of various cloud resources which are provided on the user’s request through internet.
These resources are mainly classified in two categories: physical resources and logical resources. Physical resources consist of the system’s hardware such as processor, memory, and other peripheral devices connected to the system. Logical resources control the physical resources on a temporary basis and mainly consist of an operating system, power sources, APIs, databases, networking bandwidth, and protocols, etc., [58,59]. Cloud framework can be heterogeneous for the proper utilization of homogeneous, as well as heterogeneous resources. Further, quality of service (QoS), specific runtime and virtual technology, can be utilized to provide application’s objectivity and the virtualization.

6.1.1. Fast Computing Utility

These resources provide the computational power to the users for the execution of their applications. Cloud computing provides computation as a service (CaaS) as a utility to cloud users. These resources can be treated as the collection of physical machines, mainly processing power, memory, algorithms, operating system, and APIs. To provide these resources to users, the physical machines are deployed on the virtual environment, which is considered as a virtual machine [58,59].

6.1.2. Storage Utility

One of the main issues of end-users is to store their data into any of the convenient storage media, such as hard disk, floppy disk, and flash drive, etc. However, instead of purchasing their own storage space, it is better to take the storage space from CSPs on rent. Therefore, the cloud users store the data and information into an external remote database servers. Data can be accessed and transferred from the database servers to the user’s system through internet connectivity [58,59].

6.1.3. Communication Utility

It is also known as network utility or network as a service (NaaS). Communication utility consists of physical resources, such as intermediate devices, sensors, workstations, and logical resources, such as bandwidth, delay, protocols, and communication links. From the networking perspective, intermediate devices and communicating devices consist of modems, hubs, routers, and switches, etc. Further, the networking resources installed on physical machines within a data center are mainly organized in clusters [58,59].

6.1.4. Power Utility

Cloud computing consists of thousands of data servers and various types of other interconnecting devices. Energy efficiency is one of the main issues due to a lot of power consumption for the overall functioning of these service utilities. The power is consumed by the data servers and by cooling and supporting infrastructure, power distribution equipment, and networking equipment. Data centers usually take power from power utility providers, such as local power storage, especially from renewable energy sources including wind and solar energy [58,59].

6.1.5. Security

Cloud users demand various service utilities like IaaS, PaaS, and SaaS from the cloud provider. Therefore, these services must be highly secure, reliable, and available. Trust, integrity, privacy, authentication, and availability must be considered for the perspective of issues in security [58,59].

6.2. Workflows Dataset

There is no cost of infrastructure and services needed to test these applications in a repeatable and controlled environment. Initially, less number of virtual machines are created for the execution of smaller workflows. Later on, the number of virtual machines can be increased dynamically according to the execution of larger workflows. The different test cases are created by using the smaller as well as larger workflows as listed in Table 5.

6.3. Simulation Environment

The simulation environment evaluates a different kind of resource leasing on the provider’s side under different conditions with different load distribution. Implementation of the existing and proposed scheduling algorithms have been done on the WorkflowSim [60] simulator by setting it up on Net Beans IDE. The homogeneous virtual machines used in the simulation process have 512 MB memory, a CPU with 1000 MIPS, a bandwidth of 1000 BPS and 10 GB of image size. WorkflowSim extends the existing CloudSim [61] simulator that allows modeling and matching the cloud environment, data centers, virtual machines, and cloudlets, tackling only one load but is not suitable for scheduling workflows as many tasks need to be integrated. WorkflowSim, therefore, provides a higher level of workflow management over the CloudSim layer.

6.4. Statistical Analysis

Statistical analysis is the collection and interpretation of data to uncover patterns and trends. It is a part of data analytics. Statistical analysis can be used in collecting research descriptions, statistical modeling or designing surveys and studies. There are two main types of statistical analysis, i.e., descriptive and inference. After collecting the data, there is a need to analyze it by extracting the data, such as creating a pie chart, line graph, bar graph, and histogram, etc. All of these come down to using the right methods for statistical analysis, i.e., processing and collecting data samples to uncover patterns and trends. For this analysis, there are five different heuristics, such as average, standard deviation, regression, hypothesis test, and sample size determination [62]. The following terms are used in the descriptive analysis for this research:
Definition 15.
Mean, Standard Deviation, and Variance The mean (average) of a dataset is obtained by adding all the numbers in the dataset divided by the number of values in the dataset. A standard deviation is a measure of the distribution of a dataset from its mean. It measures the absolute diversity of a distribution; the higher the distribution or variance, the higher will be the standard deviation and the greater will be the magnitude of deviation of the value from its mean. The variance measures how well the dataset is distributed. A variance of zero indicates that all data values are the same. However, a high variance indicates that data points are still widely distributed from the mean and each other. The variance can be defined as the average of the squared distances from each point to the mean.
Definition 16.
Confidence Interval and Margin of Error The confidence interval indicates that the parameter is likely to fall between a pair of values near the mean. Confidence interval measures the level of uncertainty or certainty of samples They are usually built using 95% or 99% confidence levels. The margin of error tells about the percentage of points in results that differ from the actual population. For example, a 95% confidence interval with a 5% margin of error indicates that the actual statistics will be between 5% of the substantial population by 95% of the time. The following null hypothesis is considered for this research:
H0 = There is a significant difference of size, shape, and structure of the workflow on the proposed scheduling algorithm.
To verify the validity of the proposed algorithm and null hypothesis, the statistic “Two-factor ANOVA without Replication” has been selected and executed successfully at a confidence interval of 95% with 5% a margin of error. The basic difference between the two-factor ANOVA with and without replication is that the sample size is different [63]. In the replication technique, all the sample are mostly unique, and if this happens, then there is a need to calculate the mean independently. In this experiment also, we have different sized workflows as depicted in Table 5.
Definition 17.
Two factor ANOVA without Replication A factor is an independent variable. A level is some aspect of a factor; these are also called groups or treatments.The two factors considered are: factor A (Algorithms) and factor B (Workflow Size). Factor ‘Algorithms’ has 6 levels (FCFS, min–min, max–min, MCT, MaxChild, and P2C). Factor ‘Workflow Size’ has 4 levels (e.g., for montage workflow: Montage_25, Montage_50, Montage_100, Montag_1000) as depicted in Table 5. The levels for the factor ‘Algorithms’ are organized as rows and the levels for factor ‘Workflow Size’ are organized as columns. The two-factor ANOVA will either test for the main effects of factor ‘Algorithms’ or factor ‘Workflow Size’ as represented in Equation (22) or Equation (23) and depicted in Table 6.
H 0 : μ 1 = μ 2 = μ 3 μ m ( F a c t o r A l g o r i t h m s )
or
H 0 : μ 1 = μ 2 = μ 3 μ n ( F a c t o r W o r k f l o w S i z e )
Table 6 indicates that total (sum of all the tasks of CyberShake workflow in all four variants) execution time and average execution time for the proposed P2C algorithm is less as compared to other comparative scheduling. Further, the value of variance for the proposed approach is higher than the other scheduling algorithms. The descriptive results of Two factor ANOVA test are represented in Table 7.
Since the p-Value for the factor Algorithms (0.000 < 0.05) or (F = 7.996 > 2.901 = F-crit) as depicted in Table 7. Therefore, the null hypothesis is rejected, and there is no significant difference of size, shape, and structure of the CyberShake workflow on the proposed P2C algorithm. Similarly, the summary as well as descriptive results of two-factor ANOVA for montage workflow are depicted in Table 8 and Table 9.
Table 8 indicates that total and average execution time for montage workflow of proposed P2C algorithm is less than other comparative scheduling algorithms. Since the p-Value for the factor Algorithm (0.000 < 0.05) or (F = 12.752 > 2.901 = F-crit) as depicted in Table 9. Therefore, the null hypothesis is rejected, and there is no significant difference of size, shape, and structure of the montage workflow on the proposed P2C algorithm.
Table 10 indicates that total and average execution time for epigenomics workflow for the proposed P2C algorithm is less or equal to other comparative scheduling algorithms. Since the p-Value for the factor Algorithms (0.024 < 0.05) but (F = 2.526 < 2.901 = F-crit) as depicted in Table 11. Therefore, the null hypothesis for comparative scheduling algorithms with respect to proposed P2C algorithm rejected in some cases while accepted in a majority of the remaining cases. Due to this variable behavior, a significant difference of size, shape and structure of the epigenomics workflow on proposed scheduling algorithm can not be determined. Further, by carefully examining the total and average execution time for epigenomics workflow, it has been observed that either the proposed P2C algorithm outperforms or behaves similar to some existing algorithms. It could be possible due to the parallel, as well as pipeline, structure of the epigenomics workflow.
Table 12 clearly indicates that total and average execution time for inspiral workflow of proposed P2C algorithm is less as compared to other comparative scheduling algorithms.
Since the p-Value for the factor Algorithm (0.000 < 0.05) or (F = 40.834 > 2.901 = F-crit) as depicted in Table 13. Therefore, the null hypothesis is rejected, and there is no significant difference of size, shape, and structure of the inspiral workflow on proposed scheduling algorithm.
Table 14 indicates that total and average execution time of SIPHT Workflow for the proposed P2C algorithm is less compared to other comparative scheduling algorithms. Since the p-Value for the factor Algorithm (0.025 < 0.05) or (F = 3.471 > 2.901 = F-crit) as depicted in Table 15. Therefore, the null hypothesis is rejected, and there is no significant difference of size, shape, and structure of the SIPHT workflow on the proposed scheduling algorithm.

6.5. Results

Initially, all the scheduling algorithms have been implemented for existing scientific applications such as CyberShake, montage, epigenomics, inspiral and SIPHT with different tasks and different numbers of VMS but having the same structure. The experimental results have been compared with a different number of virtual machines for different sizes of workflows. The existing, as well as proposed, algorithms have been validated on all the test cases listed in Table 5 and results are shown in Figure 12, Figure 13, Figure 14, Figure 15 and Figure 16, etc.
It is clearly visible in Figure 12 that for CyberShake workflow, the proposed approach outperforms the existing approaches in terms of total execution time for less number of tasks. However, due to its parallel nature, in case of availability of more number of tasks, the proposed approach behaves similar to max–min scheduler and takes the same time as of max–min scheduler. Similarly, in some cases of Inspiral and Epigenomics workflow, the proposed algorithms behaves similar to max–min algorithms. It is clearly visible in Figure 13 for montage workflow and Figure 16 for SIPHT workflow, the proposed algorithm P2C outperforms the existing scheduling in terms of total execution time.

6.6. Complexity Analysis

The task scheduling problem in the cloud environment takes a large solution space. Moreover, the scheduling of n tasks onto m resources have been considered as an NP-Hard problem. Thus, it will take O ( n m ) time which is non-polynomial since, there is no existence of any algorithm that can find the optimal solution in a polynomial run time. The proposed P2C algorithm is based on a priority queue structure. In general, a priority queue ranks their tasks by a particular key with an order relation. Here, each element has its key, and such keys are not necessarily unique. Some major operations have been associated with priority queue such as T a s k L i s t , R e a d y Q u e u e , A v a i l a b l e Q u e u e , R E A D Y Q U E U E S T A T U S ( ) , etc. The priority-based task scheduling consists of the following properties:
1.
Each task has a priority associated with it;
2.
A task with high priority must precedes the low priority tasks;
3.
Two tasks can have same priority, however, such tasks will be scheduled as per their order in the queue.
Insertion time: Implementing priority queue without sorting would take O ( 1 ) time. However, implementing priority queue with linear sorting would take O ( n ) time.
Removal time: The functions like T a s k L i s t , R e a d y Q u e u e , A v a i l a b l e Q u e u e over un-sorted priority queue would take O ( n ) time. However, in case of linear sorted list it would take O ( n 2 ) time. Here, first r e m o v e ( ) task takes O ( n ) , the second O ( n 1 ) and so on until the last removal task takes only O ( 1 ) time. The total time needed to complete the first pass is: O ( n + ( n 1 ) + ( n 2 ) + + 2 + ( 1 ) ) = O ( i = 1 n i ) = n ( n + 1 ) 2 . Hence the total computational complexity for r e m o v a l ( ) is O ( n 2 ) .
In case of the non-linear sorting, if priority queue is using heap then, i n s e r t ( ) and r e m o v e ( ) each take O ( l o g n ) , where n is the number of tasks. Here, each pass takes O ( n l o g n ) time for i n s e r t ( ) , as well as r e m o v e ( ) .

7. Conclusions and Future Scope

Workflow scheduling is one of the significant issue in cloud computing among other popular issues, such as virtual machine migration, databases management, security, performance, fault tolerance, and server consolidation, etc. Workflow scheduling is a challenging job to find the proper sequence of workflow tasks for execution. It further depends upon the QoS requirements of the cloud applications. Due to heterogeneity; uncertainty and resource mobilization; resource scheduling is one of the hotspot area of research in demand. Different criteria for scheduling various resources and the parameters, requires different categories of resource scheduling techniques. In this paper, existing time-based scheduling algorithms such as first come first serve (FCFS), min–min, max–min, and minimum completion time (MCT), along with dependency-based scheduling algorithm MaxChild have been considered. The main objective of the existing time based schedulers is to reduce the workflow’s execution time, but there is no importance given to resource utilization. The proposed (P2C) algorithm mainly focuses on utilization of the resources efficiently. Further, The proposed P2C algorithm outperforms existing time and dependency based scheduling algorithms in terms of total execution time. The experimental results conclude that the existing schedulers have varying execution time based on the size, shape, and number of resources and virtual machines available. From the statistical analysis, it has been analyzed that proposed algorithm has no significance of size, shape, and structure of the workflow. There are still many challenges that need to be overcome to achieve more effective and comprehensive results. Further, at present, the results have been computed for the standard CyberShake, montage, epigenomics, inspiral, and SIPHT workflow with varying number of tasks and virtual machines. However, in the future, the work could be extended to other scientific systems beyond the natural environment, and verification can be tested over real-time cloud space.

Author Contributions

The initial writing on the ideas; Preparation, creation and/or presentation of the published work by those from the original research group, specifically critical review, commentary or revision—including pre-or post-publication stages have been done by first author V.P. The other authors, S.B. and L.G. have played the role of supervisors with oversight and leadership responsibility for the research activity planning and execution, including mentorship external to the core team. Further, they have provided valuable suggestions specifically on the critical review, commentary or revision—including pre-or post-publication stages. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Harmon, P. Business Process Change: A Business Process Management Guide For Managers and Process Professionals; Morgan Kaufmann: San Francisco, CA, USA, 2019. [Google Scholar]
  2. Li, J.; Li, X.; He, D. A directed acyclic graph network combined with CNN and LSTM for remaining useful life prediction. IEEE Access 2019, 7, 75464–75475. [Google Scholar] [CrossRef]
  3. Deelman, E.; Vahi, K.; Rynge, M.; Mayani, R.; da Silva, R.F.; Papadimitriou, G.; Livny, M. The evolution of the pegasus workflow management software. Comput. Sci. Eng. 2019, 21, 22–36. [Google Scholar] [CrossRef]
  4. De Carvalho Silva, J.; de Oliveira Dantas, A.B.; de Carvalho Junior, F.H. A Scientific Workflow Management System for orchestration of parallel components in a cloud of large-scale parallel processing services. Sci. Comput. Program. 2019, 173, 95–127. [Google Scholar] [CrossRef]
  5. Pandey, S.; Vahi, K.; da Silva, R.F.; Deelman, E.; Jiang, M.; Harrison, C.; Chu, A.; Casanova, H. Event-Based Triggering and Management of Scientific Workflow Ensembles. In Proceedings of the HPC Asia, Tokyo, Japan, 28–31 January 2018. [Google Scholar]
  6. Senkul, P.; Toroslu, I.H. An architecture for workflow scheduling under resource allocation constraints. Inf. Syst. 2005, 30, 399–422. [Google Scholar] [CrossRef]
  7. Yu, J.; Buyya, R. A taxonomy of workflow management systems for grid computing. J. Grid Comput. 2005, 3, 171–200. [Google Scholar] [CrossRef]
  8. Ma, X.; Gao, H.; Xu, H.; Bian, M. An IoT-based task scheduling optimization scheme considering the deadline and cost-aware scientific workflow for cloud computing. EURASIP J. Wirel. Commun. Netw. 2019, 2019, 249. [Google Scholar] [CrossRef] [Green Version]
  9. Marozzo, F.; Talia, D.; Trunfio, P. A workflow management system for scalable data mining on clouds. IEEE Trans. Serv. Comput. 2016, 11, 480–492. [Google Scholar] [CrossRef]
  10. Khennaoui, R.; Belala, N. Towards a Formal Context-Aware Workflow Model for Ambient Environment. In Proceedings of the International Conference on Smart Homes and Health Telematics, Hammamet, Tunisia, 24–26 June 2020; Springer: Berlin/Heidelberg, Germany, 2020; pp. 415–422. [Google Scholar]
  11. Talwani, S.; Singla, J. Comparison of Various Fault Tolerance Techniques for Scientific Workflows in Cloud Computing. In Proceedings of the 2019 International Conference on Machine Learning, Big Data, Cloud and Parallel Computing (COMITCon), Faridabad, India, 14–16 February 2019; IEEE: Piscataway, NJ, USA, 2019; pp. 454–459. [Google Scholar]
  12. Deelman, E.; Mandal, A.; Jiang, M.; Sakellariou, R. The role of machine learning in scientific workflows. Int. J. High Perform. Comput. Appl. 2019, 33, 1128–1139. [Google Scholar] [CrossRef] [Green Version]
  13. Deelman, E.; Vahi, K.; Juve, G.; Rynge, M.; Callaghan, S.; Maechling, P.J.; Mayani, R.; Chen, W.; Da Silva, R.F.; Livny, M.; et al. Pegasus, a workflow management system for science automation. Future Gener. Comput. Syst. 2015, 46, 17–35. [Google Scholar] [CrossRef] [Green Version]
  14. Singh, S.; Chana, I. A survey on resource scheduling in cloud computing: Issues and challenges. J. Grid Comput. 2016, 14, 217–264. [Google Scholar] [CrossRef]
  15. Kijak, J.; Martyna, P.; Pawlik, M.; Balis, B.; Malawski, M. Challenges for scheduling scientific workflows on cloud functions. In Proceedings of the 2018 IEEE 11th International Conference on Cloud Computing (CLOUD), San Francisco, CA, USA, 2–7 July 2018; IEEE: Piscataway, NJ, USA, 2018; pp. 460–467. [Google Scholar]
  16. George, S.S.; Pramila, R.S. A review of different techniques in cloud computing. In Materials Today: Proceedings; Elsevier: Amsterdam, The Netherlands, 2021. [Google Scholar]
  17. Zhou, J.; Wang, T.; Cong, P.; Lu, P.; Wei, T.; Chen, M. Cost and makespan-aware workflow scheduling in hybrid clouds. J. Syst. Archit. 2019, 100, 101631. [Google Scholar] [CrossRef]
  18. Barrett, E.; Howley, E.; Duggan, J. A learning architecture for scheduling workflow applications in the cloud. In Proceedings of the 2011 IEEE Ninth European Conference on Web Services, Lugano, Switzerland, 14–16 September 2011; IEEE: Piscataway, NJ, USA, 2011; pp. 83–90. [Google Scholar]
  19. Isaac, E.U.; Izuchukwu, A.C. Development of a Model Architecture for Job Scheduling. Sci. J. Circuits Syst. Signal Process. 2020, 9, 16. [Google Scholar] [CrossRef]
  20. Yu, J.; Buyya, R. A taxonomy of scientific workflow systems for grid computing. ACM Sigmod Rec. 2005, 34, 44–49. [Google Scholar] [CrossRef]
  21. Kintsakis, A.M.; Psomopoulos, F.E.; Mitkas, P.A. Reinforcement learning based scheduling in a workflow management system. Eng. Appl. Artif. Intell. 2019, 81, 94–106. [Google Scholar] [CrossRef]
  22. Varalakshmi, P.; Ramaswamy, A.; Balasubramanian, A.; Vijaykumar, P. An optimal workflow based scheduling and resource allocation in cloud. In Proceedings of the International Conference on Advances in Computing and Communications, Kochi, India, 22–24 July 2011; Springer: Berlin/Heidelberg, Germany, 2011; pp. 411–420. [Google Scholar]
  23. Zhou, X.; Zhang, G.; Sun, J.; Zhou, J.; Wei, T.; Hu, S. Minimizing cost and makespan for workflow scheduling in cloud using fuzzy dominance sort based HEFT. Future Gener. Comput. Syst. 2019, 93, 278–289. [Google Scholar] [CrossRef]
  24. Prakash, V.; Bala, A.G. An Efficient Workflow Scheduling Approach in Cloud Computing. Ph.D. Thesis, Thapar Institute of Engineering and Technology, Patiala, India, 2014. [Google Scholar]
  25. Masdari, M.; ValiKardan, S.; Shahi, Z.; Azar, S.I. Towards workflow scheduling in cloud computing: A comprehensive analysis. J. Netw. Comput. Appl. 2016, 66, 64–82. [Google Scholar] [CrossRef]
  26. Tong, Z.; Chen, H.; Deng, X.; Li, K.; Li, K. A scheduling scheme in the cloud computing environment using deep Q-learning. Inf. Sci. 2020, 512, 1170–1191. [Google Scholar] [CrossRef]
  27. Mansouri, N.; Javidi, M.M. Cost-based job scheduling strategy in cloud computing environments. In Distributed and Parallel Databases; Springer: Berlin/Heidelberg, Germany, 2019; pp. 1–36. [Google Scholar]
  28. Kumar, A.; Bawa, S. DAIS: Dynamic access and integration services framework for cloud-oriented storage systems. In Cluster Computing; Springer: Berlin/Heidelberg, Germany, 2020; pp. 1–20. [Google Scholar]
  29. Buyya, R.; Yeo, C.S.; Venugopal, S.; Broberg, J.; Brandic, I. Cloud computing and emerging IT platforms: Vision, hype, and reality for delivering computing as the 5th utility. Future Gener. Comput. Syst. 2009, 25, 599–616. [Google Scholar] [CrossRef]
  30. Hu, J.; Gu, J.; Sun, G.; Zhao, T. A scheduling strategy on load balancing of virtual machine resources in cloud computing environment. In Proceedings of the 2010 3rd International Symposium on Parallel Architectures, Algorithms and Programming, Dalian, China, 18–20 December 2010; IEEE: Piscataway, NJ, USA, 2010; pp. 89–96. [Google Scholar]
  31. Prakash, V.; Bala, A. Workflow Scheduling Algorithms in Grid and Cloud Environment—A Survey. J. Comput. Technol. 2014, 3, 2278–3814. [Google Scholar]
  32. Njenga, K.; Garg, L.; Bhardwaj, A.K.; Prakash, V.; Bawa, S. The cloud computing adoption in higher learning institutions in Kenya: Hindering factors and recommendations for the way forward. Telemat. Inform. 2019, 38, 225–246. [Google Scholar] [CrossRef]
  33. Yu, Y.; Su, Y. Cloud Task Scheduling Algorithm Based on Three Queues and Dynamic Priority. In Proceedings of the 2019 IEEE International Conference on Power, Intelligent Computing and Systems (ICPICS), Shenyang, China, 12–14 July 2019; IEEE: Piscataway, NJ, USA, 2019; pp. 278–282. [Google Scholar]
  34. Xia, W.; Shen, L. Joint resource allocation using evolutionary algorithms in heterogeneous mobile cloud computing networks. China Commun. 2018, 15, 189–204. [Google Scholar] [CrossRef]
  35. Patra, M.K.; Sahoo, S.; Sahoo, B.; Turuk, A.K. Game theoretic approach for real-time task scheduling in cloud computing environment. In Proceedings of the 2019 International Conference on Information Technology (ICIT), Bhubaneswar, India, 19–21 December 2019; IEEE: Piscataway, NJ, USA, 2019; pp. 454–459. [Google Scholar]
  36. Li, J.; Ma, T.; Tang, M.; Shen, W.; Jin, Y. Improved FIFO scheduling algorithm based on fuzzy clustering in cloud computing. Information 2017, 8, 25. [Google Scholar] [CrossRef] [Green Version]
  37. Nazar, T.; Javaid, N.; Waheed, M.; Fatima, A.; Bano, H.; Ahmed, N. Modified shortest job first for load balancing in cloud-fog computing. In Proceedings of the International Conference on Broadband and Wireless Computing, Communication and Applications, Taichung, Taiwan, 15 July–15 August 2018; Springer: Berlin/Heidelberg, Germany, 2018; pp. 63–76. [Google Scholar]
  38. Devi, D.C.; Uthariaraj, V.R. Load balancing in cloud computing environment using improved weighted round robin algorithm for nonpreemptive dependent tasks. Sci. World J. 2016, 2016, 3896065. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  39. Mazumder, A.M.R.; Uddin, K.A.; Arbe, N.; Jahan, L.; Whaiduzzaman, M. Dynamic task scheduling algorithms in cloud computing. In Proceedings of the 2019 3rd International Conference on Electronics, Communication and Aerospace Technology (ICECA), Coimbatore, India, 12–14 June 2019; IEEE: Piscataway, NJ, USA, 2019. [Google Scholar]
  40. Ghosh, S.; Banerjee, C. Dynamic time quantum priority based round robin for load balancing in cloud environment. In Proceedings of the 2018 Fourth International Conference on Research in Computational Intelligence and Communication Networks (ICRCICN), Kolkata, India, 22–23 November 2018; IEEE: Piscataway, NJ, USA, 2018; pp. 33–37. [Google Scholar]
  41. Samadi, Y.; Zbakh, M.; Tadonki, C. E-HEFT: Enhancement heterogeneous earliest finish time algorithm for task scheduling based on load balancing in cloud computing. In Proceedings of the 2018 International Conference on High Performance Computing & Simulation (HPCS), Orleans, France, 16–20 July 2018; IEEE: Piscataway, NJ, USA, 2018; pp. 601–609. [Google Scholar]
  42. Gao, Y.; Zhang, S.; Zhou, J. A hybrid algorithm for multi-objective scientific workflow scheduling in IaaS Cloud. IEEE Access 2019, 7, 125783–125795. [Google Scholar] [CrossRef]
  43. Bugingo, E.; Zheng, W.; Zhang, D.; Qin, Y.; Zhang, D. Decomposition based multi-objective workflow scheduling for cloud environments. In Proceedings of the 2019 Seventh International Conference on Advanced Cloud and Big Data (CBD), Suzhou, China, 21–22 September 2019; IEEE: Piscataway, NJ, USA, 2019; pp. 37–42. [Google Scholar]
  44. Wu, N.; Zuo, D.; Zhang, Z. Dynamic fault-tolerant workflow scheduling with hybrid spatial-temporal re-execution in clouds. Information 2019, 10, 169. [Google Scholar] [CrossRef] [Green Version]
  45. Arabnejad, V.; Bubendorfer, K.; Ng, B. Budget and deadline aware e-science workflow scheduling in clouds. IEEE Trans. Parallel Distrib. Syst. 2018, 30, 29–44. [Google Scholar] [CrossRef]
  46. Kaur, G.; Bala, A. An efficient resource prediction–based scheduling technique for scientific applications in cloud environment. Concurrent Eng. 2019, 27, 112–125. [Google Scholar] [CrossRef]
  47. Kaur, G.; Bala, A. Prediction based task scheduling approach for floodplain application in cloud environment. In Computing; Springer: Berlin/Heidelberg, Germany, 2021; pp. 1–22. [Google Scholar]
  48. Niehaus, D.; Ramamritham, K.; Stankovic, J.A.; Wallace, G.; Weems, C.; Burleson, W.; Ko, J. The Spring scheduling co-processor: Design, use, and performance. In Proceedings of the 1993 Proceedings Real-Time Systems Symposium, Raleigh, NC, USA, 1–3 December 1993; IEEE: Piscataway, NJ, USA, 1993; pp. 106–111. [Google Scholar]
  49. Burleson, W.; Ko, J.; Niehaus, D.; Ramamritham, K.; Stankovic, J.A.; Wallace, G.; Weems, C. The spring scheduling coprocessor: A scheduling accelerator. IEEE Trans. Very Large Scale Integr. Syst. 1999, 7, 38–47. [Google Scholar] [CrossRef]
  50. Trakadas, P.; Nomikos, N.; Michailidis, E.T.; Zahariadis, T.; Facca, F.M.; Breitgand, D.; Rizou, S.; Masip, X.; Gkonis, P. Hybrid clouds for data-intensive, 5G-enabled IoT applications: An overview, key issues and relevant architecture. Sensors 2019, 19, 3591. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  51. Graves, R.; Jordan, T.H.; Callaghan, S.; Deelman, E.; Field, E.; Juve, G.; Kesselman, C.; Maechling, P.; Mehta, G.; Milner, K.; et al. CyberShake: A physics-based seismic hazard model for southern California. Pure Appl. Geophys. 2011, 168, 367–381. [Google Scholar] [CrossRef]
  52. Bharathi, S.; Chervenak, A.; Deelman, E.; Mehta, G.; Su, M.H.; Vahi, K. Characterization of scientific workflows. In Proceedings of the 2008 Third Workshop on Workflows in Support of Large-Scale Science, Austin, TX, USA, 17 November 2008; IEEE: Piscataway, NJ, USA, 2008; pp. 1–10. [Google Scholar]
  53. Juve, G.; Chervenak, A.; Deelman, E.; Bharathi, S.; Mehta, G.; Vahi, K. Characterizing and profiling scientific workflows. Future Gener. Comput. Syst. 2013, 29, 682–692. [Google Scholar] [CrossRef]
  54. Braun, T.D.; Siegel, H.J.; Beck, N.; Bölöni, L.L.; Maheswaran, M.; Reuther, A.I.; Robertson, J.P.; Theys, M.D.; Yao, B.; Hensgen, D.; et al. A comparison of eleven static heuristics for mapping a class of independent tasks onto heterogeneous distributed computing systems. J. Parallel Distrib. Comput. 2001, 61, 810–837. [Google Scholar] [CrossRef] [Green Version]
  55. Dong, F.; Akl, S.G. Scheduling Algorithms for Grid Computing: State of the Art and Open Problems; Technical Report; Queen’s University: Kingston, ON, Canada, 2006. [Google Scholar]
  56. Prakash, V.; Bala, A. A novel scheduling approach for workflow management in cloud computing. In Proceedings of the 2014 International Conference on Signal Propagation and Computer Technology (ICSPCT 2014), Ajmer, India, 12–13 July 2014; IEEE: Piscataway, NJ, USA, 2014; pp. 610–615. [Google Scholar]
  57. Di Cosmo, R.; Lienhardt, M.; Treinen, R.; Zacchiroli, S.; Zwolakowski, J.; Eiche, A.; Agahi, A. Automated synthesis and deployment of cloud applications. In Proceedings of the 29th ACM/IEEE International Conference On Automated Software Engineering, Vasteras, Sweden, 15–19 September 2014; pp. 211–222. [Google Scholar]
  58. Jennings, B.; Stadler, R. Resource management in clouds: Survey and research challenges. J. Netw. Syst. Manag. 2015, 23, 567–619. [Google Scholar] [CrossRef]
  59. Parikh, S.M.; Patel, N.M.; Prajapati, H.B. Resource management in cloud computing: Classification and taxonomy. arXiv 2017, arXiv:1703.00374. [Google Scholar]
  60. Chen, W.; Deelman, E. Workflowsim: A toolkit for simulating scientific workflows in distributed environments. In Proceedings of the 2012 IEEE 8th International Conference on E-Science, Chicago, IL, USA, 8–12 October 2012; IEEE: Piscataway, NJ, USA, 2012; pp. 1–8. [Google Scholar]
  61. Calheiros, R.N.; Ranjan, R.; Beloglazov, A.; De Rose, C.A.; Buyya, R. CloudSim: A toolkit for modeling and simulation of cloud computing environments and evaluation of resource provisioning algorithms. Software Pract. Exp. 2011, 41, 23–50. [Google Scholar] [CrossRef]
  62. Glen, S. Statistical Analysis. Available online: https://www.statisticshowto.com/statistical-analysis/ (accessed on 15 April 2021).
  63. Zaiontz, C. Two Factor ANOVA without Replication. Available online: https://www.real-statistics.com/two-way-anova/two-factor-anova-without-replication/ (accessed on 15 April 2021).
Figure 1. Elements of workflow management system [7].
Figure 1. Elements of workflow management system [7].
Electronics 10 01320 g001
Figure 2. Aspects of workflow design [7].
Figure 2. Aspects of workflow design [7].
Electronics 10 01320 g002
Figure 3. Components of workflow scheduling [20].
Figure 3. Components of workflow scheduling [20].
Electronics 10 01320 g003
Figure 4. CyberShake workflow.
Figure 4. CyberShake workflow.
Electronics 10 01320 g004
Figure 5. Montage workflow.
Figure 5. Montage workflow.
Electronics 10 01320 g005
Figure 6. Epigenomics workflow.
Figure 6. Epigenomics workflow.
Electronics 10 01320 g006
Figure 7. Inspiral workflow.
Figure 7. Inspiral workflow.
Electronics 10 01320 g007
Figure 8. SIPHT workflow.
Figure 8. SIPHT workflow.
Electronics 10 01320 g008
Figure 9. Montage Workflow for Dataset File Montage_25.
Figure 9. Montage Workflow for Dataset File Montage_25.
Electronics 10 01320 g009
Figure 10. Flowchart of proposed parent to child (P2C) Algorithm.
Figure 10. Flowchart of proposed parent to child (P2C) Algorithm.
Electronics 10 01320 g010
Figure 11. Classification of cloud resources [58].
Figure 11. Classification of cloud resources [58].
Electronics 10 01320 g011
Figure 12. Comparison of execution time for CyberShake workflow.
Figure 12. Comparison of execution time for CyberShake workflow.
Electronics 10 01320 g012
Figure 13. Comparison of execution time for montage workflow.
Figure 13. Comparison of execution time for montage workflow.
Electronics 10 01320 g013
Figure 14. Comparison of execution time for epigenomics workflow.
Figure 14. Comparison of execution time for epigenomics workflow.
Electronics 10 01320 g014
Figure 15. Comparison of execution time for inspiral workflow.
Figure 15. Comparison of execution time for inspiral workflow.
Electronics 10 01320 g015
Figure 16. Comparison of execution time for SIPHT workflow.
Figure 16. Comparison of execution time for SIPHT workflow.
Electronics 10 01320 g016
Table 1. Creation of the parent-child table at level 1.
Table 1. Creation of the parent-child table at level 1.
Parent12345
Child6797810121391011912141314
Table 2. Sorting of parent in decreasing order of children at level 1.
Table 2. Sorting of parent in decreasing order of children at level 1.
Parent24135
Child7810121391214679910111314
Table 3. Creation of the parent-child table at level 4.
Table 3. Creation of the parent-child table at level 4.
Parent16
Child1718192021
Table 4. Sorting of parent in decreasing order of children at level 4.
Table 4. Sorting of parent in decreasing order of children at level 4.
Parent16
Child2019211718
Table 5. Creation of Test Cases for different size of different workflows.
Table 5. Creation of Test Cases for different size of different workflows.
WorkflowNumber of TaskNumber of VMDataset File
CyberShake305CyberShake_30
5010CyberShake_50
10020CyberShake_100
100050CyberShake_1000
Montage255Montage_25
5010Montage_50
10020Montage_100
100050Montage_1000
Epigenomics245Epigenomics_24
4610Epigenomics_46
10020Epigenomics_100
99750Epigenomics_997
Inspiral305Inspiral_30
5010Inspiral_50
10020Inspiral_100
100050Inspiral_1000
SIPHT305SIPHT_30
6010SIPHT_60
10020SIPHT_100
100050SIPHT_1000
Table 6. Two way ANOVA Summary results for CyberShake Workflow.
Table 6. Two way ANOVA Summary results for CyberShake Workflow.
FactorsSUMMARYSumAverageVariance
AlgorithmsFCFS1566.79391.6948,472.73
Min-min1608.55402.1357,146.5
Max-min1499.74374.9346,607.81
MCT1566.79391.6948,472.73
MaxChild1563.36390.8447,722.96
P2C1456.5364.1249,551.19
Workflow SizeCyberShake_301579.98263.33172.31
CyberShake_501581.96263.6655.93
CyberShake_1001785.53297.58305.15
CyberShake_10004314.26719.04511.36
Bold numbers are results of proposed approach.
Table 7. Two way ANOVA descriptive results for CyberShake Workflow.
Table 7. Two way ANOVA descriptive results for CyberShake Workflow.
Source of VariationSSdfMSFp-ValueF-crit
Algorithms3798.6545759.737.99602.901
Workflow Size892,496.73297,498.913131.2423.39E+003.287
Error1425.1481595.009
Total897,720.523
Table 8. Two way ANOVA Summary results for Montage Workflow.
Table 8. Two way ANOVA Summary results for Montage Workflow.
FactorsSUMMARYSumAverageVariance
AlgorithmsFCFS866.94216.7375,982.15
Min-min870.54217.6376,802.02
Max-min866.08216.5275,850.97
MCT866.94216.7375,982.15
MaxChild866.63216.6575,903.93
P2C833.23208.374,158.48
Workflow SizeMontage_25340.7956.791.73
Monatge_50458.1876.367.33
Montage_100606.7101.1119.26
Montage_10003764.69627.4432.63
Bold numbers are results of proposed approach.
Table 9. Two way ANOVA descriptive results for Montage Workflow.
Table 9. Two way ANOVA descriptive results for Montage Workflow.
Source of VariationSSdfMSFp-ValueF-crit
Algorithms246.771549.35412.7525.85373E−052.901
Workflow1,363,981.1313454,660.38117,479.765.34794E−333.287
Error58.051153.87
Total1,364,285.95523
Table 10. Two way ANOVA Summary results for Epigenomics Workflow.
Table 10. Two way ANOVA Summary results for Epigenomics Workflow.
FactorsSUMMARYSumAverageVariance
AlgorithmsFCFS161,211.0240,302.752,520,942,674
Min-min174,101.1543,525.282,741,892,698
Max-min171,923.142,980.772,713,164,848
MCT161,211.0240,302.752,520,942,674
MaxChild161,211.0240,302.752,520,942,674
P2C161,211.3340,302.332,520,942,674
Workflow SizeEpigenomics_2433,578.685596.440.04
Epigenomics_4646,459.457743.240
Epigenomics_100230,865.938,477.6515,052,972.16
Epigenomics_997682,378.61113,729.7610,729,351.4
Table 11. Two way ANOVA descriptive results for Epigenomics Workflow.
Table 11. Two way ANOVA descriptive results for Epigenomics Workflow.
Source of VariationSSdfMSFp-ValueF-crit
Algorithms43,480,779.1658,696,155.8322.5260.0242.901
Workflow45,928,839,780315,309,613,2602688.07103.287
Error85,430,838.92155,695,389.261
Total46,057,751,39823
Table 12. Two way ANOVA Summary results for inspiral workflow.
Table 12. Two way ANOVA Summary results for inspiral workflow.
FactorsSUMMARYSumAverageVariance
AlgorithmsFCFS10,800.332700.084,953,313
Min-min12,054.63013.654,808,698
Max-min10,771.882692.974,884,867
MCT10,800.332700.084,953,313
MaxChild10,802.752700.684,957,883
P2C10,748.332687.084,884,225
WorkflowInspiral_3010,469.641744.9417,561.79
Inspiral_509741.81623.6331,777.85
Inspiral_1009350.481558.4110,026.64
Inspiral_100036416.36069.3812,884.62
Bold numbers are results of proposed approach.
Table 13. Two way ANOVA descriptive results for inspiral workflow.
Table 13. Two way ANOVA descriptive results for inspiral workflow.
Source of VariationSSdfMSFp-ValueF-crit
Algorithms336,530.6567,306.140.83402.901
Workflow Size88,302,172.0532.9E+0717,857.5903.287
Error24,723.993151648.27
Total88,663,426.6423
Table 14. Two way ANOVA summary results for SIPHT workflow.
Table 14. Two way ANOVA summary results for SIPHT workflow.
FactorsSUMMARYSumAverageVariance
AlgorithmsFCFS20,100.225025.051,058,606.8
Min-min20,575.485143.871,589,971.89
Max-min18,846.064711.51172,348.68
MCT20,100.225025.051,058,606.8
MaxChild20,199.085049.771,162,325.93
P2C18,375.084593.77296,019.17
WorkflowSIPHT_3026,058.774343.1231,144.96
SIPHT_6027,849.24641.5330.09
SIPHT_10026,859.254476.5448.18
SIPHT_100037,428.926238.15549,233.12
Bold numbers are results of proposed approach.
Table 15. Two way ANOVA descriptive results for SIPHT workflow.
Table 15. Two way ANOVA descriptive results for SIPHT workflow.
Source of VariationSSdfMSFp-ValueF-crit
Algorithms955,130.5555191,026.1113.4710.02562.901
Workflow Size14,066,486.634,688,828.87536.1203.287
Error1,947,151.2915129,810.086
Total16,968,768.523
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Prakash, V.; Bawa, S.; Garg, L. Multi-Dependency and Time Based Resource Scheduling Algorithm for Scientific Applications in Cloud Computing. Electronics 2021, 10, 1320. https://doi.org/10.3390/electronics10111320

AMA Style

Prakash V, Bawa S, Garg L. Multi-Dependency and Time Based Resource Scheduling Algorithm for Scientific Applications in Cloud Computing. Electronics. 2021; 10(11):1320. https://doi.org/10.3390/electronics10111320

Chicago/Turabian Style

Prakash, Vijay, Seema Bawa, and Lalit Garg. 2021. "Multi-Dependency and Time Based Resource Scheduling Algorithm for Scientific Applications in Cloud Computing" Electronics 10, no. 11: 1320. https://doi.org/10.3390/electronics10111320

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop