Need for Dynamic Load Balancing under PVM

In PVM, the user tasks which comprise a parallel application are distributed over the nodes in a network. The nodes could be parallel/vector machines or workstations. Thus, a heterogenous computing environment is used to solve the problem. To extract the maximum performance from the parallel application, it is important to keep the idle time of tasks to a barest minimum. To ensure that each process gets an optimum load to be processed, a dynamic load balancing scheme is desirable. The relevant issues that the application developer has to take into consideration are :

1. All nodes in the network do not have identical computation capacities. Even among workstation class machines, a variety of processor architectures, network interfaces, and I/O subsystems can be encountered. Some machines may have processing rates which are an order of magnitude higher than some other machines in the network.
2. A parallel application running over a wide area network will encompass a range of networks which have different transfer rates and latencies. For e.g., ATM supports 622 Mb/s and 100 Mb/s Ethernet LANs have 10 Mb/s rates, WANs have 64 Kb/s (X.25), Token ring LANs have 16 Mb/s peak transfer rates. This results in different message transfer rates between tasks in the parallel application.

3. There are background jobs in most time sharing multiuser systems (typical unix workstations) so the CPU time that a task under PVM can get may vary from one run to another depending on the usage of the workstation.

4. A large number of algorithms are not structured nicely, and on breaking the data to be processed into work packets, each work packet may require a different processing time (even if we keep the above factors constant). This unequal processing time requirement cannot be always determined apriori and can be difficult to account for while designing the application.