Given on Septemeber 28, 2000

1st Presentation

• I am doing my research on PVM. I enjoyed the seminar course that some of us took my sophomore year and I wanted to learn more about PVM and parallel computing. It is something that I am considering working on or with in the future.

1. Parallel Programming
• One way for increasing computational speed is by using multiple processors operating together on a single problem. The overall problem is split into parts, each of which is performed by a seperate processor in parallel. Parallel programming requires a suitable computing platform, either a single computer with multiple internal processors or a multiple of interconnected computers.
• It can be used as a low-cost and high-performance local computing resource for an organization which may need high computing power but cannot afford a supercomputer system or lack a sufficient network connection to an outside server.
• Shared Memory vs. Message Passing
1. Shared-Memory Porgramming
• Multiple processors connected to multiple memory modules, such that each processor can access any memory module. A shared memory multiprocessor system employs a single address space which means that each process has its own address block. Processes are communicating through shared variables.
2. Message Passing
• Complete computers communicate with each other by sending data back and forth across a connection. A process can only access the memory on the local machine. The message-passing multicomputer carries data from one process to the another as indicated by the program.
• Styles of Parallel Processing
1. SISD - Single Instruction, Single Data
• traditional single processor machine
2. SIMD - Singel Instruction, Multiple Data
• single operation acts on a number of variables
3. MISD - Multiple Instruction, Single Data
• shared memory machines, multiple processors accessing same memory space
4. MIMD - Multiple Instruction, Multiple Data
• distrbuted memory machines, beowulf clusters, Network of Workstations (NoWs). Multiple programs (often same program, but can be at different statges of excecution) running on differnt machines, with own memory space. Linked by high speed network. Typically requires explicit message passing calls.
2. PVM - Parallel Virtual Machine
• Background - Vaidy Sunderam and Al Geist started the development in 1989 because they wanted to do some research on heterogeneous computing. Heterogeneous in this context simply means consisting of dissimilar architectures. They needed a framework to explore this new area so developed the Parallel Virtual Machine. Jack Dongarra was also interested in heterogeneous computing and was instrumental in making PVM Version 2.0 publicly available. A key part of the design is this idea of a virtual machine - a set of hetergeneous hosts connected by a network that appears logically to the user as a single large parallel computer. PVM Version 3.0 was released in 1993 with an entirely new API. The API change was needed to allow a PVM application to run across a virtual machine composed of multiple large multiprocessors.
• Ups
• simple since it only has 38 funtions
• already dynamic and fault tolerant
• has both process and virtual machine control
• the virtual machine abstraction has made it popular
• easy to compile, install, and understand
• supports many implementations of Unix and Windows
• Downs
• perceived as slower than MPI on MPP's
• not a standard
• limited message calls - blocking send, blocking and non-blocking recv, and multicast
• Core Functions
• pvm_mytid(); - Enroll into PVM
• pvm_spawn(...); - Startup of other tasks
• pvm_send(dest,tag); - Blocking send
• pvm_pk*(...); - Pack data function
• pvm_recv(src,tag); - Blocking recv
• pvm_nrecv(src,tag); - Non blocking recv
• pvm_exit(); - Exit PVM
• Portability vs Interoperability
• PVM programs can be copied to different architectures, compiled and executed without modification
• PVM programs can run on heterogeneous computers at the same time
• It loses some performance in favor of flexibility
• Virtual Machine
1. Process Control
• ability to start and stop tasks
• find out which tasks are running
• possibly where they are running
2. Resource Control
• add/delete hosts from a virtual machine, either from the console or the program
• powerful for load balancing, task migrating, and fault tolerance
• serial part of the program can be run elsewhere and graphical interfaces can be added
• Fault Tolerance
• pvmds automatically detect the loss of network or hosts and reconfigure the VM
• hosts which cannot communicate with master pvmd for 3 minutes automatically terminate running PVM tasks and pvmd
• timeout period can be adjusted in src/ddpro.h
• Name Service
• nice feature since pvm is dynamic
• gives processes an easy way of finding each other
• Differences between MPI and PVM
• PVM is more common on clusters and MPI is more common on MPP's
• MPI has 130 funcitons vs. 34
• MPI is not fault tolerant
• MPI lacks dynamics
• MPI does not have process control
• Many people develope their own implementation of MPI
3. My Research Plans
• The pvm standard form has all nodes exposed to the network and any node can directly communicate with another node. The clusterized version permits a cluster (group of nodes) to be on a private network and not exposed to the general net - it then has a node or group of nodes that are dual-ported with multiple network cards - one to the internal net and the other to the external net. This machine then acts as a portal through which communication is routed to the cluster nodes not exposed to the general net. It is similar to ip masquerading techniques. The physical configuration of the nics does not matter - clusterized refers to the nodes not being exposed to the public net.
• I will be exploring the limitations and capabilities of the "clusterized" version of codes
• - will require writing application codes that use PVM
• - explore the scalability limitations of routing through a node
• - other scalability issues...
• - other limitations...
• - how to best exploit this capability in multiple cluster environments
• - documentation for use (we have little for this now)
• What's Next
• Get last pvm program from CS40 up and running on cluster
• Write three sample programs that use MISD, MIMD and SIMD
• View PVM source and look for comments about clusterized
• Read PVM User's Guide