PVM version 3.3 Frequently Asked Questions - Technical 22 June 1994 rev. 5 ------------------------------------------------------------------------ Index Installation Q4: Can you tell me how to install PVM in a place other than ~/pvm3? Q9: Machines at my site share a single /tmp directory. Q10: I would like have 2 different versions of PVM on the same machine. Q13: What do I have to type to install PVM? Q14: I get errors when building the PVM release. Q15: What versions of PVM are interoperable? The System Q1: Memory used by the pvmd increases when program runs. Q6: What does the "speed" field in the machine configuration mean? Q11: What do I call the host file? Q12: How do I read the on-line manual pages? Q16: How do I shut down PVM? Programming Q2: How can I spawn a process with a nice priority value? Q3: Where does stuff printed in tasks go? Q5: When I use pvm_mcast(), the sender doesn't get a copy of the message. Q7: Is message order between tasks preserved? Q8: How can I pack a struct in a message? Q17: The compiler can't find header files when I compile my application. Q18: I get missing symbols when I link my application. Debugging ------------------------------------------------------------------------ Q1: When I run a program under PVM the memory used by the pvmd increases. After the program stops, the percentage of memory doesn't decrease. A1: The pvmd data size will grow as it runs. It uses malloc() (which uses sbrk() probably) to get buffers and malloc never returns memory to the system. However, the buffers are freed, so malloc should use the same space the next time it needs buffers. I would expect the data size to grow when your program is first run, then stay more or less constant for subsequent runs. There could be a memory leak in the pvmd, though we don't know of any big ones. ------------------------------------------------------------------------ Q2: How can I spawn a process with a nice priority value? A2: Sorry, you can't. You'll have to call setpriority() in your code or you could replace your program with a shell script, f.e.: % cd ~/pvm3/bin/SUN4 % mv prog prog- % echo 'P=$0"-"; shift; exec nice -10 $P $@' > prog % chmod 755 prog Then when you spawn "prog" the shell script will exec "prog-" (the real thing) at a nice level. ------------------------------------------------------------------------ Q3: Why does stuff printed by child tasks appear in the master pvmd log file, while output from the parent task appears on the window where I start the parent? A3: The spawned tasks' output goes to their pvmds; their stdouts are not inherited from the parent task. Each pvmd collects output from its tasks through pipes and passes it to the master pvmd, which logs it in /tmp/pvml.UID and on its stdout. Additionally, stderr of spawned tasks is dup()ed to stdout - they're not separate. Finally, spawned tasks have no stdin, /dev/null is opened as file 0. Tasks started by hand have normal stdin, stdout and stderr connections. You can redirect output (stdout/err) from spawned tasks to the console, and to a file if you like. Type "help spawn" for more information. If the output is not redirected to the console, it goes to the master pvmd logfile as before. Output direction is inherited, so if you spawn a task from the console and redirect its output to the console, you'll also get output from all its children. You can also call pvm_catchout() in the master task of your program before spawing child tasks. Output from children will be sent to the master and printed on its stdout. Be sure to call pvm_exit() to have to master wait for all output from children before exiting. ------------------------------------------------------------------------ Q4: Can you tell me how to install PVM in a place other than ~/pvm3? A4: I'd suggest installing it in /usr/local/pvm3 . Unpack it there, setenv PVM_ROOT to /usr/local/pvm3 and type make to build everything. Then any user on the system should be able to setenv PVM_ROOT to /usr/local/pvm3 in .cshrc and have it work. They will share files from /usr/local/pvm3/{lib,include,conf}. Each user's executables will still be expected to be in ~/pvm3/bin/ARCH/, unless they change that path with the ep= host file option. Oh, users might want symlinks from their normal bin directories to /usr/local/pvm3/lib/{pvm,pvmd,aimk}. For example, foreach c (pvm pvmd aimk) ln -s /usr/local/pvm3/lib/$c ~/bin end Alternatively, /usr/local/pvm3/lib can be added to a user's shell path. ------------------------------------------------------------------------ Q5: When I use pvm_mcast(), the sender doesn't get a copy of the message, even if I include its tid in the address list. A5: The spec for mcast() has changed slightly from early releases of version 3 - it now removes the sender from the list. It seems to be more often the case that the sender doesn't want to get a copy of the message, even if it is in the list of addresses. If the sender must get the message too, you'll need to send() the message explicitly with, f.e.: pvm_send(pvm_mytid(), code); Or to do it more efficiently, after the mcast() do: pvm_setrbuf(pvm_getsbuf()); It will be as though the sender had just recv()d the message. ------------------------------------------------------------------------ Q6: What does the "speed" field in the machine configuration mean? Can I set it? A6: It's currently part of the outline of a load-balancing mechanism, the idea being that if you're running more than one task per host in the virtual machine, faster machines should get more tasks. The scheduler used by pvm_spawn() should look at the speeds and current loads when assigning new tasks to hosts. You can actually set the speed of each host (which defaults to 1000) to a value from 1..1 000 000 by using the host file sp= option. But spawn doesn't use the values right now. ------------------------------------------------------------------------ Q7: Is message order between tasks preserved? A7: Between two given tasks, A and B, message order is preserved. That is, if A sends ten messages to B using pvm_send(), B will download them in that same order. Note messages can be picked out of the middle of the receive queue in B, though, by causing pvm_recv() or pvm_nrecv() to select a particular message. The order of arrival _between_ direct messages (pvm_send()) and multicast messages (pvm_mcast() or pvm_bcast()) is not guaranteed. I think it stays consistent right now (v3.2) because the same routing path is taken by each message. Note that the path is not the same if using PvmRouteDirect routing - direct messages will go from task to task, while multicast messages will still be routed via the pvmds. The order between a direct-routed direct message and a multicast message is not preserved. ------------------------------------------------------------------------ Q8: How can I pack a struct in a message? A8: Unfortunately, that's a hard thing to do in C without some sort of preprocessing before compilation. You'll need to write special pack and unpack routines for any struct you want to pack, for example: struct foo { int *values; /* vector of (length) elements */ short length; double perf; int flags; }; int pkfoo(foop, cnt, std) struct foo *foop; /* vector of foos */ int cnt; /* number of foos */ int std; /* stride between foos */ { int cc; while (cnt-- > 0) { if (cc = pvm_packf("%hd %*d %lf %d", foop->length, (int)foop->length, foop->values, foop->perf, foop->flags)) return cc; foop += std; } return PvmOk; } int upkfoo(foop, cnt, std) struct foo *foop; int cnt; int std; { int cc; while (cnt-- > 0) { if ((cc = pvm_unpackf("%hd", &foop->length)) || (!(foop->values = (int*)malloc(foop->length * sizeof(int))) && (cc = PvmNoMem)) || (cc = pvm_packf("%*d %lf %d", (int)foop->length, foop->values, &foop->perf, &foop->flags))) return cc; foop += std; } return PvmOk; } Note that if the struct contains only primitive types (or other structs that contain only primitive types), that is, no pointers, _and_ you're passing it to a host of identical type (not just one with the same data representations (because they might not build structs the same way)), you could get away with: struct foo foo1; pvm_pkbyte((char*)&foo1, n * sizeof(struct foo), 1); ... pvm_upkbyte((char*)&foo1, n * sizeof(struct foo), 1); ------------------------------------------------------------------------ Q9: Machines at my site share a single /tmp directory. More PVM daemons can't start up because the file /tmp/pvmd.uid# already exists. A9: Get release version >= 3.2.3. This includes a compile-time option to change the name of the file in /tmp to /tmp/pvmd.uid#.hostname, avoiding the conflict. If you want this option enabled for all machine types, edit src/Makefile.aimk and add -DSHAREDTMP to the (uncommented) OPTIONS= line near the top. If you want it set only for certain architectures, edit the config files for those machines in conf/*.def and add -DSHAREDTMP to the ARCHCFLAGS= line in each. You'll need to rebuild the distribution and relink any applications using libpvm. ------------------------------------------------------------------------ Q10: I would like have 2 different versions of PVM (namely 3.1 and 3.2) on the same machine(s) and, possibly, to run them at the same time. A10: There are two problems: First, PVM doesn't like to run more than one pvmd per user (login) on a host. A build option (OVERLOADHOST) was added in version 3.2 allows more pvmds to run per host, provided they're in separate virtual machines. Essentially: if this option is enabled, the pvmd starting up doesn't quit if the /tmp/pvmd.uid file already exists (another pvmd is running). This means that tasks not spawned from PVM can only connect to the first pvmd running on a host (because it owns the address file in /tmp). The other problem is that this option didn't exist in version 3.1. I suppose it would work if you always start the v3.1 pvmd before the v3.2 pvmd (bulit with OVERLOADHOST defined). ------------------------------------------------------------------------ Q11: I don't know how to configure machines into PVM. When I create a file containing the host names I want to configure, in what directory should I put the file? What file name should I give to this file? A11: You can call it anything you like and put it anywhere. It's passed to PVM as a command line parameter. For example: ig 3; cat > H thud honk ^D ig 4; pvm H 3.3.1 t40001 pvm> conf 3 hosts, 2 data formats HOST DTID ARCH SPEED ig 40000 ALPHA 1000 thud 80000 SUN4 1000 honk c0000 ALPHA 1000 pvm> halt libpvm [t40001]: mxfer() EOF on pvmd sock ig 5; ------------------------------------------------------------------------ Q12: How do I read the on-line manual pages? A12: On-line manual pages compatible with most Unix machines are shipped with the source distribution. These reside in $PVM_ROOT/man and can be copied to some other place (for example /usr/local/man or used in-place. If the man program on your machine uses the MANPATH environment variable, try adding something like the following near the end of your .cshrc or .login file: if (! $?MANPATH) setenv MANPATH /usr/man:/usr/local/man setenv MANPATH ${MANPATH}:$PVM_ROOT/man Then you should be able to read both normal system man pages and PVM man pages by simply typing man subject. ------------------------------------------------------------------------ Q13: What do I have to type to install PVM? A13: The following commands can be used to download, unpack, build and install a release: (start in directory just above PVM root, for example $HOME or /usr/local) % ftp netlib2.cs.utk.edu Name: anonymous Password: your id, user@host.domain ftp> cd pvm3 ftp> bin ftp> get pvm3.3.1.tar.z.uu ftp> quit % uudecode pvm3.3.1.tar.z.uu % zcat pvm3.3.1.tar.Z | tar xf - % cd pvm3 % setenv PVM_ROOT $cwd % make ------------------------------------------------------------------------ Q14: I get errors when building the PVM release. A14: The compiler may print a few warning messages; we suggest you ignore these unless the build doesn't complete or until you have some other reason to think there is a problem. If you can't build the unmodified distribution ``out of the box'' on a supported architecture, let us know. ------------------------------------------------------------------------ Q15: What versions of PVM are interoperable? A15: The protocols used in building PVM are evolving, with the result that newer releases are not compatible with older ones. Compatibility is determined by the pvmd-task and task-task protocol revision numbers. These are compared when two PVM entities connect; they will refuse to interoperate if the numbers don't match. The protocol numbers are defined in src/ddpro.h and src/tdpro.h (DDPROTOCOL, TDPROTOCOL). As a general rule, PVM releases with the same second digit in their version numbers (for example 3.2.0 and 3.2.6) will interoperate. Changes that result in incompatibility are held until a major version change (for example, from 3.2 to 3.3). ------------------------------------------------------------------------ Q16: How do I shut down PVM? A16: The preferred way to shut down a virtual machine is to type "halt" at the PVM console, or to call libpvm function pvm_halt(). When shutting PVM down from the console, you may see an error message such as "EOF on pvmd sock". This is normal and can be ignored. You can instead kill the pvmd process. It will shut down, killing any local tasks with SIGTERM. If you kill a slave pvmd, it will be deleted from the virtual machine. If you kill the master pvmd, the slaves will all exit too. Always kill the pvmd with a catchable signal, for example SIGTERM. If you kill it with SIGKILL, it won't be able to clean up after itself, and you'll have to do that by hand. ------------------------------------------------------------------------ Q17: The compiler can't find header files when I compile my application. A17: PVM applications written in C should include header file pvm3.h, as follows: #include Programs using the trace functions should additionally include pvmtev.h, and resource manager programs should include pvmsdpro.h. You may need to specify the PVM include directory in the compiler flags as follows: cc ... -I$PVM_ROOT/include ... A header file for Fortran (fpvm3.h) is also supplied. Syntax for including files in Fortran is variable; the header file may need to be pasted into your source. A statement commonly used is: INCLUDE '/usr/local/pvm/include/fpvm3.h' ------------------------------------------------------------------------ Q18: I get missing symbols when I link my application. A18: PVM applications written in C must be linked with at least the base PVM library, libpvm3.a. Fortran applications must be linked with both libfpvm3.a and libpvm3.a. Programs that use group functions must also be linked with libgpvm3.a. On some operating systems, PVM programs must be linked with still other libraries (for the socket or XDR functions). Note that the order of libraries in the link command is important; Unix machines generally process the list from left to right, searching each library once. You may also need to specify the PVM library directory in the link command. A correct order is shown below (your compiler may be called something other than cc or f77). cc/f77 [ compiler flags ] [ source files ] [ loader flags ] -L$PVM_ROOT/lib/$PVM_ARCH -lfpvm3 -lgpvm3 -lpvm3 [ libraries needed by PVM ] [ other libraries ] The aimk program supplied with PVM automatically sets environment variable PVM_ARCH to the PVM architecture name and ARCHLIB to the necessary system libraries. Before running aimk, you must have PVM_ROOT set to the path where PVM is installed. You can use these variables to write a portable, shared makefile (Makefile.aimk). ------------------------------------------------------------------------