From octave-maintainers-request at bevo dot che dot wisc dot edu Thu Nov 21 12:23:12 2002 Subject: Re: Patching Octave-MPI From: Andy Jacobson To: octave-maintainers at bevo dot che dot wisc dot edu Date: 21 Nov 2002 13:23:07 -0500 >>>>> "JWE" == John W Eaton writes: JWE> There is also a statement in the MPI docs that I saw saying JWE> something about being sure to initialize MPI before doing JWE> much of anything (I think I/O and opening files were JWE> mentioned). Does anyone know the rationale for that? What JWE> difference would that make? If it really does make a JWE> difference, then there is another reason to put the call to JWE> MPI_Init in the core Octave. I believe that all I/O is implementation-specific. It is omitted from the MPI-1 standard (http://www.mpi-forum.org/docs/mpi-11-html/node6.html#Node6). I'm only familiar with the LAM implementation of MPI, and in that case one I/O issue involves determining whether or not stdout and stderr are (a) linked to a terminal, (b) need to be forwarded to a remote terminal, or (c) dumped to /dev/null. It's my understanding that these decisions are implementation-dependent, so I expect that MPICH handles them a different way. JWE> This is what I was thinking also, that we would JWE> * Compile support for MPI into Octave based on configure JWE> flags (so you could omit it completely and still build a JWE> working Octave. JWE> * Even for Octave built with MPI support, only enable MPI JWE> functionality at run time if Octave is given the appropriate JWE> command-line option. This seems like a perfectly reasonable solution to me. However, I would like to point out a couple of human interface issues. MPI apps are usually started by some external code. In LAM's case you first have to start the LAM daemon on all the computers (via the "lamboot" program). Then you call "mpirun", which takes responsibility for executing the member programs which make up the MPI application (e.g. loading N instances of octave on M computers). After the application is finished, you generally have to shut down the LAM daemons manually (via "wipe" or "lamhalt"). Users will have to know the details of starting an MPI application according to their MPI implementation (LAM, MPICH, etc) in order to get octave to run in parallel. In addition, users need to know the compilation and linker flags necessary to get an MPI-aware octave to compile successfully. Once more, this is implementation-specific. I don't know whether autoconf can be made smart enough to figure out how to do this. LAM very unfortunately hides these flags from the user by wrapping the compiler in a script. While it is possible to determine the proper flags from this wrapper script, this scheme feels like a needless obfuscation to me. Finally, for developers, debugging parallel applications can be quite challenging. Enough said, I hope. Empirically, I have found that these are fairly high barriers to the use of MPI codes by non alpha geeks. I think MPI-enabled octave, especially if linked to ScaLAPACK or some such, would be very useful to some people, and raise a lot of eyebrows--but it is unlikely to ever be easy to use. I suggested one particular balance between access to low-level functionality and high-level convenience with the MPI patches to octave-2.1.31. I thought the best part was that you could send arbitrary octave_value objects between nodes (via a minor hack to load/save). Alex Verstak struck a different balance by providing a raw interface to the MPI send/receive functions. This is very appealing just as a means to learn how parallel applications work. I think it would be great if both levels were available in the final product. Regards, Andy -- Andy Jacobson arj at gfdl dot gov Program in Atmospheric and Oceanic Sciences Sayre Hall, Forrestal Campus Princeton University PO Box CN710 Princeton, NJ 08544-0710 USA Tel: 609/258-5260 Fax: 609/258-2850