From octave-maintainers-request at bevo dot che dot wisc dot edu Mon Nov 10 09:52:33 2003 Subject: Re: Shared memory interface to Octave From: "" To: "" Date: Mon, 10 Nov 2003 08:55:24 -0700 The only reason I chose shared memory instead of named pipes(FIFOs) is because of the speed issue. I was researching on the web for information on how long it takes to transfer data. As I had read through the mailing lists, it seems that some of the GUIs that have been started in the past used sockets to transfer the data between the GUI and the core of Octave, but that the speed was a limiting factor on its usefulness. I was trying to find the fastest possible method. The following url gave a rundown on the times for different types of inter-process communications: http://opensource.or.id/~dewa/References/Unix%20IPC/node10.html It indicates that transfering 256KB via named pipes would take ~32 ms while only ~256 microseconds with the shared memory. It should be noted that the document is somewhat old and was tested on Linux kernel 1.2.9 on a 90 MHz machine. It would be expected to be much, much faster on modern day 1GHz and above machines. Named pipes may be more optimized in more recent versions on Linux (and other *nix-like OSs). I agree with you that the named pipes would be much easier. Is would be very easy to redirect stdin and stdout from and to the pipes. I could then open two other pipes for used as a binary transfer channel. As an aside, I have yet to figure out how to redirect stdout to a block of shared memory other than redirecting it to a file, reading it back in, and then writing it to the shared memory block. This process may take longer than just using named pipes. Named Pipes Method 1) Redirect stdout and stdin to read from and write directly to the pipes between the GUI and the octave core. This should provide a pretty transparent interface to octave and could be implemented in any language. I will need to prototype this up, but from what I can tell, octave shouldn't know the difference between receiving the input from its regular console or the redirected stdin. Same for stdout. 2) Create two other named pipes for communicating control and variable information between the GUI and the octave core. For example, the GUI would listen for a ctrl-c and send it via this second pipe in order to interrupt the execution of the current command. Also, after the execution of each command, the octave core would send a list of all the variables and their types and sizes back to the GUI via one of these pipes. Thanks for the suggestion of named pipes. I am going to do my own performance tests, but the advantages look to outweigh the speed hit, so I will most likely end up using them. John Swensen Quoting Russell Standish : > I don't see why you need to use shared memory? Surely opening a pipe > connection between the Octave's standard input and your control > stream, and another one to its standard output suffices to communicate > between you GUI process and the octave process. > > The pipe construct can be easily converted to a distributed memory > model by means of ssh or similar ilk. Pipes are also far easier to > program than shared memory segments. > > Cheers > > jswensen at intergate dot com wrote: > > > > > > Since I am new to the maintainer list, hello to all before I get into my > questions. > > > > I have used Octave off and on through college (intermixed with Matlab). I > have > > always thought it would be good to have a GUI interface and have read a > some of > > the mailinglist archives about peoples' suggestions on how to do it. > After > > reading through all of these, and drawing on a little past experience, I > think > > that using the shared memory functions shmget, shmat, shmdt, and shmctl > would be > > an ideal way of implementing the communications between the GUI and the > octave core. > > > > Because of this, I am asking for a quick rundown on how console I/O is > > implemented in Octave. I could wade through the sources, but I feel it > would be > > beneficial if I could get a 1 page summary from someone who has done the > > development. This will make my wading that much more productive. > > > > Back to the topic of what I am trying to do, I used the octave-parallel > socket > > interface from octave-forge as a model and implemented a pretty > rudimentary > > passing of commands and results via shared memory. Many commands work, > but > > other crap out on me. My sources are broken right now, so once I get them > > working again, I will probably post some of the erro output for help on > > determining where the problem is. > > Another benefit of using the shared memory is that on clusters that use a > shared > > memory model, as far as I can tell, this method should work as a means of > using > > Octave in a distributed manner an be much faster than using sockets. > > > > I would like to know other poeples suggestions, comments, and criticisms on > my > > plan. I have used shared memory some in a few application I have written > > before, but am by no means and expert. > > > > John Swensen > > jswensen at intergate dot com > > > > ------------------------------------------------- > > This mail sent through IMP: http://horde.org/imp/ > > > > > > ---------------------------------------------------------------------------- > A/Prof Russell Standish Director > High Performance Computing Support Unit, Phone 9385 6967, 8308 3119 (mobile) > UNSW SYDNEY 2052 Fax 9385 6965, 0425 253119 (") > Australia R dot Standish at unsw dot edu dot au > Room 2075, Red Centre http://parallel.hpc.unsw.edu.au/rks > International prefix +612, Interstate prefix 02 > ---------------------------------------------------------------------------- > ------------------------------------------------- This mail sent through IMP: http://horde.org/imp/