From owner-octave-sources at bevo dot che dot wisc dot edu Mon Mar 10 18:26:02 1997 Subject: Re: benchmark 1.10 From: "John A. Turner" To: F dot Potorti at cnuce dot cnr dot it Cc: help-octave at bevo dot che dot wisc dot edu, octave-sources@bevo.che.wisc.edu Date: Mon, 10 Mar 1997 17:21:35 -0700 Francesco Potorti` writes: [snip] > The new benchmark is for octave 2.0.5 and can be found, as usual, in > , while the results are in > (mirrors welcome). No new > results have been added yet. > > Everyone is encouraged to run the benchmark with octave 2.0.5 and send > the results to me. > > I have not added yet the results for Alpha because I cannot find a > moment when the machine is idle, but it seems that, with respect to > version 1.1.1, the for loop is about 30% faster (expected), the > differential equation test is about 10% faster (expected), and the > Schur decomposition is about 30% slower (unexpected!). [snip] While I applaud this effort, I took a look at the results, and there is far to little information from which to draw real conclusions. Each entry should include: o a more complete hardware specification. For example, one of the entries is "Ultra 1". Is that a 1/140, a 1/170, or a 1/200? They have three different clock speeds. Maybe nitpicking, but there's a listing for an Ultra 167. There's no such thing. It must actually be a 1/170. (I'm omitting the E because it shouldn't matter.) It's difficult, though, because should it be the model or the chip? For example, there's an entry for a "DEC Alpha 400". It must be one of the models with a 400MHz A21164 chip, and that's probably more important than the actual model. Still, I'm thinking the precise model should be listed, maybe with additional columns for the chip and clock speed. Something like the SPECmark table at ftp://ftp.cdf.toronto.edu/pub/spectable does. Here's an excerpt: ------------------------------------------------------------------------------- System CPU ClkMHz Cache SPECint SPECfp Info Source Name (NUMx)Type ext/in Ext+I/D base95 base95 Date Obtained ================= ========== ======= ========== ======= ======= ===== ========= DEC 8[24]00/5/300 A21164 75/300 4M+96+8/8 7.43 11.7 Feb96 Digital DEC 8[24]00/5/350 A21164 88/350 4M+96+8/8 8.82 13.2 Feb96 Digital DEC 8[24]00/5/440 A21164 88/440 4M+96+8/8 11.2 16.0 Oct96 Digital Intel XXpress Pentium 66/166 1M+8/8 4.76 3.37 Jan96 www.intel Intel Alder PentiumPro 166 512+8/8 7.11 5.47 Jan96 www.intel Intel Alder PentiumPro 200 256+8/8 8.09 5.99 Jan96 www.intel SGI O2-R5kSC R5000 180 512+32/32 4.76 5.37 Oct96 www.specb SGI Indigo2-R10k R10000 195 1M+32/32 8.50 10.2 Jul96 www.specb Sun SS10/40 SuprSP 40 20/16 1.06 1.13 Mar96 c.bmarks Sun SS[45]/110 MicroSP2 110 16/8 1.37 1.88 Mar96 c.bmarks Sun Ultra1/140 UltraSP 71/143 512+16/16 4.52 7.73 Mar96 c.bmarks Sun Ultra1/170 UltraSP 83/167 512+16/16 5.26 8.45 Mar96 c.bmarks o a precise specification of the operating system, including the version (e.g. SunOS 4.1.3, Solaris 2.5.1, etc.). o a precise specification of the compiler used to compile Octave, including version *and compile flags*. Also, whether an F77 compiler was used for the Fortran, and if so, version and compile flags for it. So again, I applaud the effort, but it's difficult to do any real comparison without the entire picture. -- John Turner http://www.lanl.gov/home/turner