From help-octave-request at bevo dot che dot wisc dot edu Tue Jan 19 05:36:01 1999 Subject: RE: Problem with the floor function From: (Ted Harding) To: Daniel Tourde Cc: help-octave at bevo dot che dot wisc dot edu Date: Tue, 19 Jan 1999 11:31:15 -0000 (GMT) On 19-Jan-99 Daniel Tourde wrote: > Hello, > > I have discovered something strange with the floor function. I suspect > a bug but where ? John W. Eaton would bet that floor is correct, but > that the display routines in the C/C++ I/O library are doing > something slightly different when they display values that are very > near but just less than a representable integer. > > octave:1> floor(46.000) > ans = 46 > octave:2> k = 0.03 > k = 0.030000 > octave:3> kmin=0 > kmin = 0 > octave:4> kstep = 0.01 > kstep = 0.010000 > octave:5> modes=15 > modes = 15 > octave:6> (1+(k-kmin)*modes/kstep) > ans = 46.000 > octave:7> floor(1+(k-kmin)*modes/kstep) > ans = 45 <- It should be 46, no ? > octave:8> (k-kmin)*modes > ans = 0.45000 > octave:9> (k-kmin)*modes/kstep > ans = 45.000 > octave:10> floor((k-kmin)*modes/kstep) > ans = 44 <- It should be 45, no ? > > Strange isn't it ? > Does anyone has an idea of where it comes from and how it could be > solved ? The reason is that octave:10> 46 - (1+(k-kmin)*modes/kstep) ans = 7.10542735760100e-15 i.e. as a consequence of truncation error in the binary representation of floating-point numbers, the result of the computation [1] of (1+(k-kmin)*modes/kstep) is slightly less than 46 in the machine representation. Therefore "floor" is behaving as it should. The only way to get round this is to add a small positive number to the result of such a computation, and hope that this doesn't in turn generate an error in the opposite direction. How small the number should be can in general only be determined by careful error analysis of the computation. [1] Note the wording. It is not that a number which is an exact integer 46 (as implied by the mathematical expression) is inaccurately represented in the machine. In reality, the numbers k=0.03 and kstep=0.01 are both inaccurately represented in the first place (unavoidable), and -- even assuming that their internal representations were their exact values -- further inaccuracies are introduced when the machine computations evoked by "+", "*" and "/" are carried out. Hope this helps, Ted. -------------------------------------------------------------------- E-Mail: (Ted Harding) Date: 19-Jan-99 Time: 11:31:15 ------------------------------ XFMail ------------------------------