From help-octave-request at bevo dot che dot wisc dot edu  Tue Jan 19 01:30:49 1999
Subject: Re: Problem with the floor function
From: Michael Hanke <hanke at nada dot kth dot se>
To: Daniel Tourde <ted at ffa dot se>
cc: help-octave at bevo dot che dot wisc dot edu
Date: Tue, 19 Jan 1999 08:30:45 +0100 (MET)


Hej,

You ran in the interesting world of rounding errors. All computations
in octave (and matlab) are done in double precision IEEE numbers.
They use the base 2 while we use conveniently decimal numbers for
input/output representation. You are computing something like
15/0.01. But 0.01 does not have a finite representation in the binary
system. So you will not obtain 1500 but a rounded result. Your octave
session indicates that it is slightly lower than 1500.

This problem is very common in defining a grid in floating point
numbers:

a=...;
b=...;
h=(b-a)/m;
x=a:h:b;

gives often not the desired result. I am using instead

x=a:h:b+h/2;

in order to avoid a surprise when considering the length of x.

Hope this helps.

Michael




On Tue, 19 Jan 1999, Daniel Tourde wrote:

> Hello,
> 
> I have discovered something strange with the floor function. I suspect a
> bug but where ? John W. Eaton would bet that floor is correct, but that
> the display routines in the C/C++ I/O library are doing
> something slightly different when they display values that are very
> near but just less than a representable integer.
> 
> octave:1> floor(46.000)
> ans = 46
> octave:2> k = 0.03
> k = 0.030000
> octave:3> kmin=0
> kmin = 0
> octave:4> kstep = 0.01
> kstep = 0.010000
> octave:5> modes=15
> modes = 15
> octave:6> (1+(k-kmin)*modes/kstep)
> ans = 46.000
> octave:7> floor(1+(k-kmin)*modes/kstep)
> ans = 45                                     <-  It should be 46, no ?
> octave:8> (k-kmin)*modes
> ans = 0.45000
> octave:9> (k-kmin)*modes/kstep
> ans = 45.000
> octave:10> floor((k-kmin)*modes/kstep)
> ans = 44                                     <-  It should be 45, no ?
> 
> Strange isn't it ?
> Does anyone has an idea of where it comes from and how it could be
> solved ?
> 
> Thanks in advance
> 
>                                 Daniel
> -- 
> ***********************************************************************
> Daniel TOURDE                                   E-mail : ted at ffa dot se
> The Aeronautical Research Institute of Sweden   Tel : +46 8 55 54 93 44
> P.O. Box 11021 S-161 11 BROMMA, Sweden          Fax : +46 8    25 34 81
> ***********************************************************************
> 
> 

+---------------------------------------------------------------+
|  Michael Hanke                Royal Institute of Technology   |
|                               NADA                            |
|                               S-10044 Stockholm               |
|                               Sweden                          |
+---------------------------------------------------------------+
|  Visiting address:            Lindstedtsvaegen 3              |
|  Phone:                       + (46) (8) 790 6278             |
|  Fax:                         + (46) (8) 790 0930             |
|  Email:                       hanke at nada dot kth dot se|                              |
|                               na dot mhanke at na-net dot ornl dot gov|              |
+---------------------------------------------------------------+