From bug-request at octave dot org Mon May 30 18:04:02 2005
Subject: Re: 2.1.69: load/save -hdf5: matrix are transposed
From: Cyril Humbert <cyril dot humbert at univ-mlv dot fr>
To: David Bateman <David dot Bateman at motorola dot com>
Cc: "John W. Eaton" <jwe at bevo dot che dot wisc dot edu>, bug@octave.org
Date: Mon, 30 May 2005 11:48:36 -0500

David Bateman wrote:
> Ok, I see the issues now. The problem is that HDF5 saves in C row-major 
> ordering while octave uses fortran column-major ordering for easier 
> interfacing to the fortran LAPACK code, etc. So there is several choices 
> of how this can be dealt with.
> 
> 1) Do what is currently done and reverse the rows/columns dimensions. As 
> remarked this transposes the matrix, but means that the raw data doesn't 
> need to be reordered.. BTW, the NDArray stuff seems to be wrong in this 
> case as
> 
>  // Octave uses column-major, while HDF5 uses row-major ordering
>  for (int i = 0; i < rank; i++)
>    hdims[i] = dv (rank-i-1);
> 
> sholud read
> 
>  // Octave uses column-major, while HDF5 uses row-major ordering
>  hdims[0] = dv (1);
>  hdims[1] = dv(0);
>  for (int i = 2; i < rank; i++)
>    hdims[i] = dv (i);
> 
> Since only the rows and columns are reversed. Do we want to fix that? If 
> we do we'll have to treat the issue of hdf5 compatiability between 2.153 
> and now for NDArrays...
> 
> 2) The absolutely correct way to handle the issue would be to transpose 
> the matrix before saving it to an HDF5 file and keep the same 
> dimensioning. In this way the matrix dimensions are respected. The 
> disadvantage is that the data is manipulated prior to saving which takes 
> time and memory.
> 
> 3) Keep the octave hdf5 file formats as is, but do the transposing when 
> importing hdf5 files from external sources. This option is a relatively 
> simple addition, but will mean that hdf5 files saved prior to 2.1.53 in 
> octave will be transposed... This raises the question of should we have 
> a way of exporting hdf5 files in right order, since the octave format 
> will be transposed for programs external to octave.
> 
> I'd suggest that we add the transposition of hdf5 imported matrices, and 
> fix the issue of the NDArray dimension numbering discussed in point 1). 
> When fixing the numbering issue, I don't think too many people will be 
> affected by this change as it only affects multi-dimensional arrays 
> saved as hdf5 files, which can't be too many people. So I suggest just 
> making the change and announcing it, we don't have a stable version of 
> octave so I think this is justified.. This seems to me to be the best 
> compromise. Comments?
> 
> Regards
> David


Hello,
In fact, I expected to obtain the same matrix (dimensions and values)
when reading a hdf5 file (generated with h5fromtxt) from R and Octave.
Actually, reading from Octave gives a transposed matrix but this is 
perhaps correct. Given the Fortran indexing used in Octave, it seems 
at least coherent with the following examples of the hdf5 tutorial:

	http://hdf.ncsa.uiuc.edu/HDF5/Tutor/crtdat.html
and
	http://hdf.ncsa.uiuc.edu/HDF5/Tutor/rdwt.html

[In these examples, a matrix is created, saved and loaded using
both the C and the Fortran90 versions of the hdf5 library.]

Currently, I can't check it more carefully because my system lacks
a fortran90 compiler; thus i can't recreate all the hdf5 files given
in the above links. I'll try to install gfortran and to compile
the hdf5 library with fortran support to check further.

Regards,
-- 
Cyril



-------------------------------------------------------------
Octave is freely available under the terms of the GNU GPL.

Octave's home on the web:  http://www.octave.org
How to fund new projects:  http://www.octave.org/funding.html
Subscription information:  http://www.octave.org/archive.html
-------------------------------------------------------------