From bug-octave-request at octave dot org Thu Apr 22 14:38:45 2004
Subject: Segfault when indexing out of bounds
From: "John W. Eaton" <jwe at bevo dot che dot wisc dot edu>
To: Quentin Spencer <qspencer at ieee dot org>
Cc: bug at octave dot org
Date: Thu, 22 Apr 2004 14:37:26 -0500

On 20-Apr-2004, Quentin Spencer <qspencer at ieee dot org> wrote:

| Bug report for Octave 2.1.57 configured for i686-pc-linux-gnu
| 
| Description:
| -----------
| I get a segfault when including an out-of-bounds array element in a 
| range with the ":" operator.
| 
| Repeat-By:
| ---------
| 
| octave:1> a=1;
| octave:2> a(2)
| error: invalid vector index = 2
| error: Array::Array (const Array&, const dim_vector&): dimension mismatch
| error: Array::Array (const Array&, const dim_vector&): dimension mismatch
| error: Array::Array (const Array&, const dim_vector&): dimension mismatch
| error: Array::Array (const Array&, const dim_vector&): dimension mismatch
| octave:2> 1:a(2)
| error: invalid vector index = 2
| error: Array::Array (const Array&, const dim_vector&): dimension mismatch
| error: Array::Array (const Array&, const dim_vector&): dimension mismatch
| error: Array::Array (const Array&, const dim_vector&): dimension mismatch
| error: Array::Array (const Array&, const dim_vector&): dimension mismatch
| panic: Segmentation fault -- stopping myself...
| attempting to save variables to `octave-core'...
| save to `octave-core' complete
| Segmentation fault

Please try the following patch.  With it, I see

  octave:1> a = 1
  a = 1
  octave:2> a(2)
  error: invalid vector index = 2
  octave:2> 1:a(2)
  error: invalid vector index = 2
  error: invalid value in colon expression
  error: evaluating colon expression near line -1 column -1

Thanks,

jwe


liboctave/ChangeLog:

2004-04-22  John W. Eaton  <jwe at bevo dot che dot wisc dot edu>

	* Array.cc (Array<T>::index2, Array<T>::indexN):
	Don't set invalid dimensions on return value.


src/ChangeLog:

2004-04-22  John W. Eaton  <jwe at bevo dot che dot wisc dot edu>

	* pt-colon.cc (tree_colon_expression::rvalue): Also check for
	error_state after evaluating each subexpression.


Index: liboctave/Array.cc
===================================================================
RCS file: /usr/local/cvsroot/octave/liboctave/Array.cc,v
retrieving revision 1.114
diff -u -r1.114 Array.cc
--- a/liboctave/Array.cc	21 Apr 2004 17:30:51 -0000	1.114
+++ b/liboctave/Array.cc	22 Apr 2004 19:35:54 -0000
 at @ -2004,7 +2004,7 @@
 
       if (len == 0 && idx_arg.one_zero_only ())
 	retval = Array<T> (tmp, dim_vector (0, 0));
-      else
+      else if (len >= idx_orig_dims.numel ())
 	retval = Array<T> (tmp, idx_orig_dims);
     }
   else if (nr == 1 || nc == 1)
 at @ -2025,7 +2025,7 @@
 	  else
 	    retval = Array<T> (tmp, dim_vector (len, 1));
 	}
-      else
+      else if (len >= idx_orig_dims.numel ())
 	retval = Array<T> (tmp, idx_orig_dims);
     }
   else
 at @ -2101,8 +2101,13 @@
 
       Array<T> tmp = Array<T>::index (ra_idx, resize_ok);
 
-      if (tmp.length () != 0)
-	retval = Array<T> (tmp, idx_orig_dims);
+      int len = tmp.length ();
+
+      if (len != 0)
+	{
+	  if (len >= idx_orig_dims.numel ())
+	    retval = Array<T> (tmp, idx_orig_dims);
+	}
       else
 	retval = Array<T> (tmp, dim_vector (0, 0));
     }
 at @ -2154,7 +2159,7 @@
 
 	      retval = Array<T> (tmp, new_dims);
 	    }
-	  else
+	  else if (tmp.length () >= idx_orig_dims.numel ())
 	    retval = Array<T> (tmp, idx_orig_dims);
 
 	  (*current_liboctave_error_handler)
Index: src/pt-colon.cc
===================================================================
RCS file: /usr/local/cvsroot/octave/src/pt-colon.cc,v
retrieving revision 1.8
diff -u -r1.8 pt-colon.cc
--- a/src/pt-colon.cc	20 Nov 2002 16:56:49 -0000	1.8
+++ b/src/pt-colon.cc	22 Apr 2004 19:36:06 -0000
 at @ -96,9 +96,9 @@
 
   octave_value tmp = op_base->rvalue ();
 
-  if (tmp.is_undefined ())
+  if (error_state || tmp.is_undefined ())
     {
-      eval_error ("invalid null value in colon expression");
+      eval_error ("invalid value in colon expression");
       return retval;
     }
 
 at @ -112,9 +112,9 @@
 
   tmp = op_limit->rvalue ();
 
-  if (tmp.is_undefined ())
+  if (error_state || tmp.is_undefined ())
     {
-      eval_error ("invalid null value in colon expression");
+      eval_error ("invalid value in colon expression");
       return retval;
     }
 
 at @ -132,9 +132,9 @@
     {
       tmp = op_increment->rvalue ();
 
-      if (tmp.is_undefined ())
+      if (error_state || tmp.is_undefined ())
 	{
-	  eval_error ("invalid null value in colon expression");
+	  eval_error ("invalid value in colon expression");
 	  return retval;
 	}
 



-------------------------------------------------------------
Octave is freely available under the terms of the GNU GPL.

Octave's home on the web:  http://www.octave.org
How to fund new projects:  http://www.octave.org/funding.html
Subscription information:  http://www.octave.org/archive.html
-------------------------------------------------------------