From help-request at octave dot org Tue Dec 27 10:41:02 2005 Subject: SIFT image descriptor patch From: Etienne Grossmann To: Octave Forge , Octave Help Cc: Etienne Grossmann , David Lowe Date: Tue, 27 Dec 2005 11:32:33 -0500 --VS++wcV0S1rZb1Fb Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit Hi All, this patch could be of interest to octave users who do image processing. You can apply it to the matlab code in siftDemoV4 [1] to allow octave to extract SIFT image features [2] with David Lowe's package. The SIFT image detector identifies interest points, somewhat like a corner detector does, and gives them a local descriptor. The descriptor can then be used for matching purposes (another way to perform matching could be correlation of a small window around the feature). Wrt to corners + local window correlation, SIFT features are more robust to changes in illumination and viewpoint [3] and they are increasingly used in computer vision. The siftDemo has functions to find sift features and to perform matching. I've tested the patch under octave+linux, matlab+linux (it works) and octave+win (it fails, probs w/ imagemagick + double). Hth, Etienne [1] http://www.cs.ubc.ca/spider/lowe/keypoints/siftDemoV4.zip Synopsis: unzip siftDemoV4.zip; cd siftDemoV4; patch -p1 < mySift.patch In octave, make sure sift.m is in your path, then >> [image, descriptors, locs] = sift("/home/etienne/prog/various/siftDemoV4/book.pgm"); >> showkeys(image, locs) See also 'help match' [2] http://www.cs.ubc.ca/~lowe/keypoints/ David G. Lowe, Öbject recognition from local scale-invariant features," International Conference on Computer Vision, Corfu, Greece (September 1999), pp. 1150-1157. [3] A performance evaluation of local descriptors Krystian Mikolajczyk, Cordelia Schmid International Conference on Computer Vision & Pattern Recognition - June 2003 http://lear.inrialpes.fr/pubs/2003/MS03/mikolajczyk_cvpr2003.pdf -- Etienne Grossmann ------ http://www.cs.uky.edu/~etienne --VS++wcV0S1rZb1Fb Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="mySift.patch" diff -Nu siftDemoV4/Makefile siftDemoV4-mine/Makefile --- siftDemoV4/Makefile 2005-07-08 04:54:12.000000000 +0000 +++ siftDemoV4-mine/Makefile 2005-12-27 00:05:55.000000000 +0000 at @ -25,5 +25,9 @@ match: ${OBJ} ${CC} -o $ at ${CFLAGS} ${OBJ} ${LIBS} +clean: + rm -f *.o *~ + # Implicit rule used by Gnu Make: $(CC) -c $(CPPFLAGS) $(CFLAGS) ${OBJ}: ${DEFS} + diff -Nu siftDemoV4/TransformLineCoords.m siftDemoV4-mine/TransformLineCoords.m --- siftDemoV4/TransformLineCoords.m 1970-01-01 00:00:00.000000000 +0000 +++ siftDemoV4-mine/TransformLineCoords.m 2005-12-26 16:38:39.000000000 +0000 at @ -0,0 +1,35 @@ +% coords = TransformLine(imsize, keypoint, x1, y1, x2, y2) +% +% Draw the given line in the image, but first translate, rotate, and +% scale according to the keypoint parameters. +% +% Parameters: +% Arrays: +% imsize = [rows columns] of image +% keypoint = [subpixel_row subpixel_column scale orientation] +% +% Scalars: +% x1, y1; begining of vector +% x2, y2; ending of vector +% +% This function is called only by showkeys.m + +function coords = TransformLineCoords(imsize, keypoint, x1, y1, x2, y2) + +% The scaling of the unit length arrow is set to approximately the radius +% of the region used to compute the keypoint descriptor. +len = 6 * keypoint(3); + +% Rotate the keypoints by 'ori' = keypoint(4) +s = sin(keypoint(4)); +c = cos(keypoint(4)); + +% Apply transform +r1 = keypoint(1) - len * (c * y1 + s * x1); +c1 = keypoint(2) + len * (- s * y1 + c * x1); +r2 = keypoint(1) - len * (c * y2 + s * x2); +c2 = keypoint(2) + len * (- s * y2 + c * x2); + +%% line([c1 c2], [r1 r2], 'Color', 'c'); +coords = [r1 c1 r2 c2]; +endfunction diff -Nu siftDemoV4/imglines.m siftDemoV4-mine/imglines.m --- siftDemoV4/imglines.m 1970-01-01 00:00:00.000000000 +0000 +++ siftDemoV4-mine/imglines.m 2005-12-26 16:41:00.000000000 +0000 at @ -0,0 +1,34 @@ +## im2 = imglines (im, [r1,c1,r2,c2], col) - Plot line segments on a bitmap +## +## im : R x C : Greylevel image +## r1c1r2c2 : N x 4 : Enpoint coordinates of segments +## col : N x 1 : Color (default: lightest color of image, or +## lightest+1, if image is uniform) +## +## im2 : R x C : Output image +## +## TODO: Clip segments that stick out of image. +## Do color images too. +## +## Author: Etienne Grossmann , 2005 +function im = imglines (im, pts, col) + +sz = size (im); + +if nargin < 3, col = max (im(:)); col += (col==min(im(:))); end + +if all (size(col) == 1), col = col*ones(rows(pts),1); end + +for i = 1:rows (pts) # Draw line segments + waitbar (i/rows(pts)); + p = pts(i,:); + + len = round (1+max(abs(p(3)-p(1)),abs(p(4)-p(2)))); + + ii = sub2ind (sz, \ + round (linspace(p(1),p(3),len)),\ + round (linspace(p(2),p(4),len))); + + im(ii) = col(i); +endfor + diff -Nu siftDemoV4/match.m siftDemoV4-mine/match.m --- siftDemoV4/match.m 2005-07-08 04:54:12.000000000 +0000 +++ siftDemoV4-mine/match.m 2005-12-27 10:55:03.000000000 +0000 at @ -10,6 +10,22 @@ function num = match(image1, image2) +running_octave = exist ('OCTAVE_VERSION'); + % Find image in loadpath if needed. This is + % mostly so that example code works as-is. +if running_octave + if isempty(stat (image1)) + tmp = find_first_of_in_loadpath (image1); + if ~ tmp, error ('can''t find %s',image1); end + image1 = tmp; + end + if isempty(stat (image2)) + tmp = find_first_of_in_loadpath (image2); + if ~ tmp, error ('can''t find %s',image2); end + image2 = tmp; + end +end + % Find SIFT keypoints for each image [im1, des1, loc1] = sift(image1); [im2, des2, loc2] = sift(image2); at @ -25,6 +41,7 @@ % For each descriptor in the first image, select its match to second image. des2t = des2'; % Precompute matrix transpose +match = zeros(size(des1,1),1); for i = 1 : size(des1,1) dotprods = des1(i,:) * des2t; % Computes vector of dot products [vals,indx] = sort(acos(dotprods)); % Take inverse cosine and sort results at @ -41,21 +58,31 @@ im3 = appendimages(im1,im2); % Show a figure with lines joining the accepted matches. -figure('Position', [100 100 size(im3,2) size(im3,1)]); -colormap('gray'); -imagesc(im3); -hold on; -cols1 = size(im1,2); -for i = 1: size(des1,1) - if (match(i) > 0) - line([loc1(i,2) loc2(match(i),2)+cols1], ... - [loc1(i,1) loc2(match(i),1)], 'Color', 'c'); + +if ~running_octave % Matlab code + figure('Position', [100 100 size(im3,2) size(im3,1)]); + colormap('gray'); + imagesc(im3); + hold on; + cols1 = size(im1,2); + for i = 1: size(des1,1) + if (match(i) > 0) + line([loc1(i,2) loc2(match(i),2)+cols1], ... + [loc1(i,1) loc2(match(i),1)], 'Color', 'c'); + end end + hold off; + num = sum(match > 0); + fprintf('Found %d matches.\n', num); + +else % Octave code + + ii = find (match>0); + im3 = imglines (im3, [loc1(ii,1), loc1(ii,2), loc2(match(ii),1), + loc2(match(ii),2)+columns(im1)]); + imagesc(im3); + end -hold off; -num = sum(match > 0); -fprintf('Found %d matches.\n', num); - diff -Nu siftDemoV4/showkeys.m siftDemoV4-mine/showkeys.m --- siftDemoV4/showkeys.m 2005-07-08 04:54:12.000000000 +0000 +++ siftDemoV4-mine/showkeys.m 2005-12-27 09:33:51.000000000 +0000 at @ -11,19 +11,37 @@ disp('Drawing SIFT keypoints ...'); % Draw image with keypoints -figure('Position', [50 50 size(image,2) size(image,1)]); -colormap('gray'); -imagesc(image); -hold on; -imsize = size(image); -for i = 1: size(locs,1) +if ~ exist('OCTAVE_VERSION') % If under Matlab + + figure('Position', [50 50 size(image,2) size(image,1)]); + colormap('gray'); + imagesc(image); + hold on; + imsize = size(image); + for i = 1: size(locs,1) % Draw an arrow, each line transformed according to keypoint parameters. TransformLine(imsize, locs(i,:), 0.0, 0.0, 1.0, 0.0); TransformLine(imsize, locs(i,:), 0.85, 0.1, 1.0, 0.0); TransformLine(imsize, locs(i,:), 0.85, -0.1, 1.0, 0.0); -end -hold off; + end + hold off; + +else % If running Octave + imsize = size(image); + lines = zeros (3*size(locs,1),4); + for i = 1: size(locs,1) + % Draw an arrow, each line transformed according to keypoint parameters. + waitbar (i/size(locs,1)); + lines(3*i-2,:) = TransformLineCoords(imsize, locs(i,:), 0.0, 0.0, 1.0, 0.0); + lines(3*i-1,:) = TransformLineCoords(imsize, locs(i,:), 0.85, 0.1, 1.0, 0.0); + lines(3*i-0,:) = TransformLineCoords(imsize, locs(i,:), 0.85, -0.1, 1.0, 0.0); + end + lines = min (lines,ones(rows(lines),1)*imsize([1 2 1 2])); + lines = max (lines,1); + image = imglines (image,lines); + imagesc(image); +end % ------ Subroutine: TransformLine ------- % Draw the given line in the image, but first translate, rotate, and diff -Nu siftDemoV4/sift.m siftDemoV4-mine/sift.m --- siftDemoV4/sift.m 2005-07-08 04:54:12.000000000 +0000 +++ siftDemoV4-mine/sift.m 2005-12-27 09:37:50.000000000 +0000 at @ -38,14 +38,30 @@ fwrite(f, image', 'uint8'); fclose(f); -% Call keypoints executable -if isunix + +if ~ exist ('OCTAVE_VERSION') % Running Matlab + % Call keypoints executable + if isunix command = '!./sift '; -else + else command = '!siftWin32 '; + end + command = [command ' tmp.key']; + eval(command); +else + tmp = octave_config_info('canonical_host_type'); + if index (tmp,'linux'), islinux = 1; + elseif index (tmp,'win'), islinux = 0; + else error ('sift: Can''t run on system %s',tmp); end + if islinux + command = 'sift'; + else + command = 'siftWin32'; + end + command = find_first_of_in_loadpath (command); + command = [command, ' tmp.key']; + system(command); end -command = [command ' tmp.key']; -eval(command); % Open tmp.key and check its header g = fopen('tmp.key', 'r'); --VS++wcV0S1rZb1Fb-- ------------------------------------------------------------- Octave is freely available under the terms of the GNU GPL. Octave's home on the web: http://www.octave.org How to fund new projects: http://www.octave.org/funding.html Subscription information: http://www.octave.org/archive.html -------------------------------------------------------------