MatLab Bottleneck
arrays, matlab
Solution
When `subsref` is used, matlab makes a copy of the sub referenced elements. This may be costly for large arrays. Often it will be faster to catenate vectors like
res = [a,b,c];
This is not possible with the current code as written above, but if the code could be modified to make this work, it may save some time.
EDIT For multi-dimensional arrays you need to use `cat`
CompImage = cat(dim,Subimage,Subimage,PCImage);
where `dim` is 3 for this example.
Problem
I am working with big arrays (~6x40million) and my code is showing great bottlenecks. I am experienced programming in MatLab, but don't know much about the inner processes (like memory and such...). My code looks as follows(Just the essentials, of course all variables are initialized, specially the arrays in loops, I just don't want to bomb you all with code ): First I read the file, ``` disp('Point cloud import and subsampling') tic fid=fopen(strcat(Name,'.dat')); C=textscan(fid, '%d%d%f%f%f%d'); %<= Big! fclose(fid); ``` then create arrays out of the contents, ``` y=C{1}(1:Subsampling:end)/Subsampling; x=C{2}(1:Subsampling:end)/Subsampling; %... and so on for the other rows clear C %No one wants 400+ millon doubles just lying around. ``` And clear the cell array (1), and create some images and arrays with the new values ``` for i=1:length(x) PCImage(y(i)+SubSize(1)-maxy+1,x(i)+1-minx)=Reflectanse(i); PixelCoordinates(y(i)+SubSize(1)-maxy+1,x(i)+1-minx,:)=Coordinates(i,:); end toc ``` Everything runs more or less smoothly until here, but then I manipulate some arrays ``` disp('Overlap alignment') tic PCImage=PCImage(:,[1:maxx/2-Overlap,maxx/2:end-Overlap]); %-30 overlap? PixelCoordinates=PixelCoordinates(:,[1:maxx/2-Overlap,maxx/2:end-Overlap],:); Sphere=Sphere(:,[1:maxx/2-Overlap,maxx/2:end-Overlap],:); toc ``` and this is a big bottleneck, but it gets worst at the next step ``` disp('Planar view and point cloud matching') tic CompImage=zeros(max(SubSize(1),PCSize(1)),max(SubSize(2),PCSize(2)),3); CompImage(1:SubSize(1),1:SubSize(2),2)=Subimage; %ExportImage Cyan CompImage(1:SubSize(1),1:SubSize(2),3)=Subimage; CompImage(1:PCSize(1),1:PCSize(2),1)=PCImage; %PointCloudImage Red toc ``` Output Point cloud import and subsampling Elapsed time is 181.157182 seconds. Overlap alignment Elapsed time is 408.750932 seconds. Planar view and point cloud matching Elapsed time is 719.383807 seconds. My questions are: will clearing unused objects like `C` in 1 have any effect? (it doesn't seem like that) Am I overseeing any other important mechanisms or rules of thumb, or is the whole thing just too much and supposed to happen like this?