MATLAB parfor and C++ class mex wrappers (copy constructor required?)

copy-constructor, matlab, mex, oop, parallel-processing

Solution

When you pass an object instance into the body of a `PARFOR` loop, the behaviour is the same as if you'd saved it to a file, and then loaded it again. The simplest solution is probably to mark your `cpp_handle_` as `Transient`. Then, you need to implement `SAVEOBJ` and `LOADOBJ` to safely transport your data. See this page for more about customising the save/load behaviour of your class.

Problem

I'm trying to wrap a C++ class in a matlab mex wrapper using the approach outlined here. Basically, I have an initialization mex file which returns a C++ object handle: ``` handle = myclass_init() ``` I can then pass this to another mex file (e.g. `myclass_amethod`) which use the handle to call class methods, then ultimately to `myclass_delete` to free the C++ object: ``` retval = myclass_amethod(handle, parameter) myclass_delete(handle) ``` I've wrapped this up in a MATLAB class for ease of use: ``` classdef myclass < handle properties(SetAccess=protected) cpp_handle_ end methods % class constructor function obj = myclass() obj.cpp_handle_ = myclass_init(); end % class destructor function delete(obj) myclass_delete(obj.cpp_handle_); end % class method function amethod(parameter) myclass_amethod(obj.cpp_handle_, parameter); end end end ``` Problem: this doesn't work in parallel code This works fine in non-parallel code. However, as soon as I call it from within a `parfor`: ``` cls = myclass(); parfor i = 1:10 cls.amethod(i) end ``` I get a segfault, as a copy of the class is made in the `parfor` loop (in each worker) but as each worker is a separate process the C++ object instance is not copied, leading to an invalid pointer. I tried initially to detect when each class method was running in a parfor loop, and in those cases reallocate the C++ object too. However, as there is no way to check whether an object has been allocated for the current worker yet or not, this results in multiple reallocations and then just one delete (when the worker exits) resulting in a memory leak (see appendix at bottom of question for details). Attempted solution: copy constructors and using `matlab.mixin.Copyable` In C++, the way to handle this would be copy constructors (so that the C++ object is only reallocated once when the wrapper MATLAB class is copied). A quick search brings up the matlab.mixin.Copyable class type which seems to provide the required functionality (i.e. deep copies of MATLAB handle classes). Therefore, I've tried the following: ``` classdef myclass < matlab.mixin.Copyable properties(SetAccess=protected) cpp_handle_ end methods function obj = myclass(val) if nargin < 1 % regular constructor obj.cpp_handle_ = rand(1); disp(['Initialized myclass with handle: ' num2str(obj.cpp_handle_)]); else % copy constructor obj.cpp_handle_ = rand(1); disp(['Copy initialized myclass with handle: ' num2str(obj.cpp_handle_)]); end end % destructor function delete(obj) disp(['Deleted myclass with handle: ' num2str(obj.cpp_handle_)]); end % class method function amethod(obj) disp(['Class method called with handle: ' num2str(obj.cpp_handle_)]); end end end ``` testing this class as above, i.e.: ``` cls = myclass(); parfor i = 1:10 cls.amethod(i) end ``` Results in the output: ``` Initialized myclass with handle: 0.65548 Class method called with handle: 0.65548 Class method called with handle: 0.65548 Class method called with handle: 0.65548 Class method called with handle: 0.65548 Class method called with handle: 0.65548 Class method called with handle: 0.65548 Class method called with handle: 0.65548 Class method called with handle: 0.65548 Class method called with handle: 0.65548 Class method called with handle: 0.65548 Deleted myclass with handle: 0.65548 Deleted myclass with handle: 0.65548 Deleted myclass with handle: 0.65548 ``` In other words, it seems that the copy constructor is not called when spawning workers for the parfor. Does anyone have any pointers as to what I am doing wrong, or whether there is some way to achieve the desired behaviour of reinitializing the C++ object handle when the MATLAB wrapper class is copied? Alternative approach: detecting when running on worker Just for reference, here is the alternative approach I'm using reallocating when in a worker: ``` classdef myclass < handle properties(SetAccess=protected) cpp_handle_ end methods function obj = myclass(val) obj.cpp_handle_ = rand(1); disp(['Initialized myclass with handle: ' num2str(obj.cpp_handle_)]); end % destructor function delete(obj) disp(['Deleted myclass with handle: ' num2str(obj.cpp_handle_)]); end % class method function amethod(obj) obj.check_handle() disp(['Class method called with handle: ' num2str(obj.cpp_handle_)]); end % reinitialize cpp handle if in a worker: function check_handle(obj) try t = getCurrentTask(); % if 'getCurrentTask()' returns a task object, it means we % are running in a worker, so reinitialize the class if ~isempty(t) obj.cpp_handle_ = rand(1); disp(['cpp_handle_ reinitialized to ' num2str(obj.cpp_handle_)]); end catch e % in case of getCurrentTask() being undefined, this % probably simply means the PCT is not installed, so % continue without throwing an error if ~strcmp(e.identifier, 'MATLAB:UndefinedFunction') rethrow(e); end end end end end ``` and the output: ``` Initialized myclass with handle: 0.034446 cpp_handle_ reinitialized to 0.55625 Class method called with handle: 0.55625 cpp_handle_ reinitialized to 0.0048098 Class method called with handle: 0.0048098 cpp_handle_ reinitialized to 0.58711 Class method called with handle: 0.58711 cpp_handle_ reinitialized to 0.81725 Class method called with handle: 0.81725 cpp_handle_ reinitialized to 0.43991 cpp_handle_ reinitialized to 0.79006 cpp_handle_ reinitialized to 0.0015995 Class method called with handle: 0.0015995 cpp_handle_ reinitialized to 0.0042699 cpp_handle_ reinitialized to 0.51094 Class method called with handle: 0.51094 Class method called with handle: 0.0042699 Class method called with handle: 0.43991 cpp_handle_ reinitialized to 0.45428 Deleted myclass with handle: 0.0042699 Class method called with handle: 0.79006 Deleted myclass with handle: 0.43991 Deleted myclass with handle: 0.79006 Class method called with handle: 0.45428 ``` As can be seen above, a reallocation does indeed now occur when running in a worker. However, the destructor is only called once for every worker regardless of how many time the C++ class was reallocated, resulting in a memory leak. Solution: using `loadobj` The following works: ``` classdef myclass < handle properties(SetAccess=protected, Transient=true) cpp_handle_ end methods(Static=true) function obj = loadobj(a) a.cpp_handle_ = rand(1); disp(['Load initialized encoder with handle: ' num2str(a.cpp_handle_)]); obj = a; end end methods function obj = myclass(val) obj.cpp_handle_ = rand(1); disp(['Initialized myclass with handle: ' num2str(obj.cpp_handle_)]); end % destructor function delete(obj) disp(['Deleted myclass with handle: ' num2str(obj.cpp_handle_)]); end % class method function amethod(obj) disp(['Class method called with handle: ' num2str(obj.cpp_handle_)]); end end end ```

Original source