Data transfer between C++ and Python
c++, data-transfer, python
Solution
First path: I think the more appropriate way for you to go is ctypes. You can create a shared library, and then load the functions of the shared library in Python, and fill all the data containers you want in Python.
In Windows, you can create a DLL, and in Linux you can create a shared .so library.
Now this has the advantage that this will be independent of your Python version.
Second path: I think it's less appropriate but you can get it to work, which is the Python C Extension. With this, you can call Python data containers (`PyObject`s) and fill them inside C.
However, the code you compile here will always need to be linked to Python libraries.
Which one to use?:
- Use ctypes if you have some functions you want to call in C/C++, and then do the rest of the work in Python.
- Use Python C Extension if you have some functions you want to call in Python, and you want to do the rest in C/C++.
With both options, you can transfer huge blocks of memory between C++ and Python without necessarily involving any disk read/write operations.
Hope this helps.
Problem
I would like to share memory between C++ and Python. My problem: - I am working with big data sets (up to 6 GB of RAM) in C++. All calculations are done in c++. - Then, I want to "paste" all my results to a Python program. But I can only write my data on disk, and then read that file from Python, which is not efficient. Is there any way to "map" memory corresponding to C++ variables so that I may access the data from Python? I don't want to copy 6GB of data onto a hard drive.