OpenCV create Mat from byte array

Question

In my C++ dll I am creating Mat from byte array: ``` BYTE * ptrImageData; //Image data is in this array passed to this function Mat newImg = Mat(nImageHeight, nImageWidth, CV_8UC3, ptrImageData); ``` The image is created with some gray shade not the original one. Is this the proper way of creating Mat from byte array? Please see code ptrImageData is passed to the C++ dll from C# code. C# code to pass the image data ``` System.Drawing.Image srcImage //Has the image MemoryStream ms = new MemoryStream(); Marshal.FreeHGlobal(ptrImageData); srcImage.Save(ms, System.Drawing.Imaging.ImageFormat.Jpeg); byte[] imgArray = ms.ToArray(); ms.Dispose(); int size1 = Marshal.SizeOf(imgArray[0]) * imgArray.Length; IntPtr ptrImageData = Marshal.AllocHGlobal(size1); Marshal.Copy(imgArray, 0, ptrImageData, imgArray.Length); //Calling C++ dll function ProcessImage(ptrImageData, srcImage.Width, srcImage.Height); Marshal.FreeHGlobal(ptrImageData); ```

Accepted Answer

The C++ code appears ok, in that this creates a matrix wrapper for the supplied image data, assuming the buffer is in the conventional RGB8 format. Note that this constructor does not copy the buffer, so the buffer must remain valid for the duration of this `Mat` instance (or be copied).

```
Mat newImg = Mat(nImageHeight, nImageWidth, CV_8UC3, ptrImageData);
```

It appears the problem lies in Your C# code. I am not a C# developer, but I will do my best to help. You are creating a memory stream and using the JPEG codec to write a compressed version of the image into the buffer as if it were a file. But that is not the data format that `cv::Mat` is expecting, so you will basically see garbage (compressed data interpreted as uncompressed).

Given a `System.Image.Drawing.Image` instance, you can create a wrapper `Bitmap` object directly (or maybe use `as`, since it is a simple downcast). Then you can just use the `Bitmap.LockBits()` method tog obtain a pointer to the underlying image data.

```
Bitmap bmp = new Bitmap(sourceImage);

// Lock the bitmap's bits.  
Rectangle rect = new Rectangle(0, 0, bmp.Width, bmp.Height);
System.Drawing.Imaging.BitmapData bmpData =
    bmp.LockBits(rect, System.Drawing.Imaging.ImageLockMode.ReadWrite,
    bmp.PixelFormat);

// Get the address of the first line.
IntPtr ptr = bmpData.Scan0;

// Declare an array to hold the bytes of the bitmap.
int bytes  = Math.Abs(bmpData.Stride) * bmp.Height;
byte[] rgbBuffer = new byte[bytes];

// Copy the RGB values into the array.
System.Runtime.InteropServices.Marshal.Copy(ptr, rgbBuffer, 0, bytes);

// Do your OpenCV processing...
// ...

// Unlock the bits.
bmp.UnlockBits(bmpData);
```

and then you can pass the `rgbBuffer` to OpenCV.

I'm not convinced that the memory management in the original code is entirely correct either, but anyway the above will work provided the scope of the buffer ownership is within the lock and unlock method calls. If the image data is to outlive this code block, you will have to copy the buffer.

Be careful with your pixel formats too - you need to make sure the `Image/Bitmap` instance really contains RGB8 data. OpenCV's `cv::Mat` has various flags so you can work with a variety of in-memory image formats. But note that these are not the same as the on-disk (typically compressed) formats, such as PNG, TIFF, and so forth.

OpenCV create Mat from byte array

Solution

Problem