Convert ctype byte array to bytes

Question

I have a ctypes array that I defined like this: ``` buff= (c_ubyte*buff_size)() ``` After filling the buffer up with data, I need to have this data in bytes format. Right now I'm doing it as follows: ``` buff= [n for n in buff] buff = ''.join(map(chr, buff)) ``` The problem with this is that it converts it to a 4-byte (or whatever number of bytes) int before converting it back into a single byte string, which wastes a lot of CPU. How can I convert the ctypes buffer into bytes directly? I'm not trying to save myself a copy because I do have to do a copy anyway since I cannot keep the original buffer. Does python have a cast feature for such things? Thanks.

Accepted Answer

If you did want a copy, you could use `bytearray`:

```
>>> buff = (c_ubyte * 4)(*[97,98,99,100])
>>> bs = bytearray(buff)
>>> bs
bytearray(b'abcd')
>>> str(bs)
'abcd'
```

Edit:

For Python versions prior to 2.6 that lack `bytearray`, you can use one of the following instead:

- `cast(buff, c_char_p).value`

- `buffer(buff)[:]`

If you want to share the same buffer, you can create a `c_char` array:

```
>>> buff2 = (c_char * len(buff)).from_buffer(buff)
>>> buff2.value # string copy
'abcd'
>>> buff2[:] = 'efgh'
>>> buff[:]  # modified original
[101, 102, 103, 104]
```

Edit:

The `from_buffer` class method was added in 2.6. In prior versions you can use `cast`:

- `buff2 = cast(buff, POINTER(c_char * len(buff)))[0]`

Is there a reason you aren't using a `c_char` array to begin with? I understand if you need to work with it as both a numeric array and as a string.

Addendum:

The 2nd method is more 'cast' like since it doesn't copy the buffer. With the first approach it gets copied twice, once to make the `bytearray` and again to make the `str` (`bytes` is an alias for `str` in 2.x). But a `bytearray` has string methods and may be all you need; it's basically a mutable version of 3.x `bytes`.

`c_char` is the C `char` type. Multiplied to an array, it's a mutable buffer of bytes like your current `c_ubyte` array. However, it may be more convenient than `c_ubyte` since it has the `value` and `raw` descriptors that return Python byte strings. It also indexes and iterates as single character byte strings instead of integers.

What you're not supposed to do is create a `c_char_p` -- a pointer to character data -- from a Python string if the function will modify the it. Python strings objects are immutable; you can get weird bugs if you modify their buffer. I recently answered a question on that topic.

Convert ctype byte array to bytes

Solution

Problem