Python binary data reading
binary-data, python
Solution
>>> data
'\x00\x00\x00\x01\x00\x04AMTD\x00\x00\x00\x00\x02A\x9733A\x99\\)A\x90=qA\x91\xd7\nG\x0f\xc6\x14\x00\x00\x01\x16j\xe0h\x80A\x93\xb4\x05A\x97\x1e\xb8A\x90z\xe1A\x96\x8fWF\xe6.\x80\x00\x00\x01\x16zS|\x80\xff\xff'
>>> from struct import unpack, calcsize
>>> scount, slength = unpack("!IH", data[:6])
>>> assert scount == 1
>>> symbol, error_code = unpack("!%dsb" % slength, data[6:6+slength+1])
>>> assert error_code == 0
>>> symbol
'AMTD'
>>> bar_count = unpack("!I", data[6+slength+1:6+slength+1+4])
>>> bar_count
(2,)
>>> bar_format = "!5fQ"
>>> from collections import namedtuple
>>> Bar = namedtuple("Bar", "Close High Low Open Volume Timestamp")
>>> b = Bar(*unpack(bar_format, data[6+slength+1+4:6+slength+1+4+calcsize(bar_format)]))
>>> b
Bar(Close=18.899999618530273, High=19.170000076293945, Low=18.030000686645508, Open=18.229999542236328, Volume=36806.078125, Timestamp=1195794000000L)
>>> import time
>>> time.ctime(b.Timestamp//1000)
'Fri Nov 23 08:00:00 2007'
>>> int(b.Volume*100 + 0.5)
3680608
Problem
A urllib2 request receives binary response as below: ``` 00 00 00 01 00 04 41 4D 54 44 00 00 00 00 02 41 97 33 33 41 99 5C 29 41 90 3D 71 41 91 D7 0A 47 0F C6 14 00 00 01 16 6A E0 68 80 41 93 B4 05 41 97 1E B8 41 90 7A E1 41 96 8F 57 46 E6 2E 80 00 00 01 16 7A 53 7C 80 FF FF ``` Its structure is: ``` DATA, TYPE, DESCRIPTION 00 00 00 01, 4 bytes, Symbol Count =1 00 04, 2 bytes, Symbol Length = 4 41 4D 54 44, 6 bytes, Symbol = AMTD 00, 1 byte, Error code = 0 (OK) 00 00 00 02, 4 bytes, Bar Count = 2 FIRST BAR 41 97 33 33, 4 bytes, Close = 18.90 41 99 5C 29, 4 bytes, High = 19.17 41 90 3D 71, 4 bytes, Low = 18.03 41 91 D7 0A, 4 bytes, Open = 18.23 47 0F C6 14, 4 bytes, Volume = 3,680,608 00 00 01 16 6A E0 68 80, 8 bytes, Timestamp = November 23,2007 SECOND BAR 41 93 B4 05, 4 bytes, Close = 18.4629 41 97 1E B8, 4 bytes, High = 18.89 41 90 7A E1, 4 bytes, Low = 18.06 41 96 8F 57, 4 bytes, Open = 18.82 46 E6 2E 80, 4 bytes, Volume = 2,946,325 00 00 01 16 7A 53 7C 80, 8 bytes, Timestamp = November 26,2007 TERMINATOR FF FF, 2 bytes, ``` How to read binary data like this? Thanks in advance. Update: I tried struct module on first 6 bytes with following code: ``` struct.unpack('ih', response.read(6)) ``` (16777216, 1024) But it should output (1, 4). I take a look at the manual but have no clue what was wrong.