zlib decompression failing
compression, zlib
Solution
Thanks to the zlib authors, I have found the answer. The third party app is generating zlib streams that are not finished correctly:
78 9c f2 48 cd c9 c9 57 08 cf 2f ca 49 51 e4 e5 02 00 00 00 ff ff
That is a partial zlib stream, consisting of a zlib header and a partial deflate stream. There are two blocks, neither of which is a last block. The second block is an empty stored block, used as a marker when flushing. A zlib decoder would correctly decode what's there, and then continue to look for data after those bytes.
78 9c f3 48 cd c9 c9 57 08 cf 2f ca 49 51 e4 e5 02 00 24 e9 04 55
That is a complete zlib stream, consisting of a zlib header, a single block marked as the last block, and a zlib trailer. The trailer is the Adler-32 checksum of the uncompressed data.
So My decompression is failing - probably because the CRC is missing, or the decompression code keeps looking for more data that does not exist.
Problem
I'm writing an application that needs to uncompress data compressed by another application (which is outside my control - I cannot make changes to it's source code). The producer application uses zlib to compress data using the z_stream mechanism. It uses the Z_FULL_FLUSH frequently (probably too frequently, in my opinion, but that's another matter). This third party application is also able to uncompress it's own data, so I'm pretty confident that the data itself is correct. In my test, I'm using this third party app to compress the following simple text file (in hex): `48 65 6c 6c 6f 20 57 6f 72 6c 64 21 0d 0a` The compressed bytes I receive from the app look like this (again, in hex): `78 9c f2 48 cd c9 c9 57 08 cf 2f ca 49 51 e4 e5 02 00 00 00 ff ff` If I try and compress the same data, I get very similar results: `78 9c f3 48 cd c9 c9 57 08 cf 2f ca 49 51 e4 e5 02 00 24 e9 04 55` There are two differences that I can see: First, the fourth byte is `F2`, rather than `F3`, so the deflate "final block" bit has not been set. I assume this is because the stream interface never knows when the end of the incoming data will be, so never sets that bit? Finally, the last four bytes in the external data is `00 00 FF FF`, whereas in my test data it is `24 E9 04 55`. Searching around I found on this page http://www.bolet.org/~pornin/deflate-flush.html ...that this is a signature of a sync or full flush. When I try and decompress my own data using the `decompress()` function, everything works perfectly. However, when I try and decompress the external data the `decompress()` function call fails with a return code of `Z_DATA_ERROR`, indicating corrupt data. I have a few questions: Should I be able to use the zlib "uncompress" function to uncompress data that has been compressed with the z_stream method? In the example above, what is the significance of the last four bytes? Given that both the externally compressed data stream and my own test data stream are the same length, what do my last four bytes represent? Cheers