Can protobuf read partially?
c++, protocol-buffers
Solution
This depends on which implementation you are using. Some have "read as a sequence" APIs. For example, assuming you stored it as a "repeated Quad", then with protobuf-net that would be:
int x = ..., y = ...;
var found = Serializer.DeserializeItems<Quad>(source)
.Where(q => q.x ==x && q.y == y);
The point being: it yields a spooling (not loaded all at once) and short-circuiting sequence.
I don't know the c++ api specifically, but I would hope it has something similar - but worst case you could parse the varint headers and prepare a length-capped stream.
Problem
I want to save my terrain data to a file and load only some parts of it, because it's just too big to store it in memory as a whole. Actually I don't even know whether the protobuf is good for this purposes. For example I would have a structure like (might be invalid gramatically, I know only simple basics): ``` message Quad { required int32 x = 1; required int32 z = 2; repeated int32 y = 3; } ``` The x and z values are available in my program and by using them I would like to find the correct Quad object with the same x and z (in the file) to obtain y values. However, I can't just parse the file with the ParseFromIstream(), because (I think so) it loads whole file into memory, but in my case the file is just too big. So, is the protobuf able to load one object, send me for checking it and if the object is wrong give me the second one? Actually... I could just ask: does the ParseFromIstream() loads whole file into memory?