Is there a way to mark the end of each protobuf-net record

Question

I am saving a series of protobuf-net objects in a database cell as a Byte[] of length-prefixed protobuf-net objects: ``` //retrieve existing protobufs from database and convert to Byte[] object q = sql_agent_cmd.ExecuteScalar(); older-pbfs = (Byte[])q; // serialize the new pbf to add into MemoryStream m //now write p and the new pbf-net Byte[] into a memory stream and retrieve the sum var s = new System.IO.MemoryStream(); s.Write(older-pbfs, 0, older-pbfs.Length); s.Write(m.GetBuffer(), 0, m.ToArray().Length); // append new bytes at the end of old Byte[] sum-pbfs = s.ToArray(); //sum-pbfs = old pbfs + new pbf. Insert sum-pbfs into database ``` This works fine. My concern is what happens if there is slight db corruption. It will no longer be possible to know which byte is the length prefix and the entire cell contents would have to be discarded. Wouldn't it be advisable to also use some kind of a end-of-pbf-object indicator (kind of like the 
 or EOF indicators used in text files). Th…

Accepted Answer

If you use a vanilla message via `Serialize` / `Deserialize`, then no: that isn't part of the specification (because the format is designed to be appendable).

If, however, you use `SerializeWithLengthPrefix`, it will dump the length at the start of the message; it will then know in advance how much data is expected. You deserialize with `DeserializeWithLengthPrefix`, and it will complain loudly if it doesn't have enough data. However! It will not complain if you have extra data, since this too is designed to be appendable.

In terms of Jon's reply, the default usage of the `*WithLengthPrefix` method is in terms of the data stored exactly identical to what Jon suggests; it pretends there is a wrapper object and behaves accordingly. The differences are:

- no wrapper object actually exists

- the "withlengthprefix" methods explicitly stop after a single occurrence, rather than merging any later data into the same object (useful for, say, sending multiple discreet objects to a single file, or down a single socket)

The difference in the two "appendable"s here is that the first means "merge into a single object", where-as the second means "I expect multiple records".

Unrelated suggestion:

```
s.Write(m.GetBuffer(), 0, m.ToArray().Length);
```

should be:

```
s.Write(m.GetBuffer(), 0, (int)m.Length);
```

(no need to create an extra buffer)

Is there a way to mark the end of each protobuf-net record

Solution

Problem