Messagepack-csharp: Large size (larger than using standard serialization) of the resulting byte array.

[ns-88]

Hello!
I am using the MessagePack serializer to serialize a fairly large class with many fields and properties, each of which in turn has a specific type, also serializable using MessagePack. When I first researched MessagePack, serializing simple classes with them, I noticed how much smaller the size of the serialized data than if I used the standard BinaryFormatter serialization. But when I got to the serialization of the previously mentioned large class used in a real project, the situation changed radically. So the size of the serialized data is:

BinaryFormatter - 5413250 bytes.
MessagePack - 9075100 bytes.

I am optimizing data transfer in our project and was going to use MessagePack as part of it. It goes without saying that such results are unacceptable for me. What could be the reason for such a significant increase in the size of the serialized data, and is there a way to reduce their size at least to the level of standard serialization? I am using the following attributes from MessagePack.Annotations: MessagePackObject, IgnoreMember and Key. As a key, I use a zero-based ordinal. I make the numbering starting from some base class and further along the inheritance hierarchy.

[AArnott]

It sounds like you're taking the right steps.
There's nothing fundamental about MessagePack that would make it substantially larger than .NET's binary formatter, and in fact MessagePack is typically much smaller for many values.
I suggest you use a smaller test case and then decode the messagepack to JSON and inspect it for gross redundancy. For example, perhaps you have a bunch of null values in fields that the binary formatter was just skipping. Or perhaps you think you're using arrays but are actually using maps to serialize members. JSON decoding should reveal this.

[ns-88]

Thank you for your quick reply!
I took your advice and serialized the class of interest to JSON. Indeed, in the serialized data there are a lot of values ​​of the form: null, false, and 0, which suggests that the increased size of the serialized data compared to standard serialization is really due to the serialization of fields and properties whose values ​​are not set. Question: can the MessagePack serializer be configured so that it does not serialize such fields? I also tried serialization with Lz4Block and Lz4BlockArray compression. This approach gave very good results, allowing to significantly reduce the size of the output, but I would like to not use compression for now, but to achieve a reduction in the size of the serialized data by setting the serializer so that it does not serialize empty values, as I wrote above.

[AArnott]

You can't skip default values when serializing to arrays, because _something_ has to fill those positions to offset other values. But if you switch to maps (as binary formatter uses) it's possible to skip default values once #678 is done.