Messagepack-csharp: Ignore members with default values

[ygoe]

Is your feature request related to a problem? Please describe.

My application needs to send structured data over the network regularly. Not all members of that class are always required. For example, data that will never change only needs to be sent once to save bandwidth. Other data may only need to be sent when it has changed. The remote side will track old and new data and add/update it accordingly.

I cannot use MessagePack serialization like this because it will always include all members of the class. Even if I set the members to binary-minimal values (which is already complicated for DateTime) their names will still be included. Using numeric keys is not possible for versioning reasons.

Describe the solution you'd like

Optionally, when MessagePack sees a member that has its default value (default in C#, null for reference types, some form of 0 otherwise), it should simply leave out that member from serialization. The serialized data then looks as if that member wasn't there. On deserialization, it will be left out anyway and remain its default value already.

The DefaultValueAttribute should also be respected to determine the member's default value.

This option could be triggered at a member level with a new IgnoreMemberIfDefaultValue attribute, and a similar solution on the class level for all members not already ignored otherwise.

Describe alternatives you've considered

The IgnoreMember attribute is closest, but can only ignore members every time, not depending on their value.

My current workaround code is this:

/// <summary>
/// Implements a MessagePack formatter that leaves out object properties that have their default
/// value, as well as properties marked as ignored.
/// </summary>
/// <typeparam name="T">The type to format.</typeparam>
public class NoDefaultsFormatter<T> : IMessagePackFormatter<T>
{
    /// <summary>
    /// Serializes the object.
    /// </summary>
    public int Serialize(ref byte[] bytes, int offset, T value, IFormatterResolver formatterResolver)
    {
        var dict = new Dictionary<string, object>();
        foreach (var property in typeof(T).GetProperties())
        {
            if (property.GetCustomAttributes(typeof(IgnoreMemberAttribute), false).Any())
                continue;
            object propValue = property.GetValue(value);
            if (object.Equals(propValue, DeepConvert.GetDefaultValue(property.PropertyType)))
                continue;
            string camelCaseName = char.ToLowerInvariant(property.Name[0]) + property.Name.Substring(1);
            dict[camelCaseName] = propValue;
        }
        return formatterResolver.GetFormatterWithVerify<Dictionary<string, object>>().Serialize(ref bytes, offset, dict, formatterResolver);
    }

    /// <summary>
    /// Deserializes the object.
    /// Not implemented because this operation is performed by DeepConvert.
    /// </summary>
    public T Deserialize(byte[] bytes, int offset, IFormatterResolver formatterResolver, out int readSize)
    {
        throw new NotImplementedException();
    }
}

Note that this isn't optimised in anyway. Using reflection like this is the slowest possible implementation. This implementation targets MessagePack v1.7.

[AArnott]

I would propose we solve this using a similar pattern to JSON.NET's ShouldSerialize methods. Then it can be based on default values or anything else the code wants.

[ygoe]

@AArnott That would still leave the reflection stuff to the user. It would be a lot simpler if the library already supported this option. It could use the same optimised reflection as for other parts.

[github-actions[bot]]

This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 5 days.

[AArnott]

Let's look at this both from the perspective of skipping serialization of values (when using property names as keys) and from skipping deserialization of null values (i.e. don't call the property setter).

[vpodlnk]

Hi! I'm wondering if there are any alternatives to @ygoe workaround?

[AArnott]

The workaround fundamentally requires a custom formatter as @ygoe describes, although there are various ways of implementing it.

[vpodlnk]

Understood. I have the last one)
Do you consider to add some kind of attribute to handle this case like [IgnoreWhenDefault]?

[AArnott]

I don't think so. Most attributes including DataMemberAttribute which we already support have a property on them with this semantic, such as DataMemberAttribute.EmitDefaultValue. The other attribute we support natively is KeyAttribute, which we define and can add a similar property to.

[vpodlnk]

Got it, thanks for quick answers and support, @AArnott !

[github-actions[bot]]

This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 5 days.

[ygoe]

@github-actions I'm still here, if that's what you wanted to know.