Runtime: Old Serialization design is back but it needs refactoring.

[olivier-spinelli]

The [Serializable] is back (I don't need it since I think it is as useless as a generic in-depth Clone method) but fine. Howerver, I have one major design concern. Current design forces us to store transient serialization data in the object itself. It is awful by design and, IMO, defeats the "simple-safe-easy-to-use-principle" of the API:

• It is not obvious that storing this transient sate should be the way to go (even for beginners, it hurts).
• Failing to set this state to null in OnDeserialization (or OnDeserialized "event') will lead to memory leaks. This will happen for sure.
• Understanding OnDeserialized event vs. IDeserializationCallback.OnDeserialization is... fun but not as much as the SafeSerializationManager...

Since I understand the ".Net Framework API surface compatibility" goal, I think we (the .Net community at large) are missing here a great opportunity to fix what is clearly a bad design option.

Is there any issue with the following approach?
1 - Offer a way to the deserialization ctor to store a transient state object.

• Thanks to a new SetDeserialiazationState method on the StreamingContext?
• Ouch! it's a struct that cannot be changed/extended... let's handle another deserialization ctor with an enhanced StreamingContext or one new parameter.

2 - Give back this object to the OnDeserialization method.

• Thanks to a new IDeserializationStateCallback may be?
• And, may be also, the OnDeserialized event could allow a new "object state" in its signature (since this one is more "duck typed".

The ObjectManager should be able to manage this transient state correctly.
Am I missing something here?
I think so... SafeSerialization introduce other complexities. And the attribute/event approach is better than the potentially virtual OnDeserialization method.

Again, am I missing something here? Are there hidden/secret plans to handle this more correctly?
I guess I'm not the only one that feel a little bit sad to see those stuff coming back into the clean .Net Core code base...

[danmosemsft]

@olivier-spinelli as you observed, it's brought back purely so there's a compatible surface area -- it's easier to bring existing code (libraries) from desktop/xamarin/unity to Core. If it wasn't for compat we would not have brought it over as it has some serious drawbacks - it is really hard to maintain. We are however not planning to support cross-framework compatibility and the current plan is to not support cross-version compatibility (that may change depending on feedback)

We don't plan to invest in it but like anything we're open to ideas to improve it as long as it doesn't add risk or break compatibility in any perceptible way.

[olivier-spinelli]

I understand this but the cost seems too much to me. Look at the beautiful ConcurrentStack: one Node _head... and a T[] _serializationArray: 2 references instead of only one. You may consider me a "purist", but storing transient refs in an object state is (one of) the root of all evil :(.

I understand the "keep the old code working", but the .Net Core ConcurrentStack is NOT legacy code. It's a new born baby that we are hurting ;). We are _starting with_ a technical debt... All these clean code COULD use a new, dedicated, better serialization support without paying their fathers' debts.

[danmosemsft]

I sympathize .. it's taken us a while to come to terms with it ourselves.

[olivier-spinelli]

So... the case is closed? No hope to design something cleaner?

[danmosemsft]

@stephentoub

[karelz]

@olivier-spinelli don't forget we are also locking down for 2.0 release - see dotnet/runtime#20809 for details. We have to be very practical. If you have specific suggestions/concerns that fit into the tight deadlines, then we can consider them.
If it is fuzzy, or unclear, then we will have to postpone it for future.
Makes sense?

[olivier-spinelli]

Thanks for taking this into consideration. My real fear is very simple: this poorly designed stuff will it remain for long or do you have plans to be able to supersede it with a new, better API?
My feeling is that this MAY be possible... but could be much more easier to do it now rather than after (where we'll have to get back to all the fundamentals objects like the poor ConcurrentStack to make them use the alternate API).

May be a first step would be to consider right now an alternate ctor signature, with a more powerful SerializationContext (that basically can memorize transient states) along with new [Deserialize] private methods that would be given back this memory.

This does not solve the "SafeSerialization" issue, but this could be a way for BCL objects to avoid being polluted... AND...my feeling is that working on this "state externalization" along with the "Objects Slices" that enables [Deserialize] private methods is a good first step to Safe serialization where one of the key issue is to prevent a type to interfere in any manner with the serialization of its generalization or specialization (hence the "Slice" term I used above).

Does this make sense to you?

[stephentoub]

this poorly designed stuff will it remain for long

Whether we like it or not, it will: it's supported by the .NET Framework and has been for years and will be for years; it's supported by Mono and has been for years and will be for years; etc. .NET Core needs to play nicely in this ecosystem.

That said, we are planning to scale back how much of this we support in 2.0 (https://github.com/dotnet/corefx/issues/19119), so that a) where we do support it we can support it well (e.g. cross-runtime serialization), and so that b) we can avoid the shackles that come with supporting it everywhere else. The list is still being talked about, but I expect it'll be a core set of types, e.g. primitives, core collections, some wrapper types like Tuple, etc.

We are not planning to invest in runtime serialization moving forward. We're supporting it in .NET Core in key places because .NET Core isn't divorced from the existing ecosystem and needs to and benefits from playing nicely in it. Lots of code written for .NET Framework needs to be able to "just work" on on .NET Core. We generally recommend that other serialization mechanisms be used rather than ones based in the runtime.

Thank you for the feedback.

[karelz]

could be much more easier to do it now rather than after

Please read the reference I mentioned above in dotnet/runtime#20809 - 5/10 is our zero-bugs-goal, then we will be raising bar higher and higher, until we ship RTW. We already locked down API surface weeks ago, taking only few minor low-risk exceptions.
Adding brand new serialization is far away from low-risk feature. It has to be thought through carefully. There hasn't been any proposal so far. If there is something awesome we come up with later, we could consider it in future releases, but there is no chance it will make it into 2.0 at this point.

[olivier-spinelli]

Thank you for answering. I wont disturb you anymore with this issue, I promise!
And many, many, thanks for the great work you all guys are doing!
I don't know if I should close this issue or let it opened as a remainder: feel free to close it if you want.

[karelz]

Thanks for understanding @olivier-spinelli. Please don't take it that we are trying to shut down the discussion. I'm just trying to set realistic expectations on the time-to-resolution.

To be honest, I am not sure if we will ever invest in a brand new Serialization in BCL/Runtime. From our experience, it is a hard problem, and it can be solved externally (just on public types) ... so, maybe the best way is to push anything new out of BCL? -- I wonder what our experts on the subject think @stephentoub @jkotas @zhenlan @shmao @morganbr