Runtime: [Local GC] Decouple Object & ArrayBase layout

[sunnycase]

We are writing a wrapper between coreclr and mono api to make Unity games running on top of coreclr. (repo link)
But now we have encountered some issues about object model conflicts between mono and coreclr.

Object

Issue

In mono, an object header is inside of the object.

typedef struct {
    MonoVTable *vtable;
    MonoThreadsSync *synchronisation;
} MonoObject;

• The first field of an object starts at offset of +16 bytes in mono.

Where in coreclr an object header is before the object.

class Object
{
  protected:
    PTR_MethodTable m_pMethTab;
...
};

• The first field of an object starts at offset of +8 bytes in coreclr.

When Unity making a FCall, it writes the first field in a object which is at offset of +16 bytes in mono but in fact the field is at +8 bytes in coreclr. Then it crashes.

Solutions

1. Move ObjHeader back into Object :x:
   We tried compare but failed in GC. Currently GC has a close contact with object model. It is hard to modify.
2. Add 8 bytes padding after m_pMethTab ✔️
   It definitely works. compare. But this solution will take more memory usage.

ArrayBase

Issue

array's defination in mono:

typedef struct {
    MonoObject obj;
    /* bounds is NULL for szarrays */
    MonoArrayBounds *bounds;
    /* total number of elements of the array */
    guint32 max_length; 
    /* we use double to ensure proper alignment on platforms that need it */
    double vector [MONO_ZERO_LEN_ARRAY];
} MonoArray;

array's defination in coreclr:

class ArrayBase : public Object
{
private:
    // This MUST be the first field, so that it directly follows Object.  This is because
    // Object::GetSize() looks at m_NumComponents even though it may not be an array (the
    // values is shifted out if not an array, so it's ok). 
    DWORD       m_NumComponents;
    DWORD       pad;

    // What comes after this conceputally is:
    // TypeHandle elementType;        Only present if the method table is shared among many types (arrays of pointers)
    // INT32      bounds[rank];       The bounds are only present for Multidimensional arrays   
    // INT32      lowerBounds[rank];  Valid indexes are lowerBounds[i] <= index[i] < lowerBounds[i] + bounds[i]
...
};

It seems ArrayBase is also a core type used in GC. Even a free object is a ArrayBase. If we change the ArrayBase's layout, GC will not work once again. 🤕

Conclusion

Will it be possible to decouple Object & ArrayBase layout between GC and EE? If it's OK, we will be very appreciate that you will save us a lot of time. Thanks.

[xoofx]

@sunnycase FYI, during the hackweek at Unity last May, we did a prototype of Unity running with CoreCLR (I announced it here https://twitter.com/xoofx/status/867903250649034756). I still have to write a blog post about it so you will be able to understand how we were able to achieve this...

About our approach, we simulated the Mono API on top of the CoreCLR API, so it looks similar to what you are trying to achieve, but we made sure that we didn't have to match the internals of Mono to the internals of CoreCLR. So we had actually very few changes to make to CoreCLR to make this happen.

But there are also some fixes in Unity (in the C++ codebase) that are required in order to make this whole thing just work. There is also quite a large undergoing work started a few weeks ago at Unity to use proper write barriers, so it requires actually more work on Unity side than on the layer Mono/CoreCLR. Don't want to undermine your goal, just that you are aware of some unfortunate pitfalls.

[jkotas]

Will it be possible to decouple Object & ArrayBase layout between GC and EE?

The layout of object or array, encoding of object layout, or write barriers are part of the GC/EE contract. This is done for performance reasons. Abstracting these away would make the GC many times slower.

[sunnycase]

@xoofx

But there are also some fixes in Unity (in the C++ codebase) that are required in order to make this whole thing just work.

Actually we have considered this approach. But we can't afford a Unity source code license and we can't modify and redistribute Unity as that would conflict with Unity's terms of service.

[leohawke]

Have you ever listen of an idiom?
“厉外垃同”
You're better because you're a foreigner
I'm garbage because I'm your compatriot

@xoofx We don't have Unity Source ,We Can't change Unity Source,We don't care Unity Source.
@jkotas can you give us more information about it?

[crazii]

@xoofx
Could it be a general case/requirement that can be applied to more scenarios no just like Unity?

[jkotas]

@leohawke

If you would like to propose refactorings that do not affect performance of the default CoreCLR build and make your life easier, we can discuss those on case-by-case basis. We are pretty open to changes that make the code more flexible or better factored.

However, the performance affecting changes such as adding padding fields to object layout are not going to fly well - you will always have to build your own clone if you need the padding fields to be present, they won't be present in the default CoreCLR build.

[xoofx]

Actually we have considered this approach. But we can't afford a Unity source code license and we can't modify and redistribute Unity as that would conflict with Unity's terms of service.

What I was implying is that without these changes in Unity, you won't be able to run anything. Unity is for example accessing managed objects fields directly without going through mono API...

Could it be a general case/requirement that can be applied to more scenarios no just like Unity?

Sure. But again, in that case, you would barely have to change anything in coreclr (and typically the change of this issue doesn't make sense)

[sunnycase]

@xoofx

What I was implying is that without these changes in Unity, you won't be able to run anything. Unity is for example accessing managed objects fields directly without going through mono API...

That's why we change the object model to align with mono.

[xoofx]

That's why we change the object model to align with mono.

I understood, but for example Unity doesn't do write barriers yet, so it won't work until it is done in Unity, no matter how hard you try to align with mono.

Bottom line, if you want to align the runtime with mono, surely you can try to upstream some changes to coreclr if it makes sense (as @jkotas suggested), but many will still require a hard fork, and not only coreclr but also for some corefx classes (Unity is accessing for example some private fields implementations of System.* classes, and it would mean that it would require to align some classes also there - and good luck in finding which one and how it is accessed btw - unlikely up-stream-able, and I don't even know if it will be possible in practice to match easily internals...)...

Anyway, I'm just giving you my experience on this matter, so that you are aware of the shortcomings and the feasibility burden...