Runtime: Advanced GC Internals Documentation

[swgillespie]

The Book of the Runtime is a great resource on the GC, but it glosses over the details of many aspects of the GC that are important to understanding how the system works on a deep level. Today, it's very difficult to understand the GC just by looking at the Book of the Runtime alone - it takes a lot of time, reading gc.cpp, and debugging to get a feel for the different aspects of the GC and how they interact.

It would be great if we had centralized, detailed documentation for GC internals in addition to the Book of the Runtime in the hopes of lowering the barrier of entry into the GC.

Some issues where GC internals have been discussed (would be great candidates for documentation!)

• https://github.com/dotnet/coreclr/issues/10305 - [Question] Understanding demotion
• https://github.com/dotnet/coreclr/issues/10137 - [Question] What's function set_new_pin_info used for?
• https://github.com/dotnet/coreclr/issues/9053 - [Question] What's the relationship of GCHeap, gc_heap, heap_segment, alloc_context and generation?
• https://github.com/dotnet/coreclr/issues/8995 - [Question] How many GCHeap and gc_heap instances per .net process?
• https://github.com/dotnet/coreclr/issues/9332 - What's the meaning of local GC? (also https://github.com/dotnet/coreclr/pull/8199#issuecomment-262669283)

Some topics that aren't covered by the BOTR, off the top of my head (would also be great candidates for documentation):

• Pins in general: how the EE reports things as pinned to the GC, how the Gen 0 allocator deals with pins on the ephemeral segment, the demotion mechanism, pinned plugs and POPO
• Allocating in condemned generations
• Free lists - the BOTR mentions that the sweep phase populates free lists but doesn't talk much about how they are used when allocating in higher generations.
• How to use the GC Log to view GC debug output
• Joins in Server GC - why they are necessary, hazards associated with unbalanced heaps, measures taken to ensure that each heap has a proportional amount of work to do (e.g. mark stealing, alloc_heap vs. home_heap)

cc @karelz @Maoni0 @adityamandaleeka @sergiy-k

[swgillespie]

also cc @303248153 who asked many of the questions linked in this issue

[krs43]

I'm also interested in the conditions that trigger garbage collections. Obviously free space will, but I see a lot triggered by fragmentation, and the collection is unsuccessful in reducing the fragmentation, but keeps trying to.

[adityamandaleeka]

@krs43 Do you have pinned objects in your scenario? What is your memory load like?

Pinned objects can cause fragmentation (since, obviously, they cannot be moved). Under certain circumstances, the GC might notice that fragmentation and think it can shrink the heap if it does a collection, but of course it will be unsuccessful at this because it can't do anything about the pinned objects.

If you have other questions or would like to discuss this further, I recommend opening a separate issue.

[karelz]

I would suggest to split off specific questions into separate issues, so this one doesn't become a one giant one. We should track only actionable docs update here ... (speaking from experience with large muddied discussions on CoreFX ;-))

[303248153]

A few days ago I wrote an article about GC internals in chinese, and I'm agree with swgillespie, BOTR is great but far from enough.
I hope the new document would include these topics:

• Stress GC
• How the threshold used for trigger GC calculated (dd_desired_allocation) and RegisterForFullGCNotification
• GC Handles (pinned, strong ref, weak ref, and especially sized ref and ref counted, they are undocumented)
• References from static variables (corresponding gc handle)
• Generation in gc handle table (Ref_AgeHandles, Ref_RejuvenateHandles, rgGeneration)
• Preemptive and cooperative mode
• Card Table (mentioned in BOTR but I would like more details)
• Card Bundles
• Write Barrier
• Write Watch
• Brick Table (relocate_survivors_in_brick, make_free_list_in_brick, find_first_object)
• Mark array
• Plug and Gap
• Saved pre and post plug info (memory override by next plug info)
• The condition of promotion
• The condition of not promotion
• The condition of demotion
• Consing gen and allocating in condemned generations (as you mentioned)
• How generation boundaries change (process_ephemeral_boundaries, plan_generation_start, make_free_lists)
• How plan phase choose compact or sweep (fragmentation, decide_on_compacting)
• How allocation quantum be adjusted
• How gc_heap get balanced in server gc (heap_select::select_heap)
• Expand heap (expand_heap, rearrange_heap_segments)
• Joins in server gc (as you mentioned)
• The different between background mark phase and mark phase
• The different between background sweep phase and sweep phase
• Low card table efficiency (generation_skip_ratio, mark_through_cards_for_segments)

In addition to improving the document I also want more comments around gc codes, like explain the hundreds members in gc_heap and members in dynamic_data.

[karelz]

Here's translation of the article.

@303248153 are you interested in contributing some of these docs to the repo?
Or do you have suggestions / contributions to the comments? (not sure how practical it is to comment on fields without pointing to the big picture)

[303248153]

@karelz
Although I am willing to contributing some documents about these topics, it's better to let @swgillespie and @Maoni0 do it because they are more familiar with these code and they can provide more accurate information.
Also adding comment to members of gc_heap and dynamic_data is necessary with or without a big picture IMO, when I'm reading these code I feel so painful to guess the role of a field, I had to read thousands of lines to figure out, sometimes I just need a very little help, that's the comment.

[mattwarren]

I know that everyone it pretty busy with real (dev) work, but are there still plans to do this?

[swgillespie]

@mattwarren It's something I'd like to do, but unfortunately I just haven't found the time. 😞

[xiangzhai]

:mips-interest

[Maoni0]

this should be closed. in the BotR GC chapter I've recommended to read @kkokosa's Pro .NET Memory book which talks about a lot of details of the GC code.