Go: runtime: use MADV_FREE on linux as well

[CAFxX]

On linux currently sysUnused uses madvise(MADV_DONTNEED) to signal the kernel that a range of allocated memory contains unneeded data. After a successful call, the range (but not the data it contained before the call to madvise) is still available but the first access to that range will unconditionally incur a page fault (needed to 0-fill the range).

A potentially faster alternative is MADV_FREE, available since linux 4.5 (and already used for bsd). The mechanism is very similar, but the page fault will only be incurred if the kernel, between the call to madvise and the first access, decides to reuse that memory for something else.

Off the top of my head, there are two ways to support both pre-4.5 and 4.5+ kernels:

• Use a fallback approach (i.e. try MADV_FREE first and MADV_DONTNEED if MADV_FREE fails): this is e.g. what node does
• Detect at runtime startup whether the running kernel supports MADV_FREE (e.g. by attempting to call madvise(MADV_FREE) on an allocated but unused range)

[bradfitz]

/cc @aclements

[gopherbot]

Change https://golang.org/cl/135395 mentions this issue: runtime: use MADV_FREE on Linux if available

[alk]

Hi all. I wondering if there is any data backing up claims about MADV_FREE actually improving anything.

Asking because:

a) I have not seen any data pointing that it does help (I co-maintain gperftools and over there we're defaulting MADV_FREE to off on Linux).

b) there are numerous questions about implementation of this feature in kernel. First, while it is somewhat cheaper then MADV_DONTNEED, it is not that cheap, since page tables have to be walked and modified. It doesn't get reflected in RSS and thus interaction of this and memory reclaim facility in kernel, especially in settings with heavy usage of control groups isn't trivial. And also defaults about how this memory is preferred compared other memory during reclaim is also not entirely clear.

Since this feature was lobbied by jemalloc folks, maybe there are some Facebook folks involved in golang who do have some data?

I think this ticket should be re-opened. We should either gather and publish the data and see that indeed it helps, or revert defaults to off. IMHO.

[thepudds]

@alk I'm curious if you have tried any experiments with GODEBUG=madvdontneed=1? From the Go 1.12 release notes:

To revert to the Go 1.11 behavior (MADV_DONTNEED), set the environment variable GODEBUG=madvdontneed=1

Separately, given the volume of issues, I think some people automatically filter out notifications from closed issues, so you might want to open a new issue with your comments above.

[gopherbot]

Change https://golang.org/cl/267100 mentions this issue: runtime: default to MADV_DONTNEED on Linux