Hammerspoon: hs.notify.deliveredNotifications() leaks memory

[Pancia]

I've narrowed down increasing memory usage, up to ~20 GB overnight, to hs.notify.deliveredNotifications().
I used debug.getregistry() to find the offending table that just kept increasing in size, and it looks like the delivered notifications are the "same" but with different memory addresses each call...
I don't really understand obj-c and lua interop, but it seems like notifications pushed to lua never get released https://github.com/Hammerspoon/hammerspoon/blob/2671848973828d6bec08681d4bf107ae69af809d/extensions/notify/internal.m#L272

minimal repro case:

hs.timer.doEvery(1, function()
    hs.notify.deliveredNotifications()
end)

[asmagill]

I can confirm that hs.notify is creating userdata objects that will never be collected (until forced by hs.reload).

This was probably the first module I created for Hammerspoon that wasn't just modifying an existing one from Mjolnir or Hydra... and it shows (to me at least) in that there are a lot of assumptions and code choices that I wouldn't make now...

It's probably going to be sometime late next week or so before I can make the time to look into a fix... the module really needs a complete overhaul.

[latenitefilms]

@Pancia - awesome detective work! I was just wondering if you could provide some further information on how you used debug.getregistry() to find the offending table? I've been trying to hunt down various memory leaks in Hammerspoon for years now - but have never had much luck. Any tips or tricks?

[Pancia]

@latenitefilms - Initially i used this https://github.com/yaukeywang/LuaMemorySnapshotDump, but i found it lacking for this particular example as it just told me there were a lot of hs.notify objects at registry.45.... So i just figured i could print the registry and take a look. It's not really automatable as i had to manually inspect it, but in this case the memory leak was in the GB's and was pretty easy to spot.
However even that was not enough, as it was not clear what was causing so many notification objects to remain in memory. So i went through all my code that had anything to do with notifications, and i eventually traced it by commenting out code, and narrowing it down to hs.notify.deliveredNotifications(). Specifically I printed out the length of that registry element, and saw it increasing when i called that function.

In short, the most useful thing was that library, as i would not have really known what was leaking, or had the idea to look in the registry.

[asmagill]

@latenitefilms FWIW, I've used something like the following before to examine the registry and look for tables that only increase and never decrease in size:

~~~lua
dr = debug.getregistry()
drEval = function(l, everyThing)
l = l or debug.getregistry()
for i,v in ipairs(l) do
local ty, val = type(v), tostring(v)
if ty == "table" then
if v == _G then
val = "Global Environment"
else
local kvCt = 0
local kvIdx, _ = next(v)
while kvIdx do
kvCt = kvCt + 1
kvIdx, _ = next(v, kvIdx)
end

            local isArray, isKV = (#v > 0), (kvCt > #v)
            if not isArray and not isKV then
                val = nil
            elseif isArray and not isKV then
                val = string.format("array: %d items", #v)
            elseif not isArray and isKV then
                val = string.format("kv:    %d pairs", kvCt)
            else -- isArray and isKV then
                val = string.format("table: %d items +  %d pairs", #v, kvCt - #v)
            end
        end
    else
        if not everyThing then val = nil end
    end
    if val then print(i, ty, val) end
end

end
~~~

An example run on my machine:

~~~lua

drEval()
2 table Global Environment
5 table array: 2 items
9 table array: 16 items
10 table array: 1 items
11 table array: 1 items
12 table array: 1 items
17 table array: 3 items
18 table array: 1 items
19 table array: 2 items
24 table array: 1 items
25 table array: 3 items
26 table array: 11 items
29 table array: 1 items
31 table table: 21 items + 1 pairs
33 table array: 143 items
34 table array: 1 items
36 table array: 4 items
37 table array: 11 items
50 table table: 3 items + 1 pairs
56 table kv: 1 pairs
59 table array: 10 items
61 table array: 1 items
67 table array: 1 items
68 table array: 3 items
74 table kv: 1 pairs
76 table array: 1 items
80 table array: 2 items
83 table array: 322 items
84 table array: 345 items
85 table array: 15 items
86 table array: 6 items
88 table kv: 1 pairs
89 table array: 83 items
91 table array: 1 items
~~~

Of course, figuring out which table belongs to which module is annoying because you have to inspect the table in question (hs.inspect) and see what it's collecting, then figure out based on knowing how each module works what collects that kind of items (e.g. if I look at hs.inspect(dr[33]) it shows a combination of hs.hotkey userdata objects and functions... so it's a reasonable guess that this might be the registry table for hs.hotkey, but the only way to tell for certain is to add a new hotkey and then see if it adds one or more new items).

Which gives me an idea (not sure why I didn't think of it before, actually) about adding code in LuaSkin to add a __type field when it creates each module registry subtable, kind of like we do in userdata metatables (e.g. hs.getObjectMetatable("hs.hotkey").__type).