Hammerspoon: Save Console History across restarts?

[latenitefilms]

Would it be possible for Hammerspoon to save it's Console history across restarts?

It could be handy if it saved the history using hs.settings?

[asmagill]

It should probably be cleaned up and maybe made into a spoon at some point, but take a look at https://github.com/asmagill/hammerspoon-config-take2/blob/master/utils/_actions/consoleHistory.lua

[cmsj]

I'm gonna close this issue, since some code exists 🙂

@latenitefilms if you want a reminder about doing this, or as a flag for someone else to maybe have a go, I'd suggest filing it as an issue on the Spoons repo.

[latenitefilms]

@cmsj & @asmagill - Funnily enough, I think this might actually be the source of my memory leak issues!

Here's my current code (which is heaily based on @asmagill's link above):

https://github.com/CommandPost/CommandPost/blob/develop/src/extensions/cp/console/history.lua

Can you see anything obviously broken in here?

Rather than using a hs.timer, I wonder if we could add a callback function to Hammerspoon's console that triggers when a command is entered?

[latenitefilms]

Actually... I probably don't even need a timer, as the function is triggered on garbage collection anyway.

[asmagill]

Not sure why that lua code would trigger leaks, though I'll try and take a closer look at it this weekend; but to answer your question re a console callback, take a look at https://github.com/Hammerspoon/hammerspoon/pull/921

It probably should be documented somewhere other than just in the git history, but the original purpose was to provide a way to simulate python style help for people used to that style (e.g. help(hs.foo) rather than the lua styles of help.hs.foo or help("hs.foo"))

Note that it is a pre-parser so it has no idea if the command was any good, just that it was typed in.

[latenitefilms]

Very nice! I had no idea about hs._consoleInputPreparser - so yes, this would be good to include in the hs.console documentation. Although, I think calling the console history function on garbage collection probably solves the issues anyway?

I'm not 100% sure that this code is the cause of my issues - I'll only know for certain once I spend a few hours working with CommandPost running. But at the moment, if I'm editing in Final Cut Pro for a few hours, eventually I'll get a "Your system has run out of application memory", and the Activity Monitor Memory Pressure graph is all red and yellow. As soon as I force quit CommandPost it goes back to green. It's something in the code I've changed in the last two months - but I have no idea what or where, which is what I've been trying to track down.

My THEORY is that maybe because we're triggering the hs.timer every 1 second that's preventing garbage collection from occurring properly?

[asmagill]

I've wondered about hs.timer myself... when I last reviewed the code, I didn't see anything obvious, but my gut instinct is still telling me to look at it closer. I never did get the leaks command to work running from the command line, but I've been following what you and @cmsj have been doing with Intruments within XCode and figured I should try that next, so maybe I'll have more luck with it this weekend.

[latenitefilms]

There seems to be a few "CF" objects in hs.timer, but not CFRelease's - so I wonder if that's an issue?

[cmsj]

We switched hs.timer to use NSTimer a while ago. There are no Create or Copy CF function calls in hs.timer, so that can't be a cause of leakage. That doesn't mean that hs.timer objects themselves don't leak, or some other allocation, but CF has clear rules - iff you see Create or Copy in the function name, you own the memory.

[asmagill]

Note also that even if a CF object is owned by us, if it's converted to an Objective-C object through __bridge_transfer, then ARC will take over and you don't need to (in fact shouldn't) explicitly release it yourself... I know I've used this technique before to reduce code length, though I've no idea if its done in the timer code or not... probably not if @cmsj is correct about not using a CF Create or Copy function.

[cmsj]

(on the specific subject of converting CF to NSObject, I've come to quite like using CFBridgingRelease() - it feels like it makes the code clearer than the __bridge_foo compiler hints)

[latenitefilms]

I'm not sure that my specific problem is a memory leak per se, because I'm not really seeing any crazy amounts of leaks in the Leaks Instrument. My GUESS is that it's garbage collection just not working properly - but really I have no idea. Are there specific garbage collection settings in LuaSkin?

[asmagill]

You can check out http://www.lua.org/manual/5.3/manual.html#2.5 and http://www.lua.org/manual/5.3/manual.html#pdf-collectgarbage for more details on how to tweak the gc settings in Lua. We haven't added anything specific in LuaSkin as it's mainly a bridge between lua userdata and Objective-C objects... if LS has a collection issue it will most likely appear as a leak and not as something the Lua interpreter would deal with.

If anyone knows of or finds a good way to observe or visualize lua garbage and collection in something approaching real time, please share as its seems to me at present to be something of a black-box that either works well or doesn't most of the time.

--edit: black-box, not block-box

[latenitefilms]

FYI: This is what Leaks is currently reporting for me:

It looks like maybe hs.base64 might be worth looking into? I also reckon there might still be some memory issues with hs._asm.axuielement?

[asmagill]

@cmsj, if using the CFBridging... functions is something you feel strongly about, I can try... to me I've always liked the __bridge... syntax better because it feels a little more Objective-C like whereas the functions feel closer to stock C, but I'm flexible. We can move this to another thread if you want to discuss it or other coding style issues further.

[cmsj]

@asmagill thinking about it, we're so wedded to __bridge already, I'd rather our codebase didn't fragment in yet another way 😉

[cmsj]

So here's another great reason to have tests - it's now possible to right click on one test or a group of tests (or indeed all the tests) and choose "Profile" and get decent results for all of our code, or just one particular module.

I ran the full test suite (which, remember, only covers about 1/4 of the HS codebase) and immediately found three leaks, which should be fixed by 6608ec64

I don't yet understand why a lot of leaks show up in Apple frameworks, but for now I'm ignoring those, because I can't imagine what we would even do about them.

[latenitefilms]

Awesome!!

So we should basically try and write tests for all the Objective-C extensions then I guess? I should also probably write some tests for the hs._asm extensions as well, given I’m using a lot of them. I’ll try have a play with this over the next few weeks.

[cmsj]

(bah, had to revert one of the leak fixes because it was causing a crash)

[latenitefilms]

@cmsj - I also noticed you updated Lua? Any speed improvements/benefits with that?

[cmsj]

no, just fixes the 7 known bugs in 5.3.4: https://www.lua.org/bugs.html