I first reported this at https://bugzilla.mozilla.org/show_bug.cgi?id=1324124 because I thought it was due to a recent Nightly update, but it may have just been from the latest TST update. This is TST 0.18.2016111701. (I cannot run the previous version because it uses 'for each'.)
The profile at https://new.cleopatra.io/public/c93668c36edaf7aee24653ec696dc939d08d7459/calltree/?thread=0 shows what I am observing. But I'll cut & paste from the above bug:
Result:
Immediate hang. Laptop fan spins up. I clicked continue on 3 or 4 slow tab dialogs in a row, then clicked Stop Script on the next (in tabbox.xml). This rendered the browser nonfunctional. (I could Ctrl-L to go to the location bar and enter a url, but it would not load any new tabs when I pressed enter.)
Note that until recently, I was able to have this many tabs, and more, open (and several dozen loaded) without any noticeable performance loss.
If you go to the profile, you can click randomly in the "GeckoMain [default]" thread, and you will see lots of restyling stuff going on. Layout flushes all over. Something seems to be thrashing layout.
If you select "javascript only", half the time will disappear (but there's still 3.5 seconds left!), but you'll be able to see that all the time is under TSTBrowser_updateTabbarOverflow in treestyletab-modules/browser.js:2318.
I just rebuilt the addon with the body of TSTBrowser_updateTabbarOverflow prepended with 'return;' and the hang went away. Performance seems quite good. (I know I broke something or other, but this appears to be a viable workaround for me for now, since everything I've tried to use so far has worked.)
same issue here with 340 tabs, it makes my FF nightly unusable.
Same issue, Firefox 53.0.3 on Gentoo with flags "dbus gmp-autoupdate jack jemalloc nsplugin system-icu". Downloaded TST master at 6cbab0f0e00d8d8b241c6f31d629f1fcd779d57e and applied @hotsphink 's workaround, and I too have much better performance now without any noticeable issues yet.
TST 0.19.x is already obsolete, so I close this.
Most helpful comment
I just rebuilt the addon with the body of TSTBrowser_updateTabbarOverflow prepended with 'return;' and the hang went away. Performance seems quite good. (I know I broke something or other, but this appears to be a viable workaround for me for now, since everything I've tried to use so far has worked.)