Nativescript: Way to call function in UI Thread

[farfromrefug]

Thanks to some work on nativescript-opencv and some great help from @mbektchiev i found out that you can have JS code run on another thread. In my case it as calling JS from captureOutput:didOutputSampleBuffer

It was working great and i could have {N} js code run and apply OpenCV processing in the camera thread. Now i then needed, from there, to run some code on the UI thread (set a Image.imageSource).
This was obviously not working as UIImage.image needs to be set in UI Thread.

I found a simple way to do that by using a very simple "forward" worker like so:

(global as any).onmessage = function(msg) {
    if (isIOS) {
        const dict = valueFromPointerNumber(NSDictionary, msg.data.value.dictionaryPtr) as NSDictionary<string, any>;
        const data = dict.objectForKey('data');
        // decrease reference count
        (dict as any).release();
        sendMessageBack(data);
    }
};

function sendMessageBack(data) {
    if (isIOS) {
        const nativeDict = NSDictionary.dictionaryWithObjectForKey(data, 'data');
        const message = {
            value: { dictionaryPtr: interop.handleof(nativeDict).toNumber() }
        };
        // increase reference count to account for `dictionaryPtr`
        (nativeDict as any).retain();
        (global as any).postMessage(message);
    }
}

So from the background thread, postMessage to worker, then you receive immediatly back.
Now, assuming you ve set worker.onmessage then that onmessage will be on UI thread.
It is really fast and it even works with JS or Native objects!!!

So working but a bit tricky to integrate. Also the worker message actually goes from backgroundthread=>workerthread=>uithread, when i could directly do
backgroundthread=>uithread

So here is the FR, using the same technique {N} could add a a way to ensure call of any fonction in the UI thread.
Something like

runInUIThread(function(){console.log('test);});

behind the runInUIThread could you the technique behind worker postMessage ?

---

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

[mbektchiev]

I found an easier way to do it in iOS:

var operation = new NSOperation();
operation.completionBlock = () => console.log("hi from main thread!");
NSOperationQueue.mainQueue.addOperation(operation);

[NathanaelA]

I will warn you; mixing JS threads can cause some weird stability issues in both iOS and Android, and will probably lead to some serious "future" instability. One plugin I did just a couple months back on iOS would do its work and call the finish callback from the secondary work thread and it took me a while to figure that out.

Without me wrapping it back to the main thread in Objc _before_ I called back into the JS engine -- it caused the main thread in the JS engine state to get corrupted and later on crash. JS code ran fine in the callback; but things would then just stop working randomly and/or crash in the app later on.

In addition I've seen this same issue with OpenGL stuff on android (which has to run in a separate thread); You just don't do OpenGL on Android using NativeScript JS code...

Still it is some cool techniques guys! But I would be very leary of any code that was causing one JS engines code to run in another ones thread; that just screams at me that something is screwed up internally, and you will find it out in the very near future... :-)

Btw, If I recall correctly, settimeouts/setinterval will stop working when the threading gets messed up... So you might try adding a setTimeout/setinterval in both threads and see if they are still working. Not a guarantee you haven't screwed up the threads; but I seem to recall that was one thing that always failed right away after the threading got screwed up and was how I finally figured out it was the new plugin breaking the app... It might have been new settimeouts not already running, but that was several months ago...

[farfromrefug]

I found an easier way to do it in iOS:

var operation = new NSOperation();
operation.completionBlock = () => console.log("hi from main thread!");
NSOperationQueue.mainQueue.addOperation(operation);

I actually tested your code and it does not work.
running this from background thread:

const operation = new NSOperation();
operation.completionBlock = () => console.log('hi from main thread!', NSOperationQueue.currentQueue === NSOperationQueue.mainQueue);
NSOperationQueue.mainQueue.addOperation(operation);

prints

CONSOLE LOG [native code]: 'hi from main thread!' false

[farfromrefug]

Actually it's normal for it not to work. Your example use the completionBlock which will be called in the thread running the sample text.
THis works:

    NSOperationQueue.mainQueue.addOperationWithBlock(() => console.log('hi from main thread!', NSOperationQueue.currentQueue === NSOperationQueue.mainQueue));

[mbektchiev]

You're right @farfromrefug! Thanks for pointing it out, I obviously didn't pay enough attention to that note about the completion block and didn't check where it got executed...

[mbektchiev]

@NathanaelA Thank you for sharing your experience and advices!

I will warn you; mixing JS threads can cause some weird stability issues in both iOS and Android, and will probably lead to some serious "future" instability. One plugin I did just a couple months back on iOS would do its work and call the finish callback from the secondary work thread and it took me a while to figure that out.

Without me wrapping it back to the main thread in Objc _before_ I called back into the JS engine -- it caused the main thread in the JS engine state to get corrupted and later on crash. JS code ran fine in the callback; but things would then just stop working randomly and/or crash in the app later on.

Still it is some cool techniques guys! But I would be very leary of any code that was causing one JS engines code to run in another ones thread; that just screams at me that something is screwed up internally, and you will find it out in the very near future... :-)

The JavaScriptCore is actually thread-safe by design, so calling it from multiple threads is not something that can't be made to work correctly. In some cases though, there could still be instabilities caused by bugs in the iOS runtime. We're striving to resolve them and in the latest two or three releases of the runtime we have managed to resolve some long-standing and rarely occurring racing and locking issues.

Generally, I would not discourage people from running JS code in background threads, especially in cases like the one with OpenCV, where some computational logic is meant to happen in the background thread by the design of a library. Whether it's a good idea to pay the price for transitioning to JS, instead of implementing it in native code is arguable, but sometimes it may be worth it for the sake of simplicity and maintainability. Ideally, one should measure performance before deciding whether it's appropriate or not in a particular case.

Btw, If I recall correctly, settimeouts/setinterval will stop working when the threading gets messed up... So you might try adding a setTimeout/setinterval in both threads and see if they are still working. Not a guarantee you haven't screwed up the threads; but I seem to recall that was one thing that always failed right away after the threading got screwed up and was how I finally figured out it was the new plugin breaking the app... It might have been new settimeouts not already running, but that was several months ago...

If you are still able to reproduce that, we'd be glad to investigate the issue.

[NathanaelA]

@mbektchiev - Martin,

Well the case was this:

1. Main JS Thread -> Called into a pure OBJC framework library;
2. Objc framework created a new thread did all its work in background.
3. ObjC framework would then call JS delegate callback from the "new" work thread.
4. JS delegate would then do some stuff (and shortly crash.)

Replacing step 3/4 above with

1. ObjC framework called a simple ObjC wrapper delegate function I wrote
2. ObjC wrapper delegate would switch to main thread and then call JS Delegate
3. JS Delegate would do its stuff (and everything was great)

This code was written only a couple months back. So I can probably still duplicate this by reverting to an older copy of the plugin if you are really interested.

---

I probably should say the only reason I knew about other threads causing issues was over a year ago I wrote some custom Bluetooth drivers that interacted with NS, when I first wrote them; they didn't switch threads back to the main thread and would always cause it to crash the app in fun ways. Now all custom ObjC code I do; which creates and/or uses other threads -- always sends the response back to the UI thread and I've never had any crashing issues by following that simple rule. :-D Now if you have fixed issues in the run-times recently; then maybe those issues I was running into back then is now fixed. But I've seen thread crashes multiple times over the last couple years on iOS when I forgot to send the response back to UI thread and left it on whatever thread it was currently on...

[farfromrefug]

@NathanaelA thanks for the explanation. I think it would be good to get actual crash log. here it is working just fine with live js callback triggered for background thread at 30fps. Your issue might already be fixed , or due to long js processing, or even somerhing totally different

[mbektchiev]

The most recent improvements were discovered while implementing #1060 (see https://github.com/NativeScript/ios-runtime/pull/1060/commits). There were some rarely occurring and hard to reproduce racing conditions, which we managed to track down by enabling automatic triggering of asynchronous garbage collections on _each_ native call.

[NathanaelA]

Yeah, #1060 is way newer than my plugin work, so it is very possible you have fixed the issues I was running into. I'll see if I can find some time to grab the first version of the plugin and see if it works properly now... :grinning: