Emscripten: How can I pass object from javascript back to javascript?

[Thaina]

Sorry for asking here. I'm not really use emscripten by myself but I'm using unity webgl. Which it said it using emscripten internally

Most of the things work fine so I try to experiment more and more things. But now I'm stuck with this problem. I want to pass javascript callback function to be kept in another language system (unity C# in this case) and then pass it back to another function

How could I got a pointer to that function object and return it. And how can I marshalling it back to original object in js? Was the Module object contains any function like that?

[kripken]

Runtime.addFunction() is a method in JS that gets a JS function and returns a C function pointer, see here.

[Thaina]

@kripken Thank you, that is useful info

But it a little bit off. What I want is I want to get the pointer of javascript object itself. So I can pass into c as pointer of something and can pass it back to javascript function

[kripken]

I'm not sure I understand. Perhaps a concrete example of code would help.

[Thaina]

@kripken Right sorry

What I mean is there are 2 javascript functions that need to pass object to each other

myModule = {
    listenDataChanged : function() {
         var cb = function() { }
         firebaseDBRef.on("value",cb);
         return cb;
    },
    stopListenDataChanged : function(cb) {
         firebaseDBRef.off("value",cb);
    }
}

What I need is to have the function object cb returned from listenDataChanged and it must be that cb that was added to on function without wrapping in anything. To send to off

So I think it should be something like pointer that let us marshall any js object between the language. In C environment it would call listenDataChanged and get that pointer to kept it as void* (without using it) but then when it want to stop it would call stopListenDataChanged with that pointer

Here I don't know how to send the exact same object from javascript back into javascript

[kripken]

Still not sure I fully understand - I'd need to know who calls each of those methods (regular js, or compiled code, etc.). But to take a guess, if you want to pass a reference to a JS object "through" compiled code - i.e. you want to call compiled code with a reference to a JS object, then compiled code calls regular JS again, and that JS method needs to get the original JS object - then the usual pattern is to create a mapping in JS:
var globalMapping = {}; var id = 0; // add to map globalMapping[id] = { .. some JS object .. }; // the id is just a number, so ok to call compiled code with it callCompiledCode(id); // this is called later from compiled code function calledFromCompiledCode(id) { var jsObject = globalMapping[id]; }

[Thaina]

@kripken listenDataChanged and stopListenDataChanged was made to be called on outside js

Suppose it C#. I wrap those 2 into extern function of C#

C# will call listenDataChanged and get a SOMETHING returned. I don't know what it will be. But it would represent the js object returned from listenDataChanged

Then whenever it want to stop listen. C# will pass that SOMETHING to stopListenDataChanged

When it reach into js code of stopListenDataChanged. That SOMETHING must be coverted back to the same object listenDataChanged was returned

To cache it into some mapping is last resort. I wish that the javascript object should have some way to convert itself into something like a pointer that I could call Pointer_to_jsobj like Pointer_stringify to mashal back and forth between system

But, well it seem like emscripten have nothing like that

Thank you very much

[Thaina]

@kripken Would you please consider this as feature request?

Something like a function Pointer_fromObject and Pointer_toObject that we could get unique integer of any object

[kripken]

I think that caching/mapping is the only good option here - that's what all the emscripten glue code does, for example, in the WebGL code there is a cache/map of GL ids (numbers) to WebGL texture objects, etc.

Basically, compiled code can only accept a number. And there isn't a way to convert a full JS object into a number, so it has to be a "handle" or "id" for the object, based on a map/cache.

We could add an emscripten API for this, to save a few lines of code. Perhaps Runtime.createObjectID and Runtime.revokeObjectID? (parallel to createObjectURL/revokeObjectURL in HTML, which is sort of similar)

[Thaina]

@kripken That would be great so we could have centralized unique id system. However, from those API how can we get the object back from the ID?

But still it a bit sad that we cannot have a way to get the existing unique id, the pointer of the object, instead of trying to wrap things around in more layer

[kripken]

Basically all that is needed is code like this:
var IDHandler = { ids: [], // get an integer ID for a JS object. this keeps a reference to it, preventing GC'ing createObjectId: function(obj) { var id = IDHandler.ids.length; IDHandler.ids[id] = obj; return id; }, // get a JS object from an integer ID getObjectFromID: function(id) { return IDHandler.ids[id]; }, // releases an object that has an ID. this allows it to be GD'd revokeObjectId: function(id) { IDHandler.ids[id] = null; } };
Let's see if other people think it's worth adding to the emscripten API.

[Thaina]

@kripken If we continuously use create and revoke then the ids would growth indefinitely right?

And should createObjectId return the same id for the same object?

[kripken]

Yeah, you can fine-tune it for those use cases (like a free list, if you want to do a lot of create/revoke, etc.). That's part of why we don't have a standard way in our API - it's easy to write, and different use cases need different things.

[Thaina]

@kripken I think the most common use case is trying to treat it as the void* pointer. It's mean same object should have same ID instead of create new ID. And memory should be reused if able

I think what I asking is the common scenario. There is no point to get difference id for the same object. And also no point to leave array with null slot to keep growing

Actually I wish it should be a system that keep reference counting. It should imitate the reference type and garbage collector

Well, it would be easier if we could have some way to marshalling javascript object into void* pointer directly (And also marshalling void* pointer into javascript object)

[kripken]

There are cases where you won't query the same object twice, though - so you don't need to make sure the same JS object gets the same id. If you do need to ensure that, you need something like an extra weak map to maintain that mapping, which adds overhead (but is also pretty easy to add). I guess what I'm saying is there are multiple use cases here, and we haven't seen much demand for adding one or more of them to emscripten so far, but as I said earlier, if more people would find this useful, we definitely should add it.

Do you mean something additional by "mashalling" here? More than just mapping the object to an id and vice versa like in the code fragment I provided?

[Thaina]

@kripken The "marshalling" I mean is actually mean avoiding mapping. It mean instead of try to map id with something. It should be direct conversion from object to the pointer of that object itself

Well, because I actually use C#, not the emscripten directly, most of functionality in C# was able to map the functionality of javascript. So it felt weird that everything that should be direct conversion need a round trip between system

[kripken]

I see. In some cases marshalling like that is possible, if it's just JSON data. But it would have high overhead probably. In other cases it can't work though, like if the JS Object has a reference to another object or function (for the same reasons JSON.stringify() can't handle them).

I don't know much about C#, so yeah, maybe this is an odd area where the two don't connect well.

[Faless]

Basically all that is needed is code like this:

var IDHandler = {
ids: [],
// get an integer ID for a JS object. this keeps a reference to it, preventing GC'ing
createObjectId: function(obj) {
var id = IDHandler.ids.length;
IDHandler.ids[id] = obj;
return id;
},
// get a JS object from an integer ID
getObjectFromID: function(id) {
return IDHandler.ids[id];
},
// releases an object that has an ID. this allows it to be GD'd
revokeObjectId: function(id) {
IDHandler.ids[id] = null;
}
};

Let's see if other people think it's worth adding to the emscripten API.

@kripken
That would be great! I stumbled upon this problem myself while trying to get a more fine tuned support for websockets in Godot Engine (see https://github.com/LudiDorici/godot-lws).

It would be great to have an easy API for object mapping when interacting with Javascript.
In the meantime I'll code it into the module myself with the example you gave.

[sparkprime]

I hit this today. FWIW, Lua has incorporated such a centralized registry for its C / Lua bindings: https://www.lua.org/manual/2.4/node19.html

[Thaina]

Hello. Are there any movement around this feature or any alternative?

[liquidaty]

Am guessing this won't work but, any chance one solution is as simple as using closure to preclude the need to pass the object at all? The question really turns on whether emscripten will respect a function's closure when passed / invoked by reference:

function outer() {
  var my_context_data = { counter: 0 };
  function inner() {
    console.log(my_context_data.counter++);
  }
  var my_func_ptr = Runtime.addFunction(inner); // does this work?
  // ... pass my_func_ptr to my C function and invoke it from C
}

[kripken]

@liquidaty yes, that should work. All emscripten does is call that function, it isn't aware if it closes over anything, so that would all just work normally.

[liquidaty]

👍 thank you

[stale[bot]]

This issue has been automatically marked as stale because there has been no activity in the past year. It will be closed automatically if no further activity occurs in the next 30 days. Feel free to re-open at any time if this issue is still relevant.

[Thaina]

Has came back and it seem like there was a feature specific for this task already progressing

https://github.com/WebAssembly/reference-types/blob/master/proposals/reference-types/Overview.md

Did emscripten planned anything about this feature?

[kripken]

I have done some experiments with reference types, but so far have not found a way to benefit from them in emscripten, either on side or speed.

[Thaina]

@kripken I just think that it was a straightforward method to do interop with js. Isn't this a way we could passing reference from js and use reference as parameter to another function call? And it seem we could even call member of reference type directly. So the benefit is that we could use any third party that written in js alongside wasm isn't it?

[kripken]

You can do those things, but you can also do them without reference types - you can call JS that handles the reference for you. In practice, letting JS do more seems to be smaller in code size in my experiments actually. That's why we don't have a plan to change things atm.

But it's definitely worth trying more things here, I just experimented with replacing simple things like WebGL calls - saving the context object inside wasm, and calling directly from there, etc.

[Thaina]

without reference types - you can call JS that handles the reference for you

The most tedious parts is maintaining the system to handling reference in js itself, we need to do it ourselves, everyone need to do it instead of centralized system and it then not compatible, and all of those is just duplicate of what the reference type feature was doing for us

Still a result in larger code size is regrettably hard decision. But for the application that large in average, like games, should be worth the cost because of the require to use many third party written in js

Currently now I'm very stucked to use firebase in my game. It's almost like I need to rewritten whole firebase to wrap the js firebase library, plus the system to maintaining reference that compatible with it