Njs: external objects are not really JS objects

[drsm]

$ ./njs
interactive njscript 0.2.1

v.<Tab> -> the properties and prototype methods of v.
type console.help() for more information

>> typeof console
undefined
>> typeof console.log
undefined

Currently objects created externally (request from nginx module, for example) are implemented in a completely different way from native JS objects (NJS_EXTERNAL value type). It means that in every JS function a special branch is needed to support externals.

Instead external objects should be implemented using generic primitives used for native JS objects.
As a result external objects will be indistinguishable from native objects and will be supported by all JS functions.

[drsm]

Hi @xeioex!
I'm mostly interested in behavior of its member functions, and been faced this issue while trying to implement something like this:

var logger;
function helper(args) {
  logger('some message');
}
function handler(req, res) {
  logger = req.error.bind(req);
  ...
}

Is it no-go, and i should use a closure here?

[xeioex]

Is it no-go, and i should use a closure here?

Yes, currently it is not possible. Using closures can be a way.

[xeioex]

Partly fixed within #20.

[alveko]

Do I understand it correctly that this issue makes things like JSON.stringify() not work for the objects constructed in externally in C-code (like the request object, for example)?

The issue is currently labeled as "in progress". Is it in progress? What's the reasonable timeline for the fix here? Is it weeks, months? Thanks!

[drsm]

@alveko
Hi!
Take a look at #192.

[alveko]

@drsm thanks! Ok, so JSON.stringify() is supported now.

The second part of the question remains. Do you guys plan to make the external object indistinguishable from the native JS objects as per this issue? Is the in-progress state accurate, and if yes, when do you think it may be delivered? I'm not asking you to commit to any dates, of course, but any kind of heads up on the timeline is much appreciated.

[xeioex]

@alveko

Do you guys plan to make the external object indistinguishable from the native JS objects as per this issue?

Yes, we do.

Is the in-progress state accurate, and if yes, when do you think it may be delivered?

currently it is on pause. I plan to do this in 3-month timeframe.

[xeioex]

@drsm @hongzhidao

Take a look at the following patch (beta): https://gist.github.com/112de4cc2de34a9dbdccbc6bbbcd23c3

Now external definitions are converted to native JS object primitives. External queries like in r.headersOut are implemented using mechanism similar to exotic slots from the spec.

Definition example:

  {                                                                                                                          
          .flags = NJS_EXTERN_OBJECT,                                                                                            
          .name.string = njs_str("headersOut"),    // may also be a symbol                                                                              
          .writable = 1,                                                                                                         
          .configurable = 1,                                                                                                     
          .enumerable = 1,                                                                                                       
          .u.object = {                               
                 // may implement [[Get]] (and [[Set]], [[Delete]] if writable or configurable is true)                                                       
              .prop_handler = ngx_http_js_ext_header_out,  // njs_prop_handler_t                                                                      
              .keys = ngx_http_js_ext_keys_header_out,                                                                           
          }                                                                                                                      
      },                                                                                                                         

Result

>> console.toString()
'[object Console]'
>> console[0]='a'; console[1]='b'; console.length =2; console.__proto__ = Array.prototype; console.join('|')
'a|b'
>> Object.getOwnPropertyNames(console)
[
 'log',
 '0',
 '1',
 'length',
 'dump',
 'time',
 'timeEnd'
]
>> JSON.stringify(console)
'{"0":"a","1":"b","length":2}'
>> console
Console {
 log: [Function: native],
 0: 'a',
 1: 'b',
 length: 2,
 dump: [Function: native],
 time: [Function: native],
 timeEnd: [Function: native]
}

[hongzhidao]

@xeioex

1. 
   > Now external definitions are converted to native JS object primitives. External queries like in r.headersOut are implemented using mechanism similar to exotic slots from the spec.

I'm wondering when you plan to commit? I need to deep into code again, it changes a lot.

1. Now, what is the passing rate of test cases?

2. After this, what do you plan to continue in 0.4.0?

[xeioex]

@hongzhidao

I'm wondering when you plan to commit?

By the end of this week.

Now, what is the passing rate of test cases?

 zero. Because our external objects are not covered by test262.

After this, what do you plan to continue in 0.4.0?

Short term plan includes:

1. https://github.com/nginx/njs/issues/266 (adding API to change/modify Cookie headers).
2. js_import directive. (To avoid js_include altogether.)

nginx.conf:

js_import lib1 from 'lib1.js'; 
js_import lib2 from 'lib2.js';

# main file will be implicitly generated as follows:
# import lib1 from  'lib1.js'; 
# import lib2 from  'lib2.js'; 

location /lib1 {
     js_content "lib1.handler";
}

location /lib2 {
     js_content "lib2.handler";
}

[hongzhidao]

@xeioex
External objects become much better than before. And take a look at two places.

1. adjust the order of the fields in njs_object_t. __proto__, hash, shared_hash, and slots.
   And it seems that __proto__ does not need comment.

1. What about renaming njs_exotic_slots_t as njs_exotic_slot_t.
   As I understand, slot is similar to prop/property. In spec, there are internal and exotic objects. They have corresponding properties and methods, they are internal and exotic.
   Now have njs_object_prop_t. It's good to introduce njs_exotic_slots_t.
   slots (array) is good a trick for storing nested external objects.
   But it seems every object has one slot, is it right?
   If yes, I prefer to use slot to slots.

typedef struct {
    njs_lvlhsh_t        hash;
    ...
    njs_exotic_keys_t   keys;
} njs_exotic_slot_t;


struct njs_object_s {
    njs_object_t                      *__proto__;

    /* A private hash of njs_object_prop_t. */
    njs_lvlhsh_t                      hash;

    /* A shared hash of njs_object_prop_t. */
    njs_lvlhsh_t                      shared_hash;

    /* TODO */
    njs_exotic_slot_t                 *slot;

    /* The type is used in constructor prototypes. */
    njs_value_type_t                  type:8;
    ...
};

Others look good.
Do you plan to continue refactoring this part?
I mean object, property, exotic. Is it totally finished?

 *
 *   TODO:
 *     Object.defineProperty([1,2], '1', {configurable:false})
 */

njs_int_t
njs_property_query(njs_vm_t *vm, njs_property_query_t *pq, njs_value_t *value,
    njs_value_t *key)
{

BTW, how do you get njs_exotic_slots_t? it's good.
Do you make it clear before start coding? Or it appears while coding and take inspiration?
It's hard for me to make things clear before start to do it.
I often use the second way, but it always needs to change code many times.

[hongzhidao]

@xeioex

js_import directive. (To avoid js_include altogether.)

Does it mean js_include will be deprecated?

It's common that we need to do the same things on many servers. Such as authentication.
How to use it?

Is anyone on it? If not, I'd like to try this.

[xeioex]

@hongzhidao

1. And it seems that __proto__ does not need comment.

agree.

1. But it seems every object has one slot, is it right?
   If yes, I prefer to use slot to slots.

https://tc39.es/ecma262/#sec-object-internal-methods-and-internal-slots

In the spec, the slots correspond to [[...]] methods. The exotic slots are used to redefine the standard behavior or different built-methods like ([[Get]], [[Set]], [[Delete]], [[Keys]], ..). I believe njs_exotic_slots_t (plural) is better because the are many methods that can be redefined (some will be added later).

In my definition [[Get]], [[Set]], [[Delete]] are merged to a single prop_handler. Flags specify in which context prop_handler can be invoked.

typedef struct {                                                                                                                 
//...                                                                              
    njs_prop_handler_t  prop_handler;                                                                                            
    unsigned            writable:1;                                                                                              
    unsigned            configurable:1;                                                                                          
    unsigned            enumerable:1;                                                                                            

    njs_exotic_keys_t   keys;   
// will be more later for other types                                                                                                 
} njs_exotic_slots_t; 

3.

/* ..
 *   TODO:
 *     Object.defineProperty([1,2], '1', {configurable:false})
 */

njs_int_t
njs_property_query(njs_vm_t *vm, njs_property_query_t *pq, njs_value_t *value,
    njs_value_t *key)
{

Good catch, the issue already fixed in https://github.com/nginx/njs/commit/58ebb38a063a3be7167da944b800f6c44cf90972. The comment will be removed.

Do you make it clear before start coding? Or it appears while coding and take inspiration?
It's hard for me to make things clear before start to do it.

I had some general idea and refined it in the process of development.

[xeioex]

@hongzhidao

Does it mean js_include will be deprecated?

most probably yes, but far into the future.

It's common that we need to do the same things on many servers. Such as authentication.
How to use it?

yes. that is why splitting your logic into different modules helps. Now, you have to merge all of your code, probably for different, unrelated locations into one js_include file.

Is anyone on it? If not, I'd like to try this.

not yet, I plan to do this after this issue.

[xeioex]

@alveko

njs 0.4.0 is released: http://nginx.org/en/docs/njs/changes.html#njs0.4.0