Njs: NJS performance vs Chrome V8 & Firefox

[xbb123]

Hello all,
This is in no way a comprehensive performance test. But I just wanted to show how NJS performs compared to Firefox (cold start) and Chrome (cold start)
Linear search (array.includes) & binary search for difference array sizes. All lists are sorted. Nginx JS module timing does not include VM creation time, it’s simply how long it takes for Javascript includes to return.

Chrome V8 (Chrome 75.0.3770.100):

Testing performance diff for 864 elements
VM53:39 JS Includes: 0.0004742914331339199
VM53:40 Binary Find: 0.0005485737978535121

badASNRAM1000:
Testing performance diff for 1000 elements
VM35:39 JS Includes: 0.0005819572611686687
VM35:40 Binary Find: 0.000597040518428322

badASNRAM2000:
Testing performance diff for 2000 elements
VM35:39 JS Includes: 0.0006530127825462825
VM35:40 Binary Find: 0.0005997027038064568

badASNRAM2000:
Testing performance diff for 3000 elements
VM35:40 JS Includes: 0.0007289954804322399
VM35:41 Binary Find: 0.0006195246703701701

Nginx Javascript Module (0.3.2):

Testing performance diff for 864 elements
JS Includes: 0.003342809388437308
Binary Find: 0.005385637348037885

Testing performance diff for 1000 elements
JS Includes: 0.003771374694647219
Binary Find: 0.0055713489807288469

Testing performance diff for 2000 elements
JS Includes: 0.007814174083227382
Binary Find: 0.005828488164454793

Testing performance diff for 3000 elements
JS Includes: 0.010571277553177812
Binary Find: 0.006385623062527678

Firefox (67.0.4):

Testing performance diff for 864 elements
JS Includes: 0.002390261262875734
Binary Find: 0.0029657412712555307

Testing performance diff for 1000 elements
JS Includes: 0.0023732933018800486
Binary Find: 0.003145328472371149

Testing performance diff for 2000 elements
JS Includes: 0.003987537159844501
Binary Find: 0.0030871255431054196

Testing performance diff for 3000 elements
JS Includes: 0.005244880168064825
Binary Find: 0.003329855801523445

[lexborisov]

@xbb123

Thank you for the performance test.
Could you tell us how you test it and, please, show the test code?

[xbb123]

@lexborisov
This is the link to the NJS script: https://gist.github.com/xbb123/ceb2c764c99bfd36238fe50319d3ae08
On Chrome & Firefox, you can also use performance.now() for timing.

[drsm]

The fun thing I've noticed is that the functional code runs faster than imperative in njs:

var n = 5;

console.time();
var data = [7, 51, 213, 378, 395, 409, 527, 542, 566, 660, 664, 669, 696, 886, 958, 970, 1033, 1086, 1193, 1199, 1226, 1253, 1301, 1326, 1349, 1514, 1770, 1786, 1887, 1900, 1992, 2045, 2099, 2106, 2136, 2186, 2187, 2255, 2282, 2342, 2532, 2714, 2736, 2813, 2952, 2958, 3048, 3053, 3133, 3164, 3170, 3241, 3286, 3442, 3527, 3635, 3743, 3755, 3762, 3783, 3819, 3861, 3884, 3885, 3982, 3987, 4018, 4162, 4182, 4205, 4258, 4328, 4420, 4426, 4450, 4502, 4569, 4604, 4668, 4684, 4898, 4906, 4907, 4921, 4934, 4955, 5110, 5138, 5180, 5182, 5265, 5279, 5346, 5367, 5407, 5438, 5498, 5578, 5601, 5749, 5785, 5882, 5888, 5965, 5997, 6034, 6262, 6385, 6398, 6421, 6528, 6655, 6733, 6803, 7000, 7006, 7023, 7068, 7115, 7219, 7234, 7241, 7242, 7350, 7493, 7531, 7562, 7589, 7652, 7680, 7685, 7687, 7774, 7827, 7971, 7998, 8042, 8126, 8134, 8252, 8341, 8432, 8606, 8612, 8751, 8805, 8866, 9008, 9211, 9257, 9360, 9370, 9434, 9461, 9608, 9716, 9726, 9728, 9790, 9838, 9971, 10059, 10122, 10269, 10277, 10391, 10457, 10563, 10605, 10641, 10644, 10717, 10851, 10954, 11001, 11073, 11116, 11212, 11230, 11241, 11516, 11642, 11689, 11710, 11771, 11790, 11835, 12037, 12053, 12085, 12113, 12119, 12134, 12187, 12212, 12219, 12270, 12362, 12401, 12430, 12493, 12781, 12893, 12917, 12956, 13004, 13023, 13119, 13123, 13135, 13221, 13274, 13314, 13389, 13423, 13467, 13768, 13798, 13951, 14031, 14129, 14303, 14317, 14363, 14502, 14535, 14561, 14580, 14668, 14760, 14888, 15035, 15101, 15146, 15219, 15246, 15311, 15365, 15390, 15423, 15505, 15598, 15721, 15853, 15856, 16020, 16102, 16134, 16161, 16267, 16306, 16307, 16359, 16383, 16397, 16444, 16628, 16633, 16662, 16798, 16808, 16823, 16855, 16951, 17021, 17071, 17147, 17204, 17206, 17418, 17557, 17618, 17727, 18019, 18156, 18176, 18221, 18305, 18307, 18400, 18470, 18491, 18492, 18568, 18639, 18787, 18805, 18810, 18887, 19008, 19169, 19178, 19248, 19390, 19412, 19453, 19456, 19466, 19491, 19539, 19600, 19643, 19688, 19725, 19735, 19821, 19831, 19999, 20033, 20077, 20129, 20223, 20323, 20389, 20422, 20624, 20693, 20760, 20787, 20802, 20880, 20980, 21032, 21092, 21156, 21233, 21248, 21412, 21493, 21648, 21672, 21692, 21819, 21889, 22140, 22157, 22224, 22251, 22292, 22420, 22450, 22544, 22554, 22628, 22640, 22653, 22696, 22738, 22754, 22956, 23109, 23202, 23614, 23767, 23822, 23951, 24020, 24023, 24053, 24336, 24457, 24631, 24749, 24883, 24956, 25109, 25295, 25350, 25375, 25451, 25613, 25935, 25973, 25983, 26001, 26108, 26124, 26163, 26312, 26341, 26381, 26419, 26499, 26523, 26543, 26669, 26805, 26859, 26869, 26923, 26966, 26997, 27052, 27073, 27090, 27163, 27231, 27430, 27567, 27590, 27594, 27612, 27622, 27629, 27794, 27877, 28074, 28116, 28322, 28328, 28358, 28431, 28498, 28663, 28766, 28806, 28863, 28864, 28875, 28885, 28973, 28974, 28991, 29102, 29276, 29304, 29311, 29510, 29634, 29912, 29972, 30022, 30104, 30136, 30266, 30299, 30398, 30418, 30576, 30577, 30595, 30762, 30802, 30805, 30823, 30867, 31002, 31041, 31079, 31197, 31302, 31304, 31378, 31379, 31423, 31499, 31542, 31601, 31634, 31699, 31702, 31753, 31772, 31776, 31849, 31887, 31933, 32039, 32060, 32107, 32118, 32173, 32237, 32245, 32266, 32447, 32463, 32544, 32550, 32557, 32664, 33020, 33142, 33172, 33253, 33386, 33458, 33554, 33562, 33594, 33639, 33660, 33754, 33849, 33921, 34023, 34063, 34238, 34438, 34585, 34620, 34638, 34732, 34831, 34886, 34927, 35060, 35065, 35066, 35220, 35284, 35338, 35506, 35596, 35685, 35719, 35761, 35856, 35900, 36084, 36197, 36257, 36283, 36394, 36404, 36495, 36496, 36612, 36699, 36706, 36754, 36761, 36896, 36951, 36997, 37035, 37057, 37127, 37175, 37593, 37693, 37722, 37839, 37875, 37988, 38011, 38039, 38181, 38237, 38314, 38319, 38345, 38506, 38628, 38691, 38700, 38736, 38867, 38941, 38985, 39060, 39081, 39101, 39109, 39135, 39201, 39304, 39390, 39520, 39546, 39562, 39618, 39916, 39945, 40057, 40067, 40139, 40161, 40210, 40245, 40324, 40346, 40427, 40603, 40604, 40780, 40862, 40940, 40992, 41073, 41223, 41232, 41327, 41510, 41700, 41882, 41901, 42134, 42139, 42213, 42263, 42265, 42284, 42369, 42398, 42510, 42576, 42587, 42596, 
42622, 42651, 42684, 42776, 42880, 42981, 43329, 43419, 43454, 43551, 43555, 43580, 43604, 43616, 43667, 43786, 43837, 43854, 43871, 43924, 44040, 44115, 44187, 44246, 44290, 44367, 44387, 44447, 44497, 44543, 44567, 44600, 44605, 44756, 45022, 45135, 45183, 45377, 45389, 45411, 45661, 45726, 45755, 45937, 45956, 46036, 46148, 46271, 46337, 46353, 46496, 46540, 46598, 46860, 46893, 46920, 46975, 47022, 47259, 47349, 47381, 47383, 47443, 47459, 47535, 47576, 47642, 47837, 47853, 47930, 48091, 48143, 48153, 48168, 48205, 48223, 48315, 48397, 48406, 48639, 48664, 48759, 48921, 49083, 49106, 49114, 49122, 49141, 49158, 49308, 49322, 49453, 49465, 49471, 49590, 49631, 49637, 49657, 49659, 49805, 49812, 49902, 49924, 49926, 50126, 50134, 50158, 50211, 50235, 50287, 50381, 50434, 50521, 50600, 50616, 50774, 50879, 50920, 50944, 51004, 51075, 51131, 51194, 51223, 51259, 51319, 51387, 51411, 51456, 51517, 51623, 51629, 51632, 51647, 51738, 51759, 51822, 51844, 51903, 51932, 51951, 52106, 52170, 52357, 52374, 52460, 52509, 52616, 52639, 52661, 52773, 52786, 52834, 52854, 52899, 52909, 53089, 53117, 53147, 53190, 53327, 53359, 53383, 53405, 53553, 53582, 53690, 53855, 53914, 54013, 54032, 54271, 54276, 54386, 54397, 54472, 54482, 54549, 54715, 54720, 54761, 54826, 55003, 55105, 55202, 55402, 55527, 55559, 55561, 55590, 55680, 55684, 55766, 55896, 55947, 56209, 56243, 56323, 56403, 56404, 56470, 56493, 56560, 56577, 56613, 56678, 56704, 56789, 57131, 57226, 57482, 57513, 57520, 57523, 57564, 57608, 57655, 57692, 57764, 57789, 57945, 57989, 58072, 58278, 58325, 58331, 58364, 58516, 58547, 58767, 58903, 58953, 59032, 59101, 59302, 59350, 59381, 59420, 59475, 59626, 59817, 60103, 60327, 60335, 60364, 60551, 60595, 60624, 60745, 60781, 60868, 60913, 60949, 61029, 61132, 61184, 61283, 61480, 61517, 61714, 61851, 61873, 62026, 62108, 62152, 62254, 62333, 62387, 62496, 62529, 62598, 62637, 62644, 62654, 62708, 62754, 62780, 62881, 62907, 62969, 62976, 63161, 63225, 63229, 63337, 63404, 63522, 63565, 63664, 63742, 63766, 63898, 63924, 64012, 64126, 64168, 64177, 64211, 64371, 64393, 64528, 64530, 64552, 64805, 64817, 64840, 64845, 64940, 64945, 64979, 64980, 65181, 65187, 65312, 65352, 65387, 65523, 65534, 65535, 65716, 65741, 65906, 65987, 66034, 66084, 66093, 66139, 66156, 66211, 66232, 66290, 66326, 66455, 66679, 66719, 66859, 66875, 66944, 67138, 67222, 67435, 67478, 67621, 67647, 67685, 67697, 67718, 67737, 67784, 67811, 67871, 67908, 67932, 68114, 68141, 68217, 68238, 68284, 68291, 68325, 68453, 68468, 68509, 68520, 68565, 68598, 68605, 68640, 68665, 68680, 68794, 68830, 68877, 69083, 69093, 69100, 69133, 69151, 69235, 69315, 69353, 69375, 69380, 69629, 69646, 69694, 69735, 69801, 69853, 69886, 69944];
var data = data.reduce((a, x) => a.concat(data.map((y) => x * y)), []);
console.timeEnd();

var len = data.length;


do {
    var s0 = 0;
    console.time();
    for (var i = 0; i < len; ++i) {
        s0 += data[i];
    }
    console.timeEnd();

    console.time();
    var s1 = data.reduce((a,x) => a + x, 0);
    console.timeEnd();

    console.log(len, s0, s1);

} while (--n);

$ node test.js 
default: 1959.921ms
default: 2.341ms
default: 17.096ms
1000000 1197881172576400 1197881172576400
default: 9.116ms
default: 17.297ms
1000000 1197881172576400 1197881172576400
default: 1.140ms
default: 28.554ms
1000000 1197881172576400 1197881172576400
default: 1.200ms
default: 18.878ms
1000000 1197881172576400 1197881172576400
default: 4.597ms
default: 18.682ms
1000000 1197881172576400 1197881172576400
$ build/njs test.js 
default: 2763.371131ms
default: 94.646942ms
default: 54.841624ms
1000000 1197881172576400 1197881172576400
default: 93.125163ms
default: 55.482375ms
1000000 1197881172576400 1197881172576400
default: 93.335319ms
default: 56.862563ms
1000000 1197881172576400 1197881172576400
default: 93.871401ms
default: 59.807322ms
1000000 1197881172576400 1197881172576400
default: 97.185907ms
default: 54.690428ms
1000000 1197881172576400 1197881172576400

[VBart]

Such microbenchmarks are very sensitive to compiler and compilation flags, but far from real world usage.

Default results for array includes() on my system are:

% node bench_array.js
Testing performance diff for 1000 elements
JS Includes: 0.00140110086496533

Testing performance diff for 2000 elements
JS Includes: 0.0026726503544477476

Testing performance diff for 3000 elements
JS Includes: 0.004015834679090506

% build/njs bench_array.js
Testing performance diff for 1000 elements
JS Includes: 0.0029594681535492176

Testing performance diff for 2000 elements
JS Includes: 0.005602561170820947

Testing performance diff for 3000 elements
JS Includes: 0.008117416717876896

As you can see, njs is about twice slower than node V8.

But this simple change and compilation with -flto flag enabled:

diff -r 63cd8e6679ba njs/njs_vm.c
--- a/njs/njs_vm.c      Tue Jun 25 14:44:55 2019 +0300
+++ b/njs/njs_vm.c      Wed Jun 26 00:04:08 2019 +0300
@@ -1754,7 +1754,7 @@ njs_vmcode_strict_not_equal(njs_vm_t *vm
 }


-nxt_noinline nxt_bool_t
+nxt_bool_t
 njs_values_strict_equal(const njs_value_t *val1, const njs_value_t *val2)
 {
     size_t        size, length1, length2;

Produces the following results for njs:

% build/njs bench_array.js
Testing performance diff for 1000 elements
JS Includes: 0.00140110086496533

Testing performance diff for 2000 elements
JS Includes: 0.0025297278756002746

Testing performance diff for 3000 elements
JS Includes: 0.0036442627870750148

Which effectively makes performance of njs in array includes() operation equal to V8. So, the culprit is just a C-function call.

If you look carefully on what this particular test does, you may notice that for 3000 elements case it makes about 200 millions of iterations over array elements. Even the difference in one assembly instruction here can produce dramatic difference in final results.