Neon: Summing all numbers in an array is slow

[thehappycoder]

I started playing with neon and implemented a simple function that sums all numbers in an array. As a result, the javascript version is much faster for long arrays.

Will be grateful for any tips on how to speed this up. Just want to understand if it's me or neon.

$ neon build --release
$ node run.js 1000000
js: 25.620ms
js: 17.464ms
js: 14.848ms
js: 13.521ms
js: 13.906ms
js: 15.009ms
js: 13.585ms
js: 14.062ms
js: 16.498ms
js: 18.017ms
rust sum: 124.384ms
rust sum: 115.881ms
rust sum: 111.521ms
rust sum: 125.356ms
rust sum: 111.817ms
rust sum: 106.789ms
rust sum: 115.219ms
rust sum: 115.264ms
rust sum: 126.260ms
rust sum: 116.404ms
rust sum2: 104.958ms
rust sum2: 108.335ms
rust sum2: 111.118ms
rust sum2: 106.413ms
rust sum2: 102.614ms
rust sum2: 98.603ms
rust sum2: 102.530ms
rust sum2: 99.940ms
rust sum2: 102.802ms
rust sum2: 106.356ms

The code is here: https://github.com/thehappycoder/neon-playground

[theJian]

Looks like it's array.to_vec and array.get which makes it really slow.

[kjvalencik]

Hi! Thanks for reaching out! The reason this code is performing less than optimal is that it is crossing the FFI boundary very many times. JsArray is an inefficient way to provide data to a native module because grabbing each unique element requires crossing the boundary again. This is because getting an element from an Array could run arbitrary JS code via a getter.

This code is benchmarking the cost of crossing that boundary and it would also be a performance bottleneck in a pure C++ addon.

An alternative strategy is to provide the data in a more simple format. The Buffer type is excellent because it can be viewed as a Rust slice without any manipulation of the underlying data.

On the JS side, convert the data to a typed array:

const typedArray = new Uint32Array(array);

Then in Rust, access the array as a slice. Neon provides a helper to view a Buffer as a slice of u32 instead of the raw bytes.

fn sum_typed(mut cx: FunctionContext) -> JsResult<JsNumber> {
    // Typed arrays are views over buffers and can be treated as buffers
    let buf: Handle<JsBuffer> = cx.argument(0)?;

    // In this section, we will borrow the underlying data from the buffer. Zero copying occurs.
    let result: u64 = {
        let guard = cx.lock();
        let data = buf.borrow(&guard);

        // The underlying data is `&[u8]` but, we can view it as a `&[u32]`.
        let slice = data.as_slice::<u32>();

        // We can't use `.sum()` because we need to cast into a u64 for a higher max value
        slice.into_iter().fold(0, |sum, &n| sum + n as u64)
    };

    // This use case does not require more than 2^53, so this is safe.
    Ok(cx.number(result as f64))
}

In this example, the Rust code outperforms JS by a wide margin.

$ node run.js 10000000
10000000
js: 129.026ms
js: 129.036ms
js: 135.835ms
js: 132.317ms
js: 124.816ms
js: 126.216ms
js: 134.091ms
js: 130.642ms
js: 134.539ms
js: 131.126ms
rust sum: 1150.523ms
rust sum: 1114.698ms
rust sum: 1135.277ms
rust sum: 1122.754ms
rust sum: 1122.049ms
rust sum: 1108.155ms
rust sum: 1129.849ms
rust sum: 1635.216ms
rust sum: 1313.560ms
rust sum: 1129.232ms
rust sum2: 1060.804ms
rust sum2: 1025.344ms
rust sum2: 1370.575ms
rust sum2: 1273.353ms
rust sum2: 1159.650ms
rust sum2: 1236.882ms
rust sum2: 1558.932ms
rust sum2: 1073.337ms
rust sum2: 1065.538ms
rust sum2: 1024.357ms
js typed array: 170.628ms
js typed array: 144.921ms
js typed array: 149.442ms
js typed array: 155.474ms
js typed array: 149.290ms
js typed array: 144.837ms
js typed array: 152.824ms
js typed array: 146.975ms
js typed array: 144.235ms
js typed array: 147.698ms
rust typed array: 3.075ms
rust typed array: 2.966ms
rust typed array: 3.008ms
rust typed array: 4.067ms
rust typed array: 3.433ms
rust typed array: 3.036ms
rust typed array: 4.044ms
rust typed array: 3.117ms
rust typed array: 9.205ms
rust typed array: 5.147ms

A similar strategy can be used with JSON. Often JSON is received as a Buffer or String. Instead of deserializing it in javascript, you can pass it directly to Rust and deserialize it there.

diff --git a/native/src/lib.rs b/native/src/lib.rs
index c5d1326..e0efa4f 100644
--- a/native/src/lib.rs
+++ b/native/src/lib.rs
@@ -42,9 +42,24 @@ fn sum2(mut cx: FunctionContext) -> JsResult<JsNumber> {
     Ok(cx.number(result))
 }

+fn sum_typed(mut cx: FunctionContext) -> JsResult<JsNumber> {
+    let buf: Handle<JsBuffer> = cx.argument(0)?;
+
+    let result: u64 = {
+        let guard = cx.lock();
+        let data = buf.borrow(&guard);
+        let slice = data.as_slice::<u32>();
+
+        slice.into_iter().fold(0, |sum, &n| sum + n as u64)
+    };
+
+    Ok(cx.number(result as f64))
+}
+
 register_module!(mut m, {
     m.export_function("threadCount", thread_count)?;
     m.export_function("sum", sum)?;
     m.export_function("sum2", sum2)?;
+    m.export_function("sumTyped", sum_typed)?;
     Ok(())
 });
diff --git a/run.js b/run.js
index 1c47721..02704e0 100644
--- a/run.js
+++ b/run.js
@@ -1,3 +1,5 @@
+const assert = require('assert');
+
 const mod = require('.')

 const sum = (array) => {
@@ -16,22 +18,37 @@ for (let i = 0; i < process.argv[2]; i++) {
   array.push(i)
 }

+const typedArray = new Uint32Array(array);
+const expected = sum(array);
+
 console.log(array.length)

 for (let i = 0; i < 10; i++) {
   console.time('js')
-  sum(array)
+  assert.strictEqual(sum(array), expected);
   console.timeEnd('js')
 }

 for (let i = 0; i < 10; i++) {
   console.time('rust sum')
-  mod.sum(array)
+  assert.strictEqual(mod.sum(array), expected);
   console.timeEnd('rust sum')
 }

 for (let i = 0; i < 10; i++) {
   console.time('rust sum2')
-  mod.sum2(array)
+  assert.strictEqual(mod.sum2(array), expected);
   console.timeEnd('rust sum2')
-}
+}
+
+for (let i = 0; i < 10; i++) {
+  console.time('js typed array')
+  assert.strictEqual(sum(typedArray), expected);
+  console.timeEnd('js typed array')
+}
+
+for (let i = 0; i < 10; i++) {
+  console.time('rust typed array')
+  assert.strictEqual(mod.sumTyped(typedArray), expected);
+  console.timeEnd('rust typed array')
+}

[kjvalencik]

@thehappycoder Feel free to join me--and others in the community--in the slack workspace if you would like to discuss further.

https://rust-bindings-slackin.herokuapp.com/