Neon: typed array APIs

[dherman]

Create object types for all the typed array types:

• [ ] neon::js::binary::Int8Array
• [ ] neon::js::binary::Uint8Array
• [ ] neon::js::binary::Uint8ClampedArray
• [ ] neon::js::binary::Int16Array
• [ ] neon::js::binary::Uint16Array
• [ ] neon::js::binary::Int32Array
• [ ] neon::js::binary::Uint32Array
• [ ] neon::js::binary::Float32Array
• [ ] neon::js::binary::Float64Array

[bennycode]

Hey @dherman, thanks for providing Neon!

At Wire we see high potential in it. However, we are dependent on the Uint8Array type. Do you have an outlook when it will be possible to map this type between Rust and JavaScript?

[dherman]

@bennyn That's great to hear you have use cases for Neon! I'm happy to prioritize adding at least Uint8Array. In the meantime would the JsBuffer type work for you? (AIUI the Buffer class is a subclass of Uint8Array.)

[ffflorian]

@dherman
Thank you for the hint. By using the JsBuffer and afterwards converting it to an Uint8Array in JavaScript, we managed to make it work! :tada:

By using neon, our benchmark just got __61.5 times faster__!

---

Here is a code example with gen_keypair() from sodiumoxide:

lib.rs

fn crypto_sign_keypair(call: Call) -> JsResult<JsObject> {
    let scope = call.scope;
    let (kp_public, kp_secret) = ed25519::gen_keypair();

    let mut kp_public_buf = try!(JsBuffer::new(scope, 32));
    utils::buf_copy_from_slice(&kp_public.0, &mut kp_public_buf);

    let mut kp_secret_buf = try!(JsBuffer::new(scope, 64));
    utils::buf_copy_from_slice(&kp_secret.0, &mut kp_secret_buf);

    let js_object: Handle<JsObject> = JsObject::new(scope);
    try!(js_object.set("keyType", JsString::new(scope, "ed25519").unwrap()));
    try!(js_object.set("publicKeyBuffer", kp_public_buf));
    try!(js_object.set("privateKeyBuffer", kp_secret_buf));

    Ok(js_object)
}

index.js

const crypto_sign_keypair = () => {
    const { publicKeyBuffer, privateKeyBuffer, keyType } = sodiumneon.crypto_sign_keypair();

    return {
        publicKey: new Uint8Array(publicKeyBuffer),
        privateKey: new Uint8Array(privateKeyBuffer),
        keyType
    };
};

[dherman]

This deserves an RFC, and we should also think about whether we want to offer a way to get the different typed views on an ArrayBuffer directly within Rust. Right now you can hack your way to doing this with some unsafe Rust skulduggery:

buffer.grab(|mut slice| {
    let len = slice.len();
    let raw = slice.as_mut_ptr();
    let mut slice: CMutSlice<f32> = unsafe { CMutSlice::new(mem::transmute(raw), len / 4) };
    slice[0] = 1.8;
    slice[1] = 13.4;
});

I think it's important to be able to do this with an ArrayBuffer, without having to require the JS to pass you a view of the right type. Maybe the best way to do this is for the Lock implementation not to directly provide a CMutSlice<u8> but instead a custom type that gives you access to CMutSlice<u8>, CMutSlice<i8>, CMutSlice<u16>, CMutSlice<i16>, etc.

[DiThi]

@dherman Is there a way to do that from a buffer that was created in JS and passed as argument? At least for reading.

[kjvalencik]

@ffflorian Excellent work! Isn't that benchmark a little unfair? If I'm reading correctly, you are comparing an ECMAScripten compilation to pure JS against a native implementation.

How does it compare against something that uses native bindings like sodium-native?

[ffflorian]

@kjvalencik

How does it compare against something that uses native bindings like sodium-native?

I don't know, back then in February, sodium-native was not usable for us because of several reasons. And since we wanted to switch from pure JS to a Rust implementation, I simply compared the results after switching.