Assemblyscript: Benchmark performance question

[ncave]

Not really an issue, more of a quick question. I'm probably doing something wrong, just wondering what am I doing wrong.

Testing performance with one of the samples n-body shows that running the same code on TypeScript-generated JavaScript is almost twice as fast than on webassembly generated by AssemblyScript:

Steps: 20_000_000, increment: 0.01

    JavaScript: Elapsed: 3.0 sec
AssemblyScript: Elapsed: 5.6 sec

Env: Node.js 14.4.0, Win10, AssemblyScript 0.10.0

Test repo: https://github.com/ncave/webasm-bench

Test steps: npm install && npm test

P.S. I know n-body's not a real benchmark, sorry about the clickbait.

[MaxGraey]

try -O3 --noAssert instead --optimize here:
https://github.com/ncave/webasm-bench/blob/master/package.json#L8

[ncave]

Thanks @MaxGraey , that brought it down substantially, still slower than JavaScript but not by much:

Steps: 20_000_000, increment: 0.01

    JavaScript: Elapsed: 3.0 sec
AssemblyScript: Elapsed: 3.8 sec

[MaxGraey]

It really hard outperform JS in this benchmark due to AS use ARC/GC in single thread while JS use highly optimized concurrent M&S GC in separate thread. But you could turn off full runtime and use just simple bump allocator without ARC just add --runtime none. And performance should be the same as JS version

[MaxGraey]

So in general memory model of AS little bit slower than JS one. But much more predictable. See this picture for example:
https://github.com/nischayv/as-benchmarks/issues/3#issuecomment-623159721

[MaxGraey]

Also I see you don't use warmup before actual measurements for benchmark. For wasm is pretty important due to all browser's runtimes use multi-tiered compilation

[ncave]

You're absolutely right, @MaxGraey, using -O3 --noAssert --runtime none brought it down substantially.

Steps: 20_000_000, increment: 0.01

    JavaScript: Elapsed: 3.0 sec
AssemblyScript: Elapsed: 2.0 sec

[MaxGraey]

Also you could compare this bench results with Rust compiled to wasm:
https://github.com/AssemblyScript/examples/tree/master/n-body#benchmark

[ncave]

@MaxGraey Adding a warm-up didn't change anything, but the compiler flags most certainly did. Thanks for the detailed explanation of the allocator impact.

[ncave]

@MaxGraey Since you mentioned it, here are the Rust-webasm results (without --runtime none for AS):

Steps: 20_000_000, increment: 0.01
    JavaScript: Elapsed: 2.932 sec, Energy: -0.16903264767107579
AssemblyScript: Elapsed: 3.509 sec, Energy: -0.16893859207630157
Rust to webasm: Elapsed: 1.700 sec, Energy: -0.16903264767107579

Thanks again for your help, I appreciate it!

[MaxGraey]

I see you use f32 instead f64:
https://github.com/ncave/webasm-bench/blob/master/assembly/nbody_as.ts#L4

I guess better compare when all version use f64 / number

[MaxGraey]

That's by the way pretty interesting! When type of float equal to f32 instead f64 on latest Chrome and FF it actually slower!

You could play in this fiddle: https://webassembly.studio/?f=yp7hc6vnct

For type float = f32 on Chrome 83.0.4103.97:

dt: 2007.0299999788404 ms

For type float = f64 on Chrome 83.0.4103.97:

dt: 1735.0549999973737 ms

So f64 math faster on 15% than f32. Same for FF 77.

[MaxGraey]

I found why this actually happened. General code use <float>Math.sqrt(distanceSq) and when type float eqal to f32 that code translates to f32(f64.sqrt(distanceSq)) instead f32.sqrt(distanceSq) that's why it slowdown compare to f64. So if you want using f32 instead f64 better use Mathf.sqrt(distanceSq) and even better and more generic intrinsic - sqrt<float>(distanceSq) version. After that changes you got expected results for f32: https://webassembly.studio/?f=rjeiyydfel

dt: 1594.7750000050291 ms

[ncave]

@MaxGraey Thanks, switching to f64 fixed the precision.

Steps: 20_000_000, increment: 0.01

    JavaScript: Elapsed: 2.9 sec, Energy: -0.16903264767107579
AssemblyScript: Elapsed: 3.4 sec, Energy: -0.16903264767107579
AS--no-runtime: Elapsed: 1.7 sec, Energy: -0.16903264767107579
Rust to webasm: Elapsed: 1.7 sec, Energy: -0.16903264767107579

For some reason Chrome 83 is quite slow for me in this benchmark, about 2x slower than FF 76.
But somehow Node.js is much closer to FF perf.

[MaxGraey]

btw in future with scalar replacement optimization we could avoid alloc / dealloc for such scenario at all and performance for full runtime will be the same as AS--no-runtime and Rust