V: Lack of performance compared to C

[phykos]

OS: linux, "Manjaro ARM Linux"
Processor: 4 cpus, 64bit, little endian, BCM2835
CC version: cc (GCC) 10.2.0
V version: 0.1.29 d89986d

I ran a couple of benchmarks against C, the most relevant one probably is the fibonacci one.
V code:

fn fib(n u64) u64 {
    if n <= 1 { return 1 }
    return fib(n - 1) + fib(n - 2)
}

fn main() {
    println(fib(46))
}

C code:

#include <stdio.h>
unsigned long fib(unsigned long n) {
    if (n <= 1) return 1;
    return fib(n - 1) + fib(n - 2);
}

int main() { printf("%lu\n", fib(46)); }

What did you expect to see?
As said, the performance is within 3% of C's, but running the C code it takes ~15 seconds, and the V one is about 23.
Compilation: with GCC 10.2.0 it takes 0.3 seconds to compile the C code, but the V one takes almost half a minute and outputs 300 lines of warnings (-Ofast -pipe -pedantic -Wall -Wextra)

[medvednikov]

Can you print the build commands you used for both languages?

[phykos]

Can you print the build commands you used for both languages?

v fib.v -o fib.v.c
And for compiling both C files: gcc -Ofast -pipe -Wall -Wextra -pipe -pedantic

[medvednikov]

Both compiled programs take 4.2 seconds on my machine, which makes sense, since the V program's actual code (fn fib and main) compiles to the same C code. So you did something wrong with your measurements.

The compilation is indeed slower because tens of thousands of lines of vlib are still compiled with each program, we are working on this feature atm.

[medvednikov]

As for the warnings, the plan is to fix all of them.

[phykos]

Both compiled programs take 4.2 seconds on my machine, which makes sense, since the V program's actual code (fn fib and main) compiles to the same C code. So you did something wrong with your measurements.

The compilation is indeed slower because tens of thousands of lines of vlib are still compiled with each program, we are working on this feature atm.

I used zsh's time for doing this, these are the results using a function I wrote for the benchmark of my Ruby code:

time(4, false) { `./fib.c.out`} # run 4 times and do not avarage the result
# => [15.091374726001959, 15.089311199000804, 15.061973489999218, 15.096957727997506]
time(4, false) { `./fib.v.out`}
# => [23.84025869600009, 23.84107347400277, 23.838993475997995, 23.849020206002024]

[medvednikov]

This is physically impossible, since the code is literally the same. You can verify it by looking at main__main and main__fib.

[phykos]

This is physically impossible, since the code is literally the same. You can verify it by looking at main__main and main__fib.

I notice a bit of difference in the produced assembly, I am not that good in asm (aarch64) but V's looks much bloated comparated to C's

also try it yourself on a RPI, perhaps on different machines the behavior may be different, but on an RPi which is usually slower than a regular machine the difference is "intensified"

[medvednikov]

No, the resulting assembly is the same for the functions that are actually called: main and fib.

Can you create a zsh script with the commands you used for testing and paste it here?

I don't have an RPI, but I'm 100% confident, that the performance must be the same, just like on my machine.

[phykos]

No, the resulting assembly is the same for the functions that are actually called: main and fib.

Can you create a zsh script with the commands you used for testing and paste it here?

I don't have an RPI, but I'm 100% confident, that the performance must be the same, just like on my machine.

# assuming Ruby YARV and GCC are installed

# write to files

echo \
'#include <stdio.h>
unsigned long fib(unsigned long n) {
    if (n <= 1) return 1;
    return fib(n - 1) + fib(n - 2);
}

int main() { printf("%lu\\n", fib(46)); }' > fib.c

echo \
'fn fib(n u64) u64 {
    if n <= 1 { return 1 }
    return fib(n - 1) + fib(n - 2)
}

fn main() { println(fib(46)) }' > fib.v
echo \
'def time(times, avarage = true, &block)
  res = []
  times.times do
    t = Process.clock_gettime(Process::CLOCK_MONOTONIC)
    yield(block)
    res << Process.clock_gettime(Process::CLOCK_MONOTONIC) - t
  end
  avarage ? res.sum.fdiv(res.size) : res
end

resv = time(ARGV[0].to_i) { `./fib.v.out > resv.txt` }
resc = time(ARGV[0].to_i) { `./fib.c.out > resc.txt` }
puts("C: #{resc}\\nV: #{resv}") ' > timer.rb

# build files
v fib.v -o fib.v.c
gcc -Wall -Wextra -pedantic -pipe -O3 fib.c -o fib.c.out
gcc -Wall -Wextra -pedantic -pipe -O3 fib.v.c -o fib.v.out

# benchmark
ruby timer.rb 4 # 4 here means run 4 times and return the avarage

[nerk]

@phykos What happens if you swap these two lines? Make it execute fib.c.out first. I bet now the V version is 'faster'. Just a thought...

resv = time(ARGV[0].to_i) { `./fib.v.out > resv.txt` }
resc = time(ARGV[0].to_i) { `./fib.c.out > resc.txt` }

[phykos]

@phykos What happens if you swap these two lines? Make it execute fib.c.out first. I bet now the V version is 'faster'. Just a thought...

resv = time(ARGV[0].to_i) { `./fib.v.out > resv.txt` }
resc = time(ARGV[0].to_i) { `./fib.c.out > resc.txt` }

I don't think nothing else happens since t is redefined at each iteration

[nerk]

Variable t is not the problem. It might be that the Ruby Interpreter needs to do additional stuff when calling your time function for the first time in order to invoke the an executable or other things.Am 20.10.2020 18:59 schrieb Giulio Phykos [e-mail]:

@phykos What happens if you swap these two lines? Make it execute fib.c.out first. I bet now the V version is 'faster'. Just a thought...
resv = time(ARGV[0].to_i) { ./fib.v.out > resv.txt }
resc = time(ARGV[0].to_i) { ./fib.c.out > resc.txt }

I don't think nothing else happens since t is redefined at each iteration

—You are receiving this because you commented.Reply to this email directly, view it on GitHub, or unsubscribe.

[medvednikov]

A good example of over-engineering a simple problem :)

Why not just

cc -o a fib.c && time ./a
cc -o b fib.v.c && time ./b

[phykos]

A good example of over-engineering a simple problem :)

Why not just

cc -o a fib.c && time ./a
cc -o b fib.v.c && time ./b

force of habit you know...

[nerk]

@medvednikov C code generated from fib.v is even running about 8% faster than the original C version. I tried multiple times and the results are stable. (Not tested on Raspi).

[medvednikov]

Yeah, like I said, it's physically impossible to be slower, so definitely an issue with benchmarking. Thanks for confirming.

[phykos]

Guess I fucked up my Ruby then...

[nerk]

@phykos "Who measures measures rubbish!" Very easy to fall into that trap. Happens all the time.