V: -x64 fails by adding extra byte to strings, at least on print
V version:
V 0.1.27 e336628
OS:
alpine 3.11.3 on x86_64
What did you do?
v test-fixed
What did you expect to see?
All tests passed/skipped. This works perfectly on Ubuntu 19.10 on x86_64.
What did you see instead?
Note: The odd diamonds-with-question-marks at the end of one line and the beginning of several others below is a byte of 0x06.
FAIL [ 82/170] 2169.595 ms vlib/v/gen/x64/tests/x64_test.v
x.v: /home/jalon/.cache/vtests/x64/x.v
FAIL
============
expected: "hello from x64" len=14
============
found:"hello from x64" len=15
============
x.v: /home/jalon/.cache/vtests/x64/x.v
FAIL
============
expected: "start
hello from x64
123
end" len=28
============
found:"start
hello from x64
123
end
" len=33
============
x.v: /home/jalon/.cache/vtests/x64/x.v
FAIL
============
expected: "a == 1
b == 2
now a == 2
hello
hello
hello
hello
hello
===args===
foo:
a == 1
a == 2
foo:
a == 3" len=96
============
found:"a == 1
b == 2
now a == 2
hello
hello
hello
hello
hello
===args===
foo:
a == 1
a == 2
foo:
a == 3
" len=111
============
--------------------------------------------------------------------------------
106.957 ms <=== total time spent x64
ok, fail, skip, total = 0, 3, 0, 3
JalonSolov
All 12 comments
Update: It appears that this is not specific to Alpine. I'm hitting the same type of problem with -x64 on ubuntu 19.10 as well, and just when running examples/x64/hello_world.v
JalonSolov
on 9 May 2020
Hm, on Ubuntu 16.04 it works, strange:
0[09:02:37]delian@nemesis: /v/nv $ ./v version
V 0.1.27 7f69c2f
spytheman
on 10 May 2020
@JalonSolov can you please test if it is still broken for you?
spytheman
on 10 May 2020
$ v -v
V 0.1.27 0606d26
$ cat /etc/os-release
NAME="Alpine Linux"
ID=alpine
VERSION_ID=3.11.3
PRETTY_NAME="Alpine Linux v3.11"
HOME_URL="https://alpinelinux.org/"
BUG_REPORT_URL="https://bugs.alpinelinux.org/"
$ v -v
V 0.1.27 0606d26
$ v vlib/v/gen/x64/tests/x64_test.v
x.v: /home/jalon/.cache/vtests/x64/x.v
FAIL
============
expected: "hello from x64" len=14
============
found:"hello from x64" len=15
============
x.v: /home/jalon/.cache/vtests/x64/x.v
FAIL
============
expected: "start
hello from x64
123
end" len=28
============
found:"start
hello from x64
123
end
" len=33
============
x.v: /home/jalon/.cache/vtests/x64/x.v
FAIL
============
expected: "a == 1
b == 2
now a == 2
hello
hello
hello
hello
hello
===args===
foo:
a == 1
a == 2
foo:
a == 3" len=96
============
found:"a == 1
b == 2
now a == 2
hello
hello
hello
hello
hello
===args===
foo:
a == 1
a == 2
foo:
a == 3
" len=111
============
----------------------------------------------------------------------------------------------------------------------------
14.413 ms <=== total time spent x64
ok, fail, skip, total = 0, 3, 0, 3
$ v -x64 run examples/x64/hello_world.v
warning: import cycle detected between the following modules:
code_start_pos = 78
000078 e8 83 ff ff ff call fn main
println:
00007d 55 push rbp
00007e 48 89 e5 mov rbp,rsp
000081 48 81 ec 20 00 00 00 sub rsp,0x20
000088 c7 45 fc 00 00 00 00 mov DWORD [rbp-0x4],0 (Aallocate var `s`)
00008f 89 7d fc mov from reg
000092 c9 leave
000093 c3 ret
main:
000094 55 push rbp
000095 48 89 e5 mov rbp,rsp
000098 b8 01 00 00 00 mov eax, 1
00009d bf 01 00 00 00 mov edi, 1
0000a2 48 be 00 00 00 00 00 00 00 00 mov64 rsi, 0
0000ac ba 0a 00 00 00 mov edx, 10
0000b1 0f 05 syscall
x64.stmt(): bad node: v.ast.ForInStmt
0000b3 bf 00 00 00 00 mov edi, 0
0000b8 b8 3c 00 00 00 mov eax, 60
0000bd 0f 05 syscall
0000bf c3 ret
x64 elf binary has been successfully generated
x64 test
$
$
To show the last one a little more clearly...
$ examples/x64/hello_world
x64 test
$ examples/x64/hello_world >foo.txt
$ hexdump -C foo.txt
00000000 78 36 34 20 74 65 73 74 0a 06 |x64 test..|
0000000a
$
Notice the final 2 bytes are 0a 06. That 06 is the problem.
And, to prove it's the backend rather than the OS, here is the same thing on Ubuntu:
$ v -v
V 0.1.27 0606d26
$ cat /etc/os-release
NAME="Ubuntu"
VERSION="19.10 (Eoan Ermine)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 19.10"
VERSION_ID="19.10"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=eoan
UBUNTU_CODENAME=eoan
$ v -x64 examples/x64/hello_world.v
warning: import cycle detected between the following modules:
code_start_pos = 78
000078 e8 83 ff ff ff call fn main
println:
00007d 55 push rbp
00007e 48 89 e5 mov rbp,rsp
000081 48 81 ec 20 00 00 00 sub rsp,0x20
000088 c7 45 fc 00 00 00 00 mov DWORD [rbp-0x4],0 (Aallocate var `s`)
00008f 89 7d fc mov from reg
000092 c9 leave
000093 c3 ret
main:
000094 55 push rbp
000095 48 89 e5 mov rbp,rsp
000098 b8 01 00 00 00 mov eax, 1
00009d bf 01 00 00 00 mov edi, 1
0000a2 48 be 00 00 00 00 00 00 00 00 mov64 rsi, 0
0000ac ba 0a 00 00 00 mov edx, 10
0000b1 0f 05 syscall
x64.stmt(): bad node: v.ast.ForInStmt
0000b3 bf 00 00 00 00 mov edi, 0
0000b8 b8 3c 00 00 00 mov eax, 60
0000bd 0f 05 syscall
0000bf c3 ret
x64 elf binary has been successfully generated
$ examples/x64/hello_world >foo.txt
$ hexdump -C foo.txt
00000000 78 36 34 20 74 65 73 74 0a 06 |x64 test..|
0000000a
$
Notice the same final 2 bytes.
JalonSolov
on 10 May 2020
Ok... this doesn't make sense to me, but perhaps someone else knows why this is here. In vlib/v/gen/x64/elf.v, these are lines 88 - 94:
// Strings table
// Loop thru all strings and set the right addresses
for i, s in g.strings {
g.write64_at(segment_start + g.buf.len, int(g.str_pos[i]))
g.write_string(s)
g.write8(6)
}
line 93, g.write8(6) would certainly explain why I'm now seeing an extra byte of 0x06 at the end of every printed string, since it's being explicitly added at the end of every string. :-\
I checked back, and it appears this line has been there since the first commit of elf.v on 12/30/2019.
Why has it seemed to be working all along, but now appears to be failing? That I cannot tell you.
However, according to the ELF spec, all strings should have a NUL character after them... so the line should logically be changed to
g.write8(0)
Making that change in my playpen makes the 'hello_world' work correctly, but the x64_test.v still fails (only for different reasons).
$ v -x64 x64_test.v
/home/jalon/git/v/vlib/os/os_windows.c.v:87:34: error: redefinition of function `os.ls`
85 | }
86 |
87 | pub fn ls(path string) ?[]string {
| ^
88 | mut find_file_data := Win32finddata{}
89 | mut dir_files := []string{}
$
... why is it trying to compile the os_windows.c.v file when I'm on Linux? I don't know.
JalonSolov
on 11 May 2020
Yeah, not sure what 0x6 was doing there.
x_test.v is for tests, V runs the test framework, use a different file name that doesn't end with _test.v
medvednikov
on 11 May 2020
Replacing the 6 with a 0 only works for programs with a single string (like hello_world.v).
If I run v test-fixed, I get this:
FAIL [ 84/172] 1876.144 ms vlib/v/gen/x64/tests/x64_test.v
x.v: /home/jalon/.cache/vtests/x64/x.v
OK [1/3] 148.372 ms testing file: hello.vv
x.v: /home/jalon/.cache/vtests/x64/x.v
FAIL
============
expected: "start
hello from x64
123
end" len=28
============
found:"start" len=5
============
x.v: /home/jalon/.cache/vtests/x64/x.v
FAIL
============
expected: "a == 1
b == 2
now a == 2
hello
hello
hello
hello
hello
===args===
foo:
a == 1
a == 2
foo:
a == 3" len=96
============
found:"a == 1" len=6
============
--------------------------------------------------------------------------------
398.344 ms <=== total time spent x64
ok, fail, skip, total = 1, 2, 0, 3
I'll keep looking...
JalonSolov
on 11 May 2020
Ok, next piece of info... string lengths are off by one. Changing the 6 to a 0 made it work _visually_ with one line, but when I redirected the output to a file, it is now outputting the 0 instead of the 6.
So, now to figure out where it is setting string lengths, and why it is counting one more than is actually part of the string.
JalonSolov
on 11 May 2020
Ok... line 423 of vlib/v/gen/x64/gen.v is adding the extra character the string length. Maybe leftover from the commented out "+ \n" on line 416? At any rate, changing the line to
g.mov(.edx, s.len) // len
Makes the x64 tests pass, 'hello_world' output no longer has a trailing NUL byte, etc. on both systems I've been trying.
I'll be happy to make a PR if you like. Someone else should probably test to make sure it actually works for them as well, though. :-\
JalonSolov
on 11 May 2020
I can confirm that even on Ubuntu 16.04 where tests do pass:
0[11:56:26]delian@nemesis: /v/nv $ examples/x64/hello_world > x
0[11:56:32]delian@nemesis: /v/nv $ xxd x
00000000: 7836 3420 7465 7374 0a06 x64 test..
0[11:56:35]delian@nemesis: /v/nv $ hexdump -C x
00000000 78 36 34 20 74 65 73 74 0a 06 |x64 test..|
0000000a
0[11:56:54]delian@nemesis: /v/nv $
I am not sure why they do not fail, but the 06 byte is definitely there.
spytheman
on 11 May 2020
I found the reason why they do not fail on ubuntu 16.04:
in vlib/v/gen/x64/tests/x64_test.v, there is an explicit removing of the 0x06 byte :
var found := res.output.trim_space().trim('\n').replace('\r\n', '\n')
// remove ACK char TODO fix this in x64
buf := [byte(0x06)]
ack := string(buf)
found = found.replace(ack, '')
found = found.trim_space()
@JalonSolov yes, a PR would be definitely appreciated. Thanks for finding the bug!
spytheman
on 11 May 2020
Related issues
choleraehyq
路
3Comments
radare
路
3Comments
elimisteve
路
3Comments
lobotony
路
3Comments
jtkirkpatrick
路
3Comments
Most helpful comment
Ok... line 423 of vlib/v/gen/x64/gen.v is adding the extra character the string length. Maybe leftover from the commented out "+ \n" on line 416? At any rate, changing the line to
Makes the x64 tests pass, 'hello_world' output no longer has a trailing NUL byte, etc. on both systems I've been trying.
I'll be happy to make a PR if you like. Someone else should probably test to make sure it actually works for them as well, though. :-\