Nim: Undefined Behavior when using const/let tables (Nim v0.18.1)

[Drakoriss]

In new version of Nim I've got different results when trying to receive values from table:

import tables
const tableOfArray = {
    "one": [true, false, false],
    "two": [false, true, false],
    "three": [false, false, true]
}.toTable()
for i in 0..2:
    echo tableOfArray["two"][i]

Code above must print false true false to console, but instead it can return all values as false. Or fall into infinity loop. Or throw exception. Anything but proper result.
Same if I change const to let. But it works when I use var:

var tableOfArray = {
    "one": [true, false, false],
    "two": [false, true, false],
    "three": [false, false, true]
}.toTable()

When I use sequence instead of array it works too:
```nim
const tableOfArray = {
"one": @[true, false, false],
"two": @[false, true, false],
"three": @[false, false, true]
}.toTable()
````

[dom96]

Undefined behaviour is definitely not intended behaviour.

[Drakoriss]

@Yardanico What if I don't really care about order of output? I just need to iterate over such table but still got everything as false or even can catch infinity loop. But thanks for suggestion!

[BopoHDark]

Just note that behavior described above is true for MacOS and Linux, this code works as expected on the Windows platform.

[skilchen]

I consistently get the correct result (false, true, false) on Ubuntu 16.04 LTS using:

Nim Compiler Version 0.18.1 [Linux: amd64]
Copyright (c) 2006-2018 by Andreas Rumpf

git hash: cc5140d8b6ea8ca65ec9b74348cf23aec09d6723
active boot switches: -d:release

and

gcc version 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.9)

[Yardanico]

Did someone manage to reproduce it? I couldn't with clang on linux

[trustable-code]

Nim Compiler Version 0.18.0 [Linux: amd64]:
false
true
false
* stack smashing detected *: terminated
No stack traceback available
SIGABRT: Abnormal termination.

[Drakoriss]

Updated Nim today:

Nim Compiler Version 0.18.1 [MacOSX: amd64]
Copyright (c) 2006-2018 by Andreas Rumpf

git hash: cc5140d8b6ea8ca65ec9b74348cf23aec09d6723
active boot switches: -d:release

still exists for me (gotta interrupt execution because it's infinitely outputs true:

$ nim c -r main.nim
false
true
...
true^Ctrue
true
Traceback (most recent call last)
main.nim(10)             main
sysio.nim(430)           echoBinSafe
SIGINT: Interrupted by Ctrl-C.

[Araq]

I can reproduce it on OSX. And it's indeed a stack corruption so that the for loop variable gets overwritten producing an endless loop. Now why that stack corruption happens is still a mystery to me.

[Parashurama]

Using nim --passC: '-fsanitize=address' --passL:' -lasan' with clang/gcc on Linux AMD64
and running with
LD_PRELOAD=/usr/lib/libasan.so.4 ./test_table_undefined
I got more details on the stack corruption
Also it shows the same issue, with const, let or var versions.
Below is a crashing example which doesn't use tables at all. It seems to be related to returning an array value (transformed to passing array as arg in C backend.

var tableOfArray = @[
    [true, false, false],
    [false, true, false],
    [false, false, true]
]
proc crashingProc*[B](t: seq[B], index: Natural): B =
    discard
echo tableOfArray.crashingProc(0)

N_LIB_PRIVATE N_NIMCALL(void, crashingProc_hVH6m5qHJpAcKgG8WfZ9cKw)(tySequence_ijtzneBXRouS2n9a7Rvf7dg* t, NI index, tyArray_xSxqoPo7Fj4uIDcWiYyWDg Result) {
    chckNil((void*)Result);
    memset((void*)Result, 0, sizeof(Result)); //! ISSUE SEEMS to be here
}



md5-d9c438aa2bbfd171b8af5b5ec3308c14



=================================================================
==14713==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7fffca16fa73 at pc 0x7f8b5cf1239c bp 0x7fffca16f9d0 sp 0x7fffca16f178
WRITE of size 8 at 0x7fffca16fa73 thread T0
    #0 0x7f8b5cf1239b in __interceptor_memset /build/gcc/src/gcc/libsanitizer/asan/asan_interceptors.cc:471
    #1 0x556672c6d83a in crashingProc_hVH6m5qHJpAcKgG8WfZ9cKw /folder/path/nimcache/test_table_undefined.c:119
    #2 0x556672c6dca5 in NimMainModule /folder/path/nimcache/test_table_undefined.c:180
    #3 0x556672c6d9e4 in NimMainInner /folder/path/nimcache/test_table_undefined.c:154
    #4 0x556672c6dad1 in NimMain /folder/path/nimcache/test_table_undefined.c:162
    #5 0x556672c6db54 in main /folder/path/nimcache/test_table_undefined.c:169
    #6 0x7f8b5c5ccf49 in __libc_start_main (/usr/lib/libc.so.6+0x20f49)
    #7 0x556672c6d709 in _start (/folder/path/test_table_undefined+0x4709)

Address 0x7fffca16fa73 is located in stack of thread T0 at offset 99 in frame
    #0 0x556672c6db91 in NimMainModule /folder/path/nimcache/test_table_undefined.c:173

  This frame has 2 object(s):
    [32, 40) 'T1_'
    [96, 99) 'T2_' <== Memory access at offset 99 overflows this variable
HINT: this may be a false positive if your program uses some custom stack unwind mechanism or swapcontext
      (longjmp and C++ exceptions *are* supported)
SUMMARY: AddressSanitizer: stack-buffer-overflow /build/gcc/src/gcc/libsanitizer/asan/asan_interceptors.cc:471 in __interceptor_memset
Shadow bytes around the buggy address:
  0x100079425ef0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x100079425f00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x100079425f10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x100079425f20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x100079425f30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x100079425f40: 00 00 f1 f1 f1 f1 00 f2 f2 f2 f2 f2 f2 f2[03]f2
  0x100079425f50: f2 f2 00 00 00 00 00 00 00 00 00 00 00 00 f1 f1
  0x100079425f60: f1 f1 00 f2 f2 f2 00 00 00 00 00 00 00 00 00 00
  0x100079425f70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x100079425f80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x100079425f90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==14713==ABORTING

[Parashurama]

I think I found the issue:

In proc:

{.emit: """/*INCLUDESECTION*/
#include <stdio.h>
""".}
proc crashingProc*[B](t: seq[B], index: Natural): B =
    {.emit: ["""fprintf(stdout, "C:sizeof:%d\n", (int)sizeof(""", result, "));"].}
    echo "N:sizeof", sizeof(result)
    discard

which translate to

N_LIB_PRIVATE N_NIMCALL(void, crashingProc_32RzIjpmCbnI2n4KIEf20g)(tySequence_ijtzneBXRouS2n9a7Rvf7dg* t, NI index, tyArray_xSxqoPo7Fj4uIDcWiYyWDg Result) {
    tyArray_Re75IspeoxXy2oCZHwcRrA T1_;
    chckNil((void*)Result);
    memset((void*)Result, 0, sizeof(Result));
    fprintf(stdout, "C:sizeof:%d\n", (int)sizeof(Result));
    memset((void*)T1_, 0, sizeof(T1_));
    T1_[0] = copyString(((NimStringDesc*) &TM_Is5bcFTKJBXGUbrkjXBdPQ_5));
    T1_[1] = nimIntToStr(((NI)sizeof(tyArray_xSxqoPo7Fj4uIDcWiYyWDg)));
    echoBinSafe(T1_, 2);
}

Size of Result is 8
while sizeof(type of result) is 3 which is the expected result.
It's probably an issue of array pased as argument decaying into a pointer which has size 8 on x64 instead of mesuring the total bytes size of the array.

The solution seems to be using sizeof(type of result) for memset.

[Araq]

Yup, that's it.