Torch7: ../torch/File.lua:370 table index is nil

[honghuCode]

I use openface in linux working well,but when I use openface on windows platform,it occurs errors.

'''../torch/File.lua:370 table index is nil''
Here is the error info

[fengbin8606]

I have the same problem.

root@ubuntu:/home/fengbin/texture2# th test.lua -input_image 11.jpg -model_t7 model.t7 -cpu
/usr/local/bin/luajit: /usr/local/share/lua/5.1/torch/File.lua:361: table index is nil
stack traceback:
/usr/local/share/lua/5.1/torch/File.lua:361: in function 'readObject'
/usr/local/share/lua/5.1/nn/Module.lua:154: in function 'read'
/usr/local/share/lua/5.1/torch/File.lua:342: in function 'readObject'
/usr/local/share/lua/5.1/torch/File.lua:391: in function 'load'
test.lua:17: in main chunk
/usr/local/lib/luarocks/rocks/trepl/scm-1/bin/th:145: in main chunk

[honghuCode]

I guess it is a platform problem,you could try it on X64 platform.

[AdamMiltonBarker]

I am also getting this on Raspberry Pi 3 any solution guys ?

[lord-alfred]

I get same problem: https://github.com/jcjohnson/torch-rnn/issues/162
(move *.t7 files from Ubuntu x64 to Win x65)

[Zueuk]

Same problem on Windows x64
....\luarocks\systree/share/lua/5.1/torch\File.lua:370: table index is nil

Apparently,

Torch's binary serialization formats are incompatible between platforms (link)

which, if true, is kind of... lame unfortunate.

[albanD]

The problem of the binary serialisation AFAIK is that different platforms set different sizes to different types.
The torch tensor types match the system types so they may be different for different platforms, for windows for example, I'm not sure there is a difference between an IntTensor and a LongTensor as on windows, long tend to be the same as int.

As specified in the issue you linked, if you need a serialized model to be portable, you can use the "ascii" option for torch.save.

[Zueuk]

But woudln't this cause a loss of data anyway - when it tries to fit the loaded 64-bit longs into 32-bit Windows longs?

I don't know if it's a Lua issue, but I'd be perfectly ok to use a custom built Lua with correctly sized longs (or whatever other type(s) causing the problem) just for Torch.

[albanD]

If you're above the capacity of the 32-bit long yes.
It's not a lua issue, it's just that torch's LongTensor is a tensor of long. This means that if on your system, longs are 32-bit, torch's LongTensor will be 32-bit. You can find here the define for the LongTensor and its type being long here.

[Zueuk]

So, what is stopping us from
#define real int64_t
?

[albanD]

Some of the file reader logic needs to be changed as well, see this branch.
I did not tested these changes on a system where long are 32-bits (because I don't have one right now) but it may work.

[Zueuk]

Sounds promising! I'm going to try building it under Win64...

[Zueuk]

Ok, to actually compile it I had to fix the return type at torch/lib/TH/THFile.h:29. Also, the int64_t typedef was not defined somewhere, so I just changed the TH_LONG define to long long.

Now the next problem is cunn: I had to change variable types to __int64 in cunn/lib/THCUNN/generic/VolumetricDilatedMaxPooling.cu : lines 235 and 353 - after that it compiled successfully, but trying to require 'cunn' in th fails:

...\Torch\luarocks\systree/share/lua/5.1/trepl/init.lua:389: ...\Torch\luarocks\systree/share/lua/5.1/trepl/init.lua:389:
...\Torch\luarocks\systree/share/lua/5.1/nn\Linear.lua:25: bad argument #1 to 'size' (dimension 2 out of range of 1D tensor at ...\torch\pkg\torch\generic/Tensor.c:19

[albanD]

It is weird as the reset function from Linear should not be called when you require cunn...
Could you try to import cunn but in a plain luajit (or lua if you use lua) interpreter and not th so that you get the actual stacktrace of what called reset? This should give you more informations about what's going on there...

[Zueuk]

> require 'cunn'
...\Torch\.\luarocks\systree\share\lua\5.1\nn\Linear.lua:25: bad argument #1
to 'size' (dimension 2 out of range of 1D tensor at ...sources\torch\pkg\to
rch\generic/Tensor.c:19)
stack traceback:
        [C]: in function 'size'
        ...\Torch\.\luarocks\systree\share\lua\5.1\nn\Linear.lua:25: in function 'reset'
        ...\Torch\.\luarocks\systree\share\lua\5.1\nn\Linear.lua:12: in function '__init'
        ...\Torch\.\luarocks\systree\share\lua\5.1\torch\init.lua:91: in function <...\
Torch\.\luarocks\systree\share\lua\5.1\torch\init.lua:87>
        [C]: in function 'Linear'
        ...\Torch\.\luarocks\systree\share\lua\5.1\nn\test.lua:27: in main chunk
        [C]: in function 'require'
        ...\Torch\.\luarocks\systree\share\lua\5.1\nn\init.lua:199: in main chunk
        [C]: in function 'require'
        ...\Torch\.\luarocks\systree\share\lua\5.1\cunn\init.lua:4: in main chunk
        [C]: in function 'require'
        stdin:1: in main chunk
        [C]: at 0x013f3d1eb0

So it's in nn.Linear, __init() there actually calls reset()

_(update)_
It seems that in the int64_t build, this expression
torch.Tensor(1,2):size(2)
produces the same error as above.

[albanD]

You would need to change at least this function that now takes TH_LONG as input and not long anymore.
Don't you got compilation warning for this?

Actually you may also want to switch the Tensor structure to use TH_LONG because you could get a tensor that is too big for you to store.
As a general idea, this would end up changing most of the longs in TH (THC, THNN, THCNN also) and that is going to be a lot of work :/

[Zueuk]

Hmm, but why would anybody want to use 64-bit longs to store tensor size? Sounds like a huge waste...

[albanD]

Some people have very specific use cases and may need this (see this PR for example)

[Zueuk]

Interesting... But still, that fix seems to be for intermediate calculations.

If it turns out to be really necessary though, then the Windows version is currently kind of broken... I wonder if a trivial search-replace long -> TH_LONG would work :)