Models: It is not compatible with Python3

[sicotronic]

When running the bazel tests (MacOS, anaconda python3.4 environment), fails with the following ouput.
Checking the error messages in the log files, I found that are all related to python3 incompatibilities. (Like TypeErrors between strings and bytes, and call to functions like iteritems() from a dict)

>>>bazel test --linkopt=-headerpad_max_install_names     syntaxnet/... util/utf8/...
INFO: Found 65 targets and 12 test targets...
FAIL: //syntaxnet:lexicon_builder_test (see /private/var/tmp/_bazel_200171/ce9fef2f4e6c2ee1eb4724114b2dc017/syntaxnet/bazel-out/local-opt/testlogs/syntaxnet/lexicon_builder_test/test.log).
FAIL: //syntaxnet:parser_trainer_test (see /private/var/tmp/_bazel_200171/ce9fef2f4e6c2ee1eb4724114b2dc017/syntaxnet/bazel-out/local-opt/testlogs/syntaxnet/parser_trainer_test/test.log).
FAIL: //syntaxnet:reader_ops_test (see /private/var/tmp/_bazel_200171/ce9fef2f4e6c2ee1eb4724114b2dc017/syntaxnet/bazel-out/local-opt/testlogs/syntaxnet/reader_ops_test/test.log).
FAIL: //syntaxnet:beam_reader_ops_test (see /private/var/tmp/_bazel_200171/ce9fef2f4e6c2ee1eb4724114b2dc017/syntaxnet/bazel-out/local-opt/testlogs/syntaxnet/beam_reader_ops_test/test.log).
FAIL: //syntaxnet:graph_builder_test (see /private/var/tmp/_bazel_200171/ce9fef2f4e6c2ee1eb4724114b2dc017/syntaxnet/bazel-out/local-opt/testlogs/syntaxnet/graph_builder_test/test.log).
INFO: Elapsed time: 8.131s, Critical Path: 7.53s
//syntaxnet:arc_standard_transitions_test                       (cached) PASSED in 0.0s
//syntaxnet:parser_features_test                                (cached) PASSED in 0.1s
//syntaxnet:sentence_features_test                              (cached) PASSED in 0.1s
//syntaxnet:shared_store_test                                   (cached) PASSED in 1.7s
//syntaxnet:tagger_transitions_test                             (cached) PASSED in 0.1s
//syntaxnet:text_formats_test                                   (cached) PASSED in 4.6s
//util/utf8:unicodetext_unittest                                (cached) PASSED in 0.0s
//syntaxnet:beam_reader_ops_test                                         FAILED in 3.5s
  /private/var/tmp/_bazel_200171/ce9fef2f4e6c2ee1eb4724114b2dc017/syntaxnet/bazel-out/local-opt/testlogs/syntaxnet/beam_reader_ops_test/test.log
//syntaxnet:graph_builder_test                                           FAILED in 7.5s
  /private/var/tmp/_bazel_200171/ce9fef2f4e6c2ee1eb4724114b2dc017/syntaxnet/bazel-out/local-opt/testlogs/syntaxnet/graph_builder_test/test.log
//syntaxnet:lexicon_builder_test                                         FAILED in 1.2s
  /private/var/tmp/_bazel_200171/ce9fef2f4e6c2ee1eb4724114b2dc017/syntaxnet/bazel-out/local-opt/testlogs/syntaxnet/lexicon_builder_test/test.log
//syntaxnet:parser_trainer_test                                          FAILED in 1.8s
  /private/var/tmp/_bazel_200171/ce9fef2f4e6c2ee1eb4724114b2dc017/syntaxnet/bazel-out/local-opt/testlogs/syntaxnet/parser_trainer_test/test.log
//syntaxnet:reader_ops_test                                              FAILED in 2.6s
  /private/var/tmp/_bazel_200171/ce9fef2f4e6c2ee1eb4724114b2dc017/syntaxnet/bazel-out/local-opt/testlogs/syntaxnet/reader_ops_test/test.log

Executed 5 out of 12 tests: 7 tests pass and 5 fail locally.
There were tests whose specified size is too big. Use the --test_verbose_timeout_warnings command line option to see which ones these are.

[slavpetrov]

You will need to use Python 2.X. We'll add Python 3 support but it's not there yet.

[veggiedefender]

run it in a virtualenv or use docker

[DVLP]

@slavpetrov GPU tensorflow for Windows runs only on Python 3.X so this is a blocker and this ticket shouldn't be closed. I agree with this post https://news.ycombinator.com/item?id=10533075. Python 2.7 requirement seems to be backwards

[eyadsibai]

Will be there support for python 3

[DVLP]

Check out my fork. Just converted all PY files using 2to3 and made small modification of .bazelrc
https://github.com/dvlp/models/tree/master/syntaxnet

Now I have problems running bazel on windows, and problems with GPU on Ubuntu so can't test yet

Edit: Actually I think I missed the point. If you run syntaxnet with the latest Bazel then it automatically does 2to3 on the fly! What am I missing then?

[adrinjalali]

There are some places in syntaxnet's code where 2to3 is not sufficient. I gave it a try and now it passes all the tests on my py3.5 environment. My changes are available here: https://github.com/adrinjalali/models/tree/master/syntaxnet

I can also run the parse_eval and it gives the desired output. But when do import syntaxnet.load_parser_ops, here's whta I get:

$ python
Python 3.5.3 (default, Jan 19 2017, 14:11:04) 
[GCC 6.3.0 20170118] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import syntaxnet.load_parser_ops
*** Error in `python': free(): invalid pointer: 0x0000558ff215ec98 ***
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(+0x7908b)[0x7f1c056be08b]
/lib/x86_64-linux-gnu/libc.so.6(+0x82c3a)[0x7f1c056c7c3a]
/lib/x86_64-linux-gnu/libc.so.6(cfree+0x4c)[0x7f1c056cbd2c]
/home/adrin/Projects/tf-models/syntaxnet/.venv/lib/python3.5/site-packages/syntaxnet/parser_ops.so(+0x2ea73a)[0x7f1bec91a73a]
/lib64/ld-linux-x86-64.so.2(+0x109ca)[0x7f1c05c3a9ca]
/lib64/ld-linux-x86-64.so.2(+0x10adb)[0x7f1c05c3aadb]
/lib64/ld-linux-x86-64.so.2(+0x15a76)[0x7f1c05c3fa76]
/lib64/ld-linux-x86-64.so.2(+0x10874)[0x7f1c05c3a874]
/lib64/ld-linux-x86-64.so.2(+0x15059)[0x7f1c05c3f059]
/lib/x86_64-linux-gnu/libdl.so.2(+0xee9)[0x7f1c05441ee9]
/lib64/ld-linux-x86-64.so.2(+0x10874)[0x7f1c05c3a874]
/lib/x86_64-linux-gnu/libdl.so.2(+0x1581)[0x7f1c05442581]
/lib/x86_64-linux-gnu/libdl.so.2(dlopen+0x32)[0x7f1c05441f82]
/home/adrin/Projects/tf-models/syntaxnet/.venv/lib/python3.5/site-packages/tensorflow/python/_pywrap_tensorflow_internal.so(_ZN10tensorflow8internal11LoadLibraryEPKcPPv+0x1a)[0x7f1bf1caffca]
/home/adrin/Projects/tf-models/syntaxnet/.venv/lib/python3.5/site-packages/tensorflow/python/_pywrap_tensorflow_internal.so(+0x29c2257)[0x7f1bf1caf257]
/home/adrin/Projects/tf-models/syntaxnet/.venv/lib/python3.5/site-packages/tensorflow/python/_pywrap_tensorflow_internal.so(_ZN10tensorflow11LoadLibraryEPKcPPvPPKvPm+0x603)[0x7f1bf1b899e3]
/home/adrin/Projects/tf-models/syntaxnet/.venv/lib/python3.5/site-packages/tensorflow/python/_pywrap_tensorflow_internal.so(+0xee62c7)[0x7f1bf01d32c7]
/home/adrin/Projects/tf-models/syntaxnet/.venv/lib/python3.5/site-packages/tensorflow/python/_pywrap_tensorflow_internal.so(+0xcefaea)[0x7f1beffdcaea]
python(PyCFunction_Call+0x4f)[0x558ff145828f]
python(PyEval_EvalFrameEx+0x4939)[0x558ff140f1e9]
python(PyEval_EvalFrameEx+0x4b8f)[0x558ff140f43f]
python(+0x182d16)[0x558ff1413d16]
python(PyEval_EvalCode+0x1f)[0x558ff1414a1f]
python(+0x18d2e8)[0x558ff141e2e8]
python(PyCFunction_Call+0x106)[0x558ff1458346]
python(PyEval_EvalFrameEx+0x6fa1)[0x558ff1411851]
python(+0x182d16)[0x558ff1413d16]
python(PyEval_EvalFrameEx+0x5729)[0x558ff140ffd9]
python(PyEval_EvalFrameEx+0x4b8f)[0x558ff140f43f]
python(PyEval_EvalFrameEx+0x4b8f)[0x558ff140f43f]
python(PyEval_EvalFrameEx+0x4b8f)[0x558ff140f43f]
python(PyEval_EvalCodeEx+0x12f)[0x558ff1414b5f]
python(+0x1c91c3)[0x558ff145a1c3]
python(PyObject_Call+0x47)[0x558ff1496887]
python(_PyObject_CallMethodIdObjArgs+0x17b)[0x558ff149753b]
python(PyImport_ImportModuleLevelObject+0x589)[0x558ff13f8589]
python(+0x186c88)[0x558ff1417c88]
python(PyCFunction_Call+0x77)[0x558ff14582b7]
python(PyObject_Call+0x47)[0x558ff1496887]
python(PyEval_CallObjectWithKeywords+0x30)[0x558ff144b290]
python(PyEval_EvalFrameEx+0x3063)[0x558ff140d913]
python(+0x182d16)[0x558ff1413d16]
python(PyEval_EvalCode+0x1f)[0x558ff1414a1f]
python(+0x250b02)[0x558ff14e1b02]
python(PyRun_InteractiveOneObject+0x2b3)[0x558ff1341005]
python(PyRun_InteractiveLoopFlags+0xe8)[0x558ff13411ae]
python(PyRun_AnyFileExFlags+0x54)[0x558ff14e4824]
python(+0x112f55)[0x558ff13a3f55]
python(main+0xe1)[0x558ff13a4b31]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf1)[0x7f1c056653f1]
python(_start+0x2a)[0x558ff14ab2da]
======= Memory map: ========
558ff1291000-558ff167f000 r-xp 00000000 08:01 264540                     /home/adrin/Projects/tf-models/syntaxnet/.venv/bin/python3
558ff187e000-558ff1881000 r--p 003ed000 08:01 264540                     /home/adrin/Projects/tf-models/syntaxnet/.venv/bin/python3
558ff1881000-558ff1917000 rw-p 003f0000 08:01 264540                     /home/adrin/Projects/tf-models/syntaxnet/.venv/bin/python3
558ff1917000-558ff1948000 rw-p 00000000 00:00 0 
558ff1a3a000-558ff3041000 rw-p 00000000 00:00 0                          [heap]
7f1be8000000-7f1be8021000 rw-p 00000000 00:00 0 
7f1be8021000-7f1bec000000 ---p 00000000 00:00 0 
7f1bec630000-7f1becfbe000 r-xp 00000000 08:01 348288                     /home/adrin/Projects/tf-models/syntaxnet/.venv/lib/python3.5/site-packages/syntaxnet/parser_ops.so
7f1becfbe000-7f1bed1be000 ---p 0098e000 08:01 348288                     /home/adrin/Projects/tf-models/syntaxnet/.venv/lib/python3.5/site-packages/syntaxnet/parser_ops.so
7f1bed1be000-7f1bed1f3000 r--p 0098e000 08:01 348288                     /home/adrin/Projects/tf-models/syntaxnet/.venv/lib/python3.5/site-packages/syntaxnet/parser_ops.so
7f1bed1f3000-7f1bed1f8000 rw-p 009c3000 08:01 348288                     /home/adrin/Projects/tf-models/syntaxnet/.venv/lib/python3.5/site-packages/syntaxnet/parser_ops.so
7f1bed1f8000-7f1bed200000 rw-p 00000000 00:00 0 
7f1bed200000-7f1bed400000 rw-p 00000000 00:00 0 
7f1bed5ae000-7f1bed96e000 rw-p 00000000 00:00 0 
7f1bed96e000-7f1bed97b000 r-xp 00000000 08:01 9792                       /usr/lib/python3.5/lib-dynload/_json.cpython-35m-x86_64-linux-gnu.so
7f1bed97b000-7f1bedb7a000 ---p 0000d000 08:01 9792                       /usr/lib/python3.5/lib-dynload/_json.cpython-35m-x86_64-linux-gnu.so
7f1bedb7a000-7f1bedb7b000 r--p 0000c000 08:01 9792                       /usr/lib/python3.5/lib-dynload/_json.cpython-35m-x86_64-linux-gnu.so
7f1bedb7b000-7f1bedb7c000 rw-p 0000d000 08:01 9792                       /usr/lib/python3.5/lib-dynload/_json.cpython-35m-x86_64-linux-gnu.so
7f1bedb7c000-7f1bee23d000 rw-p 00000000 00:00 0 
7f1bee23d000-7f1bee23e000 r-xp 00000000 08:01 9048                       /usr/lib/python3.5/lib-dynload/_opcode.cpython-35m-x86_64-linux-gnu.so
7f1bee23e000-7f1bee43d000 ---p 00001000 08:01 9048                       /usr/lib/python3.5/lib-dynload/_opcode.cpython-35m-x86_64-linux-gnu.so
7f1bee43d000-7f1bee43e000 r--p 00000000 08:01 9048                       /usr/lib/python3.5/lib-dynload/_opcode.cpython-35m-x86_64-linux-gnu.so
7f1bee43e000-7f1bee43f000 rw-p 00001000 08:01 9048                       /usr/lib/python3.5/lib-dynload/_opcode.cpython-35m-x86_64-linux-gnu.so
7f1bee43f000-7f1bee440000 r-xp 00000000 08:01 348104                     /home/adrin/Projects/tf-models/syntaxnet/.venv/lib/python3.5/site-packages/google/protobuf/internal/_api_implementation.cpython-35m-x86_64-linux-gnu.so
7f1bee440000-7f1bee63f000 ---p 00001000 08:01 348104                     /home/adrin/Projects/tf-models/syntaxnet/.venv/lib/python3.5/site-packages/google/protobuf/internal/_api_implementation.cpython-35m-x86_64-linux-gnu.so
7f1bee63f000-7f1bee640000 rw-p 00000000 08:01 348104                     /home/adrin/Projects/tf-models/syntaxnet/.venv/lib/python3.5/site-packages/google/protobuf/internal/_api_implementation.cpython-35m-x86_64-linux-gnu.so
7f1bee640000-7f1bee800000 rw-p 00000000 00:00 0 
7f1bee800000-7f1beea00000 rw-p 00000000 00:00 0 
7f1beea06000-7f1beeb46000 rw-p 00000000 00:00 0 
7f1beeb46000-7f1beeb5c000 r-xp 00000000 08:01 2918                       /lib/x86_64-linux-gnu/libgcc_s.so.1
7f1beeb5c000-7f1beed5b000 ---p 00016000 08:01 2918                       /lib/x86_64-linux-gnu/libgcc_s.so.1
7f1beed5b000-7f1beed5c000 r--p 00015000 08:01 2918                       /lib/x86_64-linux-gnu/libgcc_s.so.1
7f1beed5c000-7f1beed5d000 rw-p 00016000 08:01 2918                       /lib/x86_64-linux-gnu/libgcc_s.so.1
7f1beed5d000-7f1beeed6000 r-xp 00000000 08:01 73384                      /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.22
7f1beeed6000-7f1bef0d5000 ---p 00179000 08:01 73384                      /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.22
7f1bef0d5000-7f1bef0df000 r--p 00178000 08:01 73384                      /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.22
7f1bef0df000-7f1bef0e1000 rw-p 00182000 08:01 73384                      /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.22
7f1bef0e1000-7f1bef0e5000 rw-p 00000000 00:00 0 
7f1bef0e5000-7f1bef0ec000 r-xp 00000000 08:01 23405                      /lib/x86_64-linux-gnu/librt-2.24.so
7f1bef0ec000-7f1bef2eb000 ---p 00007000 08:01 23405                      /lib/x86_64-linux-gnu/librt-2.24.so
7f1bef2eb000-7f1bef2ec000 r--p 00006000 08:01 23405                      /lib/x86_64-linux-gnu/librt-2.24.so
7f1bef2ec000-7f1bef2ed000 rw-p 00007000 08:01 23405                      /lib/x86_64-linux-gnu/librt-2.24.so
7f1bef2ed000-7f1bf28ff000 r-xp 00000000 08:01 333756                     /home/adrin/Projects/tf-models/syntaxnet/.venv/lib/python3.5/site-packages/tensorflow/python/_pywrap_tensorflow_internal.so
7f1bf28ff000-7f1bf2900000 ---p 03612000 08:01 333756                     /home/adrin/Projects/tf-models/syntaxnet/.venv/lib/python3.5/site-packages/tensorflow/python/_pywrap_tensorflow_internal.so
7f1bf2900000-7f1bf2a28000 r--p 03612000 08:01 333756                     /home/adrin/Projects/tf-models/syntaxnet/.venv/lib/python3.5/site-packages/tensorflow/python/_pywrap_tensorflow_internal.so
7f1bf2a28000-7f1bf2a30000 rw-p 0373a000 08:01 333756                     /home/adrin/Projects/tf-models/syntaxnet/.venv/lib/python3.5/site-packages/tensorflow/python/_pywrap_tensorflow_internal.so
7f1bf2a30000-7f1bf2a81000 rw-p 00000000 00:00 0 
7f1bf2a82000-7f1bf2b82000 rw-p 00000000 00:00 0 
7f1bf2b82000-7f1bf2c3b000 r-xp 00000000 08:01 328478                     /home/adrin/Projects/tf-models/syntaxnet/.venv/lib/python3.5/site-packages/numpy/random/mtrand.cpython-35m-x86_64-linux-gnu.so
7f1bf2c3b000-7f1bf2e3b000 ---p 000b9000 08:01 328478                     /home/adrin/Projects/tf-models/syntaxnet/.venv/lib/python3.5/site-packages/numpy/random/mtrand.cpython-35m-x86_64-linux-gnu.so
7f1bf2e3b000-7f1bf2e60000 rw-p 000b9000 08:01 328478                     /home/adrin/Projects/tf-models/syntaxnet/.venv/lib/python3.5/site-packages/numpy/random/mtrand.cpython-35m-x86_64-linux-gnu.so
7f1bf2e60000-7f1bf2ea2000 rw-p 00000000 00:00 0 
7f1bf2ea2000-7f1bf2eab000 r-xp 00000000 08:01 328569                     /home/adrin/Projects/tf-models/syntaxnet/.venv/lib/python3.5/site-packages/numpy/fft/fftpack_lite.cpython-35m-x86_64-linux-gnu.so
7f1bf2eab000-7f1bf30aa000 ---p 00009000 08:01 328569                     /home/adrin/Projects/tf-models/syntaxnet/.venv/lib/python3.5/site-packages/numpy/fft/fftpack_lite.cpython-35m-x86_64-linux-gnu.so
7f1bf30aa000-7f1bf30ab000 rw-p 00008000 08:01 328569                     /home/adrin/Projects/tf-models/syntaxnet/.venv/lib/python3.5/site-packages/numpy/fft/fftpack_lite.cpython-35m-x86_64-linux-gnu.so
7f1bf30ab000-7f1bf30eb000 rw-p 00000000 00:00 0 
7f1bf30eb000-7f1bf3114000 r-xp 00000000 08:01 328507                     /home/adrin/Projects/tf-models/syntaxnet/.venv/lib/python3.5/site-packages/numpy/linalg/_umath_linalg.cpython-35m-x86_64-linux-gnu.so
7f1bf3114000-7f1bf3313000 ---p 00029000 08:01 328507                     /home/adrin/Projects/tf-models/syntaxnet/.venv/lib/python3.5/site-packages/numpy/linalg/_umath_linalg.cpython-35m-x86_64-linux-gnu.so
7f1bf3313000-7f1bf3315000 rw-p 00028000 08:01 328507                     /home/adrin/Projects/tf-models/syntaxnet/.venv/lib/python3.5/site-packages/numpy/linalg/_umath_linalg.cpython-35m-x86_64-linux-gnu.so
7f1bf3315000-7f1bf3318000 rw-p 000c6000 08:01 328507                     /home/adrin/Projects/tf-models/syntaxnet/.venv/lib/python3.5/site-packages/numpy/linalg/_umath_linalg.cpython-35m-x86_64-linux-gnu.so
7f1bf3318000-7f1bf331c000 r-xp 00000000 08:01 328505                     /home/adrin/Projects/tf-models/syntaxnet/.venv/lib/python3.5/site-packages/numpy/linalg/lapack_lite.cpython-35m-x86_64-linux-gnu.so
7f1bf331c000-7f1bf351c000 ---p 00004000 08:01 328505                     /home/adrin/Projects/tf-models/syntaxnet/.venv/lib/python3.5/site-packages/numpy/linalg/lapack_lite.cpython-35m-x86_64-linux-gnu.so
7f1bf351c000-7f1bf351d000 rw-p 00004000 08:01 328505                     /home/adrin/Projects/tf-models/syntaxnet/.venv/lib/python3.5/site-packages/numpy/linalg/lapack_lite.cpython-35m-x86_64-linux-gnu.so
7f1bf351d000-7f1bf351f000 rw-p 00019000 08:01 328505                     /home/adrin/Projects/tf-models/syntaxnet/.venv/lib/python3.5/site-packages/numpy/linalg/lapack_lite.cpython-35m-x86_64-linux-gnu.so
7f1bf351f000-7f1bf359f000 rw-p 00000000 00:00 0 
7f1bf359f000-7f1bf37b8000 r-xp 00000000 08:01 18462                      /lib/x86_64-linux-gnu/libcrypto.so.1.0.0
7f1bf37b8000-7f1bf39b8000 ---p 00219000 08:01 18462                      /lib/x86_64-linux-gnu/libcrypto.so.1.0.0
7f1bf39b8000-7f1bf39d4000 r--p 00219000 08:01 18462                      /lib/x86_64-linux-gnu/libcrypto.so.1.0.0
7f1bf39d4000-7f1bf39e0000 rw-p 00235000 08:01 18462                      /lib/x86_64-linux-gnu/libcrypto.so.1.0.0
7f1bf39e0000-7f1bf39e3000 rw-p 00000000 00:00 0 
7f1bf39e3000-7f1bf39e9000 r-xp 00000000 08:01 9047                       /usr/lib/python3.5/lib-dynload/_hashlib.cpython-35m-x86_64-linux-gnu.so
7f1bf39e9000-7f1bf3be8000 ---p 00006000 08:01 9047                       /usr/lib/python3.5/lib-dynload/_hashlib.cpython-35m-x86_64-linux-gnu.so
7f1bf3be8000-7f1bf3be9000 r--p 00005000 08:01 9047                       /usr/lib/python3.5/lib-dynload/_hashlib.cpython-35m-x86_64-linux-gnu.so
7f1bf3be9000-7f1bf3bea000 rw-p 00006000 08:01 9047                       /usr/lib/python3.5/lib-dynload/_hashlib.cpython-35m-x86_64-linux-gnu.so
7f1bf3bea000-7f1bf3c2a000 rw-p 00000000 00:00 0 
7f1bf3c2a000-7f1bf3c4e000 r-xp 00000000 08:01 58261                      /lib/x86_64-linux-gnu/liblzma.so.5.2.2
7f1bf3c4e000-7f1bf3e4e000 ---p 00024000 08:01 58261                      /lib/x86_64-linux-gnu/liblzma.so.5.2.2
7f1bf3e4e000-7f1bf3e4f000 r--p 00024000 08:01 58261                      /lib/x86_64-linux-gnu/liblzma.so.5.2.2
7f1bf3e4f000-7f1bf3e50000 rw-p 00025000 08:01 58261                      /lib/x86_64-linux-gnu/liblzma.so.5.2.2
7f1bf3e50000-7f1bf3e57000 r-xp 00000000 08:01 9794                       /usr/lib/python3.5/lib-dynload/_lzma.cpython-35m-x86_64-linux-gnu.so
7f1bf3e57000-7f1bf4056000 ---p 00007000 08:01 9794                       /usr/lib/python3.5/lib-dynload/_lzma.cpython-35m-x86_64-linux-gnu.so
7f1bf4056000-7f1bf4057000 r--p 00006000 08:01 9794                       /usr/lib/python3.5/lib-dynload/_lzma.cpython-35m-x86_64-linux-gnu.so
7f1bf4057000-7f1bf4059000 rw-p 00007000 08:01 9794                       /usr/lib/python3.5/lib-dynload/_lzma.cpython-35m-x86_64-linux-gnu.so
7f1bf4059000-7f1bf4068000 r-xp 00000000 08:01 3153                       /lib/x86_64-linux-gnu/libbz2.so.1.0.4
7f1bf4068000-7f1bf4267000 ---p 0000f000 08:01 3153                       /lib/x86_64-linux-gnu/libbz2.so.1.0.4
7f1bf4267000-7f1bf4268000 r--p 0000e000 08:01 3153                       /lib/x86_64-linux-gnu/libbz2.so.1.0.4
7f1bf4268000-7f1bf4269000 rw-p 0000f000 08:01 3153                       /lib/x86_64-linux-gnu/libbz2.so.1.0.4
7f1bf4269000-7f1bf426d000 r-xp 00000000 08:01 9777                       /usr/lib/python3.5/lib-dynload/_bz2.cpython-35m-x86_64-linux-gnu.so
7f1bf426d000-7f1bf446c000 ---p 00004000 08:01 9777                       /usr/lib/python3.5/lib-dynload/_bz2.cpython-35m-x86_64-linux-gnu.so
7f1bf446c000-7f1bf446d000 r--p 00003000 08:01 9777                       /usr/lib/python3.5/lib-dynload/_bz2.cpython-35m-x86_64-linux-gnu.so
7f1bf446d000-7f1bf446e000 rw-p 00004000 08:01 9777                       /usr/lib/python3.5/lib-dynload/_bz2.cpython-35m-x86_64-linux-gnu.so
7f1bf446e000-7f1bf466e000 rw-p 00000000 00:00 0 
7f1bf468f000-7f1bf470f000 rw-p 00000000 00:00 0 
7f1bf470f000-7f1bf48a1000 r-xp 00000000 08:01 328170                     /home/adrin/Projects/tf-models/syntaxnet/.venv/lib/python3.5/site-packages/numpy/core/umath.cpython-35m-x86_64-linux-gnu.so
7f1bf48a1000-7f1bf4aa0000 ---p 00192000 08:01 328170                     /home/adrin/Projects/tf-models/syntaxnet/.venv/lib/python3.5/site-packages/numpy/core/umath.cpython-35m-x86_64-linux-gnu.so
7f1bf4aa0000-7f1bf4aa6000 rw-p 00191000 08:01 328170                     /home/adrin/Projects/tf-models/syntaxnet/.venv/lib/python3.5/site-packages/numpy/core/umath.cpython-35m-x86_64-linux-gnu.so
7f1bf4aa6000-7f1bf4ae8000 rw-p 00000000 00:00 0 
7f1bf4ae8000-7f1bf8ae8000 rw-p 00000000 00:00 0 
7f1bf8ae8000-7f1bf8ae9000 ---p 00000000 00:00 0 
7f1bf8ae9000-7f1bf92e9000 rw-p 00000000 00:00 0 
7f1bf92e9000-7f1bfb2e9000 rw-p 00000000 00:00 0 
7f1bfb2e9000-7f1bfb2ea000 ---p 00000000 00:00 0 
7f1bfb2ea000-7f1bfbaea000 rw-p 00000000 00:00 0 
7f1bfbaea000-7f1bffaea000 rw-p 00000000 00:00 0 
7f1bffaea000-7f1bffaeb000 ---p 00000000 00:00 0 
7f1bffaeb000-7f1c002eb000 rw-p 00000000 00:00 0 
7f1c002eb000-7f1c002ec000 ---p 00000000 00:00 0 
7f1c002ec000-7f1c00aec000 rw-p 00000000 00:00 0 
7f1c00afc000-7f1c00cfc000 rw-p 00000000 00:00 0 
7f1c00cfc000-7f1c00ed8000 r-xp 00000000 08:01 348202                     /home/adrin/Projects/tf-models/syntaxnet/.venv/lib/python3.5/site-packages/google/protobuf/pyext/_message.cpython-35m-x86_64-linux-gnu.so
7f1c00ed8000-7f1c010d8000 ---p 001dc000 08:01 348202                     /home/adrin/Projects/tf-models/syntaxnet/.venv/lib/python3.5/site-packages/google/protobuf/pyext/_message.cpython-35m-x86_64-linux-gnu.so
7f1c010d8000-7f1c010e7000 rw-p 001dc000 08:01 348202                     /home/adrin/Projects/tf-models/syntaxnet/.venv/lib/python3.5/site-packages/google/protobuf/pyext/_message.cpython-35m-x86_64-linux-gnu.so
7f1c010e7000-7f1c010e8000 rw-p 00000000 00:00 0 
7f1c010e8000-7f1c010ec000 r-xp 00000000 08:01 10449                      /lib/x86_64-linux-gnu/libuuid.so.1.3.0
7f1c010ec000-7f1c012eb000 ---p 00004000 08:01 10449                      /lib/x86_64-linux-gnu/libuuid.so.1.3.0
7f1c012eb000-7f1c012ec000 r--p 00003000 08:01 10449                      /lib/x86_64-linux-gnu/libuuid.so.1.3.0
7f1c012ec000-7f1c012ed000 rw-p 00004000 08:01 10449                      /lib/x86_64-linux-gnu/libuuid.so.1.3.0
7f1c012ed000-7f1c013dd000 r-xp 00000000 08:01 328521                     /home/adrin/Projects/tf-models/syntaxnet/.venv/lib/python3.5/site-packages/numpy/.libs/libgfortran-ed201abd.so.3.0.0
7f1c013dd000-7f1c015dc000 ---p 000f0000 08:01 328521                     /home/adrin/Projects/tf-models/syntaxnet/.venv/lib/python3.5/site-packages/numpy/.libs/libgfortran-ed201abd.so.3.0.0
7f1c015dc000-7f1c015de000 rw-p 000ef000 08:01 328521                     /home/adrin/Projects/tf-models/syntaxnet/.venv/lib/python3.5/site-packages/numpy/.libs/libgfortran-ed201abd.so.3.0.0
7f1c015de000-7f1c015df000 rw-p 00000000 00:00 0 
7f1c015df000-7f1c015e7000 rw-p 000f2000 08:01 328521                     /home/adrin/Projects/tf-models/syntaxnet/.venv/lib/python3.5/site-packages/numpy/.libs/libgfortran-ed201abd.so.3.0.0
7f1c015e7000-7f1c038fb000 r-xp 00000000 08:01 328520                     /home/adrin/Projects/tf-models/syntaxnet/.venv/lib/python3.5/site-packages/numpy/.libs/libopenblasp-r0-39a31c03.2.18.so
7f1c038fb000-7f1c03afa000 ---p 02314000 08:01 328520                     /home/adrin/Projects/tf-models/syntaxnet/.venv/lib/python3.5/site-packages/numpy/.libs/libopenblasp-r0-39a31c03.2.18.so
7f1c03afa000-7f1c03b19000 rw-p 02313000 08:01 328520                     /home/adrin/Projects/tf-models/syntaxnet/.venv/lib/python3.5/site-packages/numpy/.libs/libopenblasp-r0-39a31c03.2.18.so
7f1c03b19000-7f1c03b7c000 rw-p 00000000 00:00 0 
7f1c03b7c000-7f1c03c12000 rw-p 02425000 08:01 328520                     /home/adrin/Projects/tf-models/syntaxnet/.venv/lib/python3.5/site-packages/numpy/.libs/libopenblasp-r0-39a31c03.2.18.so
7f1c03c12000-7f1c03de0000 r-xp 00000000 08:01 328171                     /home/adrin/Projects/tf-models/syntaxnet/.venv/lib/python3.5/site-packages/numpy/core/multiarray.cpython-35m-x86_64-linux-gnu.so
7f1c03de0000-7f1c03fe0000 ---p 001ce000 08:01 328171                     /home/adrin/Projects/tf-models/syntaxnet/.venv/lib/python3.5/site-packages/numpy/core/multiarray.cpython-35m-x86_64-linux-gnu.so
7f1c03fe0000-7f1c03fef000 rw-p 001ce000 08:01 328171                     /home/adrin/Projects/tf-models/syntaxnet/.venv/lib/python3.5/site-packages/numpy/core/multiarray.cpython-35m-x86_64-linux-gnu.so
7f1c03fef000-7f1c04002000 rw-p 00000000 00:00 0 
7f1c04002000-7f1c04008000 rw-p 0098a000 08:01 328171                     /home/adrin/Projects/tf-models/syntaxnet/.venv/lib/python3.5/site-packages/numpy/core/multiarray.cpython-35m-x86_64-linux-gnu.so
7f1c04008000-7f1c04048000 rw-p 00000000 00:00 0 
7f1c04048000-7f1c04067000 r-xp 00000000 08:01 9786                       /usr/lib/python3.5/lib-dynload/_ctypes.cpython-35m-x86_64-linux-gnu.so
7f1c04067000-7f1c04266000 ---p 0001f000 08:01 9786                       /usr/lib/python3.5/lib-dynload/_ctypes.cpython-35m-x86_64-linux-gnu.so
7f1c04266000-7f1c04267000 r--p 0001e000 08:01 9786                       /usr/lib/python3.5/lib-dynload/_ctypes.cpython-35m-x86_64-linux-gnu.so
7f1c04267000-7f1c0426b000 rw-p 0001f000 08:01 9786                       /usr/lib/python3.5/lib-dynload/_ctypes.cpython-35m-x86_64-linux-gnu.so
7f1c0426b000-7f1c0426c000 rw-p 00000000 00:00 0 
7f1c0426c000-7f1c04291000 r-xp 00000000 08:01 2052                       /lib/x86_64-linux-gnu/libtinfo.so.5.9
7f1c04291000-7f1c04490000 ---p 00025000 08:01 2052                       /lib/x86_64-linux-gnu/libtinfo.so.5.9
7f1c04490000-7f1c04494000 r--p 00024000 08:01 2052                       /lib/x86_64-linux-gnu/libtinfo.so.5.9
7f1c04494000-7f1c04495000 rw-p 00028000 08:01 2052                       /lib/x86_64-linux-gnu/libtinfo.so.5.9
7f1c04495000-7f1c044d6000 r-xp 00000000 08:01 2071                       /lib/x86_64-linux-gnu/libreadline.so.7.0
7f1c044d6000-7f1c046d6000 ---p 00041000 08:01 2071                       /lib/x86_64-linux-gnu/libreadline.so.7.0
7f1c046d6000-7f1c046d8000 r--p 00041000 08:01 2071                       /lib/x86_64-linux-gnu/libreadline.so.7.0
7f1c046d8000-7f1c046de000 rw-p 00043000 08:01 2071                       /lib/x86_64-linux-gnu/libreadline.so.7.0
7f1c046de000-7f1c046df000 rw-p 00000000 00:00 0 
7f1c046f6000-7f1c046fc000 r-xp 00000000 08:01 9821                       /usr/lib/python3.5/lib-dynload/readline.cpython-35m-x86_64-linux-gnu.so
7f1c046fc000-7f1c048fb000 ---p 00006000 08:01 9821                       /usr/lib/python3.5/lib-dynload/readline.cpython-35m-x86_64-linux-gnu.so
7f1c048fb000-7f1c048fc000 r--p 00005000 08:01 9821                       /usr/lib/python3.5/lib-dynload/readline.cpython-35m-x86_64-linux-gnu.so
7f1c048fc000-7f1c048fe000 rw-p 00006000 08:01 9821                       /usr/lib/python3.5/lib-dynload/readline.cpython-35m-x86_64-linux-gnu.so
7f1c048fe000-7f1c049be000 rw-p 00000000 00:00 0 
7f1c049ef000-7f1c04aef000 rw-p 00000000 00:00 0 
7f1c04aef000-7f1c04bf7000 r-xp 00000000 08:01 15774                      /lib/x86_64-linux-gnu/libm-2.24.so
7f1c04bf7000-7f1c04df6000 ---p 00108000 08:01 15774                      /lib/x86_64-linux-gnu/libm-2.24.so
7f1c04df6000-7f1c04df7000 r--p 00107000 08:01 15774                      /lib/x86_64-linux-gnu/libm-2.24.so
7f1c04df7000-7f1c04df8000 rw-p 00108000 08:01 15774                      /lib/x86_64-linux-gnu/libm-2.24.so
7f1c04df8000-7f1c04e13000 r-xp 00000000 08:01 5010                       /lib/x86_64-linux-gnu/libz.so.1.2.11
7f1c04e13000-7f1c05012000 ---p 0001b000 08:01 5010                       /lib/x86_64-linux-gnu/libz.so.1.2.11
7f1c05012000-7f1c05013000 r--p 0001a000 08:01 5010                       /lib/x86_64-linux-gnu/libz.so.1.2.11
7f1c05013000-7f1c05014000 rw-p 0001b000 08:01 5010                       /lib/x86_64-linux-gnu/libz.so.1.2.11
7f1c05014000-7f1c0503b000 r-xp 00000000 08:01 15510                      /lib/x86_64-linux-gnu/libexpat.so.1.6.2
7f1c0503b000-7f1c0523b000 ---p 00027000 08:01 15510                      /lib/x86_64-linux-gnu/libexpat.so.1.6.2
7f1c0523b000-7f1c0523d000 r--p 00027000 08:01 15510                      /lib/x86_64-linux-gnu/libexpat.so.1.6.2
7f1c0523d000-7f1c0523e000 rw-p 00029000 08:01 15510                      /lib/x86_64-linux-gnu/libexpat.so.1.6.2
7f1c0523e000-7f1c05240000 r-xp 00000000 08:01 23447                      /lib/x86_64-linux-gnu/libutil-2.24.so
7f1c05240000-7f1c0543f000 ---p 00002000 08:01 23447                      /lib/x86_64-linux-gnu/libutil-2.24.so
7f1c0543f000-7f1c05440000 r--p 00001000 08:01 23447                      /lib/x86_64-linux-gnu/libutil-2.24.so
7f1c05440000-7f1c05441000 rw-p 00002000 08:01 23447                      /lib/x86_64-linux-gnu/libutil-2.24.so
7f1c05441000-7f1c05444000 r-xp 00000000 08:01 15769                      /lib/x86_64-linux-gnu/libdl-2.24.so
7f1c05444000-7f1c05643000 ---p 00003000 08:01 15769                      /lib/x86_64-linux-gnu/libdl-2.24.so
7f1c05643000-7f1c05644000 r--p 00002000 08:01 15769                      /lib/x86_64-linux-gnu/libdl-2.24.so
7f1c05644000-7f1c05645000 rw-p 00003000 08:01 15769                      /lib/x86_64-linux-gnu/libdl-2.24.so
7f1c05645000-7f1c05803000 r-xp 00000000 08:01 15733                      /lib/x86_64-linux-gnu/libc-2.24.so
7f1c05803000-7f1c05a02000 ---p 001be000 08:01 15733                      /lib/x86_64-linux-gnu/libc-2.24.so
7f1c05a02000-7f1c05a06000 r--p 001bd000 08:01 15733                      /lib/x86_64-linux-gnu/libc-2.24.so
7f1c05a06000-7f1c05a08000 rw-p 001c1000 08:01 15733                      /lib/x86_64-linux-gnu/libc-2.24.so
7f1c05a08000-7f1c05a0c000 rw-p 00000000 00:00 0 
7f1c05a0c000-7f1c05a24000 r-xp 00000000 08:01 23155                      /lib/x86_64-linux-gnu/libpthread-2.24.so
7f1c05a24000-7f1c05c24000 ---p 00018000 08:01 23155                      /lib/x86_64-linux-gnu/libpthread-2.24.so
7f1c05c24000-7f1c05c25000 r--p 00018000 08:01 23155                      /lib/x86_64-linux-gnu/libpthread-2.24.so
7f1c05c25000-7f1c05c26000 rw-p 00019000 08:01 23155                      /lib/x86_64-linux-gnu/libpthread-2.24.so
7f1c05c26000-7f1c05c2a000 rw-p 00000000 00:00 0 
7f1c05c2a000-7f1c05c50000 r-xp 00000000 08:01 14678                      /lib/x86_64-linux-gnu/ld-2.24.so
7f1c05c56000-7f1c05c96000 rw-p 00000000 00:00 0 
7f1c05c96000-7f1c05e31000 r--p 00000000 08:01 29558                      /usr/lib/locale/locale-archive
7f1c05e31000-7f1c05e35000 rw-p 00000000 00:00 0 
7f1c05e43000-7f1c05e44000 rw-p 00000000 00:00 0 
7f1c05e44000-7f1c05e45000 rwxp 00000000 00:00 0 
7f1c05e45000-7f1c05e4c000 r--s 00000000 08:01 24108                      /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache
7f1c05e4c000-7f1c05e4f000 rw-p 00000000 00:00 0 
7f1c05e4f000-7f1c05e50000 r--p 00025000 08:01 14678                      /lib/x86_64-linux-gnu/ld-2.24.so
7f1c05e50000-7f1c05e51000 rw-p 00026000 08:01 14678                      /lib/x86_64-linux-gnu/ld-2.24.so
7f1c05e51000-7f1c05e52000 rw-p 00000000 00:00 0 
7fff1f9ac000-7fff1f9cd000 rw-p 00000000 00:00 0                          [stack]
7fff1f9d2000-7fff1f9d4000 r--p 00000000 00:00 0                          [vvar]
7fff1f9d4000-7fff1f9d6000 r-xp 00000000 00:00 0                          [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  [vsyscall]
Aborted (core dumped)

[DVLP]

@adrinjalali this might be related https://github.com/tensorflow/tensorflow/issues/9328

[adrinjalali]

@DVLP I'm not sure if that issue is related to this one. That issue is on an older Ubuntu, and I'm using Ubuntu 17.04 for this test.

Another point is that the code runs perfectly find if I run it in the bazel environment using echo "how's life in berlin?" | syntaxnet/models/parsey_universal/parse.sh downloads/English. But if you check the environment variables when run with bazel, you see:

'TERM': 'xterm-256color'
'LOGNAME': 'adrin'
'HOME': '/home/adrin'
'MAIL': '/var/mail/adrin'
'DBUS_SESSION_BUS_ADDRESS': 'unix:path=/run/user/1000/bus'
'OLDPWD': '/home/adrin/Projects/tf-models'
'LS_COLORS': 'rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36:'
'LANG': 'en_US.UTF-8'
'PYTHONPATH': '/home/adrin/Projects/tf-models/syntaxnet/bazel-bin/syntaxnet/parser_eval.runfiles:/home/adrin/Projects/tf-models/syntaxnet/bazel-bin/syntaxnet/parser_eval.runfiles/protobuf/python:/home/adrin/Projects/tf-models/syntaxnet/bazel-bin/syntaxnet/parser_eval.runfiles/six_archive:/home/adrin/Projects/tf-models/syntaxnet/bazel-bin/syntaxnet/parser_eval.runfiles/org_tensorflow:/home/adrin/Projects/tf-models/syntaxnet/bazel-bin/syntaxnet/parser_eval.runfiles/__main__:/home/adrin/Projects/tf-models/syntaxnet/bazel-bin/syntaxnet/parser_eval.runfiles/protobuf'
'SHELL': '/bin/bash'
'VIRTUAL_ENV': '/home/adrin/Projects/tf-models/syntaxnet/.venv'
'XDG_SESSION_ID': '160'
'LD_LIBRARY_PATH': '/usr/local/cuda-8.0/lib64:/home/adrin/cuda/lib64'
'_': 'bazel-bin/syntaxnet/parser_eval'
'LESSOPEN': '| /usr/bin/lesspipe %s'
'SSH_CLIENT': '217.250.162.210 60840 22'
'SSH_CONNECTION': '217.250.162.210 60840 10.0.0.4 22'
'USER': 'adrin'
'XDG_RUNTIME_DIR': '/run/user/1000'
'LESSCLOSE': '/usr/bin/lesspipe %s %s'
'PWD': '/home/adrin/Projects/tf-models/syntaxnet'
'PATH': '/home/adrin/Projects/tf-models/syntaxnet/.venv/bin:/home/adrin/bin:/home/adrin/.local/bin:/usr/local/cuda-8.0/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin'
'SHLVL': '2'
'SSH_TTY': '/dev/pts/2'

Please note the PYTHONPATH, which is completely changing where files are looked for. Now, if I also go to the respective bazel folder, set PYTHONPATH exactly as above, and run the same code which gives me the segfault, it runs perfectly fine.

This pointed me to the fact that there's another tensorflow under org_tensorflow which is coming from the subdirectory available in syntaxnet's repository, which pulls the version from several months ago. And it's different than what I have installed (1.2.1). And the code works with that old one, but not when I don't set the PYTHONPATH and therefore using my newer tensorflow.