Taichi: Multi-threaded testing

[archibate]

Concisely describe the proposed feature
Use multi-threading when doing python test.

Describe the solution you'd like
May abandon pytest, I feel it not so flexible if we want to do something more.
Then import threading, implement our own test system.

Additional comments
Might help #479 problems too.

[yuanming-hu]

Serial testing is indeed a problem. Sometimes I use https://docs.pytest.org/en/3.0.1/xdist.html which enables pytest -n num_threads command to run all Python tests in parallel, which does gives a 2x-3x performance boost.

Simply adding xdist to dependencies and add -n 4 here https://github.com/taichi-dev/taichi/blob/2097096488de7641c242b5274cd020ca1a1e5013/python/taichi/main.py#L20 should be way easier then implementing our own test system :-)

(Another option is https://pypi.org/project/pytest-parallel/, which I've never tried.)

[archibate]

Really faster!

[archibate]

But when pytest called from taichi/main.py, it get into a Bus Error at first, and continues testing...

========================================= test session starts ============================================
platform linux -- Python 3.8.1, pytest-5.3.4, py-1.8.1, pluggy-0.13.1
rootdir: /home/bate/Develop/taichi
plugins: xdist-1.31.0, forked-1.1.3
gw0 ok / gw1 ok / gw2 ok / gw3 okFatal Python error: Bus error

Thread 0x00007f58ef008700 (most recent call first):
  File "/usr/lib/python3.8/site-packages/execnet/gateway_base.py", line 400 in read
  File "/usr/lib/python3.8/site-packages/execnet/gateway_base.py", line 432 in from_io
  File "/usr/lib/python3.8/site-packages/execnet/gateway_base.py", line 967 in _thread_receiver
  File "/usr/lib/python3.8/site-packages/execnet/gateway_base.py", line 220 in run
  File "/usr/lib/python3.8/site-packages/execnet/gateway_base.py", line 285 in _perform_spawn

[archibate]

  try:
    from multiprocessing import cpu_count
    cpu_count = cpu_count()
  except:
    cpu_count = 4
  args += ['-n', str(cpu_count)]

[archibate]

Everything, ok! Closing now.

[yuanming-hu]

Good news: found the source of the mysterious bus error: https://github.com/taichi-dev/taichi/issues/501