Zstd: Add windows multithreading support to cmake scripts

Created on 30 Aug 2017  路  9Comments  路  Source: facebook/zstd

I see that currently only pthreads are supported.

build issue enhancement help wanted

All 9 comments

Multithreading should work natively on windows. We have a pthreads wrapper for Windows https://github.com/facebook/zstd/blob/dev/lib/common/threading.h#L20.

Thanks. Trying to build on windows with cmake, cmake generates an error when
ZSTD_MULTITHREAD_SUPPORT is turned on.

Is this a bug in the build system ?

@boxerab what is the error?

CMake Error at programs/CMakeLists.txt:68 (MESSAGE):
  ZSTD currently does not support thread libraries other than pthreads

I believe that is just a shortcoming of the CMake build system. The CMake build system is maintained by the community, and a patch that adds support for windows threads would be welcome.

Alright, changing issue title to reflect this.

I think this issue can be closed. I just built zstd on Windows using CMake, and I encountered no errors.

I've been looking at this issue on a Windows 10 box,
and indeed, cmake script seems to properly generate a binary with ZSTD_MULTITHREAD_SUPPORT enabled.

On my system, it compiles with mingw64, and uses the pthread path for multithreading code.

However, I also noticed that multi-threading performance is sluggish.
I actually don't see a benefit between 1 and 2 threads when benchmarking (though the output slightly differs, indicating that jobs were cut differently). Using more > 2 threads actually reduce performance (and this is a 4-cores laptop).
Looking at top, cpu usage crosses the 100% mark, but remain at ~130%, far from the potential.

My interpretation is that mingw64/msys2 may have a hard time starting new threads.
This is noticeable at fork() level, when any invocation generate delay in _seconds_.
So maybe something equivalent happens at thread level.
The delay in setting up a new threads would be large enough to nullify any benefit from multithreading compression.

I tried to force the win32 path, but it doesn't recognise symbol _beginthreadex().

But I guess, this is more a mingw64 topic than a cmake on Windows one now...

Was this page helpful?
0 / 5 - 0 ratings

Related issues

rgdoliveira picture rgdoliveira  路  3Comments

indygreg picture indygreg  路  3Comments

sergeevabc picture sergeevabc  路  3Comments

icebluey picture icebluey  路  3Comments

robert3005 picture robert3005  路  4Comments