Cibuildwheel: Cygwin?

[ax3l]

Hi,

just a pretty naive question: do people build windows wheels with Cygwin and is this possible with cibuildwheel?

I know that one builds with MSVC from VS by default but am wondering, since building with Cygwin provides some interesting additional features (such as a POSIX conformant filesystem view).

[YannickJadoul]

I suppose it should be possible, if Cygwin respects MSVC's calling conventions etc. Worth a try, anyway.

But does Cygwin also link against some runtime/shared library? If so, you'd need to make sure that's also installed/packaged.

Officially, the same compiler versions should be used as the ones used to compile Python, but pybind11 claims that there's a way around this, and I've indeed managed to make wheels for Windows that make use of a newer (MSVC) compiler. The one problem that might occur is that the libraries are not found, so I added a note in my documentation. I can image a similar thing would be the case for Cygwin?

[ax3l]

Thanks for the pointers. I guess the most reliable approach would be to use the wheel only with the Cygwin CPython package that a wheel was built against.

Not sure if and how Pip enforces this. According to my test running pip wheel ., this gets at least encoded in the file name: ./<package>-<version>-cp37-cp37m-cygwin_3_1_4_x86_64.whl for current CPython packages cmake,gcc-core,gcc-g++,make,python37-devel,python37-numpy,python37-pip,python37-wheel.

How would I specify the same install routines and compiler selection hints with cibuildwheel? It's probably very simple via a script called in CIBW_BEFORE_BUILD, and I am just afraid to do something wrong.

But does Cygwin also link against some runtime/shared library? If so, you'd need to make sure that's also installed/packaged.

As far as I know, and I am a total beginner on Cygwin, it comes with own shared libraries and std libs but no extra runtime.

[Czaki]

The problem which I see is that pypi dose not accept cygwin tag. https://github.com/pypa/warehouse/blob/master/warehouse/forklift/legacy.py

So I'm not sure if modify code for run on cygwin is worth.

[YannickJadoul]

this gets at least encoded in the file name: ./<package>-<version>-cp37-cp37m-cygwin_3_1_4_x86_64.whl

Hmmm, interesting. So this is considered to be a different platform by pip/wheel/...? So it probably won't be compatible with "standard" Windows Python, or how do these things work? Does cygwin load standard Windows extensions? Is there a relevant PEP or so?

I was about to say that if it's easy, we could provide a way of building this with cibuildwheel. Not too eager to start messing with more Windows, though :-( And after @Czaki's comment: if it's not standardized/uploadable to PyPI, maybe it's indeed not an immediate fit for cibuildwheel.

[ax3l]

Thanks for the info @Czaki, I guess that's basically making PyPI.org Cygwin wheel releases impossible as of today. Uploading Cygwin wheels to one's own index is still possible, but I won't go down that road for now.

Generally, Cygwin and also forks such as MSys2 seem to be an environment and package manager. They provide own Python builds based on a GNU stack and pip and can in principle be extended with compatible wheels as I tested. I guess they are considered their own platform but I cannot find active PEPs either that try to standardize this platform tag.

Thanks again for all the great input! :sparkles:

[myselfhimself]

The problem which I see is that pypi dose not accept cygwin tag. https://github.com/pypa/warehouse/blob/master/warehouse/forklift/legacy.py

So I'm not sure if modify code for run on cygwin is worth.

where and what is the rejected cygwin tag in this legacy pypi piece of code?

[myselfhimself]

Hello, time has passed since the last reply to this issue. So... trying to look for deeper answers as I am stuck a bit.

gmic-py is a C/Python binding for the C++ G'MIC image processing library which preferably builds with unix-length variable typing on Windows OS (especially the long type), despite properly using GDI for graphics display instead of X display calls. Hence, the G'MIC core maintainer does not like MSVCC on Windows and compiles libgmic, the core library with MSYS2 instead.

I am looking for a simple way to make windows wheels for gmic-py. Someone who is not from the Unix world is trying to help out with msys2-based handmade building.
Has someone here succeeded in building wheels for Windows using cygwin, MSYS2 or cibuildwheel?

gmic-py's dependencies are:
(C API libraries)

• libjpeg
• libtiff
• libpng
• zlib
• libomp
• libcurl
  (Python packages)
• pkgconfig (pure Python) -- for setup.py compilation setup time only
• wurlitzer (pure Python)

I do not have a Windows VM for now to test things live..

Related to https://github.com/myselfhimself/gmic-py/issues/5

[Czaki]

These lines define the allowed platform tags:
https://github.com/pypa/warehouse/blob/dbaf40b097d20264a7b41a07b4a62f7fe2d733e4/warehouse/forklift/legacy.py#L108-L163

as @ax3l mentioned platform tag for cygwin looks like cygwin_3_1_4_x86_64 so it will not match rules from the above lines.

As preparation to have a universal build I suggest using git submodules with gmic instead of download it with some bash script.

I suggest extracts all style checkers to separate jobs. Do not mix tasks (pre-commit may be useful).

gmic-py is a C/Python binding for the C++ G'MIC image processing library which preferably builds with unix-length variable typing on Windows OS (especially the long type), despite properly using GDI for graphics display instead of X display calls. Hence, the G'MIC core maintainer does not like MSVCC on Windows and compiles libgmic, the core library with MSYS2 instead.

I do not understand. Why you cannot use fixed-length type like uint16_t, int32_t and similar? (from cstdint)

[myselfhimself]

These lines define the allowed platform tags:
https://github.com/pypa/warehouse/blob/dbaf40b097d20264a7b41a07b4a62f7fe2d733e4/warehouse/forklift/legacy.py#L108-L163

thanks I had looked for a blacklist.. but the only constraint type is a whitelist obviously..
as @ax3l mentioned platform tag for cygwin looks like cygwin_3_1_4_x86_64 so it will not match rules from the above lines.

I had missed that already mentioned cygwin string pattern in the discussion, thank you for pointing it out
As preparation to have a universal build I suggest using git submodules with gmic instead of download it with some bash script.

I am amazed by the depth of your study at my code thanks! Yes, the archive downloading is something I kept from the very beginning. I am realizing now that the upstream gmic has git tags so I will be able to use submodules for this.
I suggest extracts all style checkers to separate jobs. Do not mix tasks (pre-commit may be useful).

I would like any job to stop if code styling is not respected.. I could indeed make interdependent jobs, where the style job is called first within the build job. Thank you very much for your remark.

gmic-py is a C/Python binding for the C++ G'MIC image processing library which preferably builds with unix-length variable typing on Windows OS (especially the long type), despite properly using GDI for graphics display instead of X display calls. Hence, the G'MIC core maintainer does not like MSVCC on Windows and compiles libgmic, the core library with MSYS2 instead.

I do not understand. Why you cannot use fixed-length type like uint16_t, int32_t and similar? (from cstdint)

I do not know precisely, the G'MIC C++ project was started about 10 years ago and windows was - knowing its author - possibly not a target

[Czaki]

For me, building a wheel for windows is always a big problem. It has its own flag system for compilator. There is no audithweel, deloacate alternative for windows so you need to manually add all dependencies to wheel.

So without Windows VM or other access to Windows for the tests, it may be time consuming to do it.

(as I see in setup.py that there is setting flag, looking for dependecies, and compilation of base library outside setup.py).

[myselfhimself]

Hello

Understood.. The Github Actions environment for msys2 works good actually.
So after a few dozen commits to run my script "in the blind" on a windows
image, with some logging, my msys2 recipe now works for building .dll and
.whl of the gmic-py c++ python binding project.
I would now like to include all necessary .dll files:

from a very similar executable to what my library needs, I know that will
have to use this command:

$ cd /mingw64/bin/ && cp libgcc_s_seh-1.dll libwinpthread-1.dll libgomp-1.dll \
  libstdc++-6.dll libcurl-4.dll libbrotlidec.dll libbrotlicommon.dll \
  libcrypto-1_1-x64.dll libidn2-0.dll libiconv-2.dll libintl-8.dll
libunistring-2.dll \
  libnghttp2-14.dll libpsl-5.dll libssh2-1.dll zlib1.dll
libssl-1_1-x64.dll libzstd.dll \
  libfftw3-3.dll libjpeg-8.dll libpng16-16.dll libtiff-5.dll
liblzma-5.dll PATH_TO_EXE/

except that I do not know where to store those .dll files before wheel
building.

Should I follow this ?
https://stackoverflow.com/questions/24071491/how-can-i-make-a-python-wheel-from-an-existing-native-library

st 4. 11. 2020 v 12:22 odesílatel Grzegorz Bokota [e-mail]
napsal:

For me, building a wheel for windows is always a big problem. It has its
own flag system for compilator. There is no audithweel, deloacate
alternative for windows so you need to manually add all dependencies to
wheel.

So without Windows VM or other access to Windows for the tests, it may be
time consuming to do it.

(as I see in setup.py that there is setting flag, looking for dependecies,
and compilation of base library outside setup.py).

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/joerick/cibuildwheel/issues/329#issuecomment-721674695,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAJU5QUN6CXJXF3PPEEG5BTSOE2OZANCNFSM4MTMK54Q
.