Redex: Support Windows build of ReDex

[satnam6502]

Make sure we can reliably build and support ReDex for Windows 10. One approach might be to use the new native support for bash as suggested by @jp1017 in #17

[Orvid]

Alternatively, I'm doing significant work towards getting Folly building under MSVC, so you may soon be able to get away without needing bash on windows.

[hantsy]

if possible provide a ReDex Docker ready build environment for Windows user.

[necros2k7]

+1

[jjhesk]

+1, try it on docker + this container

[necros2k7]

yeah but docker requires lin. or mac

[WizKid]

https://docs.docker.com/docker-for-windows/ ?

[necros2k7]

my bad, thanx 8-)

[deephole]

This is really needed!!!

[minjang]

Heads-up: we (Facebook) also now have some demand running Redex on Windows. Unfortunately, this is still low priority. If one is extremely interesting, it's worthy to try with Visual Studio. E.g., create a command-line executable C++ project with all source code (except testing) and proper defines. You will see compile errors from Unix/Linux-only headers. So, it requires some work. A good news is that we recently updated code to be more C++ standard (e.g., not using pthread). While we can't give an ETA, it's not that far from supporting Windows.

[dphillipsprosper]

So, this issue's been open for a couple years now, I was wondering if there's any update?

Reading the docs, it appears Redex can be compiled for Windows, albeit it's in an "experimental" state. So I was wondering, what does "experimental" really mean here?

Is it that it'll compile, and it'll work but it's not thoroughly tested so it's "experimental"?
Or is it that compiling itself is still experimental?

[youssef212]

when building for windows i get a lot of compile errors and build fails tried with both cmake and msbuild

[necros2k7]

welcome to the club bud)

[youssef212]

is it worth the try ? like how much does it optimize? because i already wasted like day and a half on trying to build it

[necros2k7]

if youll succeed youll be worshipped to, 4 years no binary for people...

[agampe]

I've cleaned up Redex a bit with some known incompatibilities. The support is still experimental, but feedback is welcome. You no longer need MSVC or WSL, a pure OSS setup works. Thus it should work on older releases of Windows, too.

(There's a good chance the MSVC build works again, too, but that is untested.)

Follow the (light) instructions here and here.

[agampe]

The MSYS-style build is now continuously tested on CircleCI.

[necros2k7]

Any link to the binary?

[agampe]

Any link to the binary?

We do not provide binaries for any platform at this point, and there are no plans to. Please follow the quite simple instructions here to build your own.

[agampe]

Since people have complained: I was not responsible for the structure of the installation document, which tries to keep common parts common but thus pulls the steps apart. The written-down steps are also IMO clearly explained. But let me try a more streamlined version here.

Preparation

• Install MSYS2. The link is in the docs, or go here.
• Install Python 3.6+ for Windows. The link is in the docs, or go here.

MSYS Compilation

The following steps do all need to be run in a "MingW64 Shell." Installing MSYS2 should have added this option to the Start Menu. Alternatively, run the MingW64 executable in C:\msys2 (or wherever you installed MSYS2). (# denotes a comment, does not need to be typed/copied.)

# Possibly update MSYS and package caches
pacman -Syuu  # If this updates MSYS and tells you, close the shell and open a new one
# Install dependencies
pacman -Sy make mingw-w64-x86_64-boost mingw-w64-x86_64-cmake mingw-w64-x86_64-gcc mingw-w64-x86_64-jsoncpp mingw-w64-x86_64-make git
# Download redex from Github
git clone https://github.com/facebook/redex.git
cd redex
# Prepare and build
mkdir build-cmake
cd build-cmake
cmake -G "MSYS Makefiles" ..
make -j4  # This typically needs 8GB of RAM. You may need to lower the number

Collect All Files

At this point, the binary is created. Collect all the necessary things for redex from under the MSYS home folder (by default C:\msys2\home\{USERNAME}\redex (or in the MingW64 shell, typing pwd -W will give the Windows path). You need

redex.py
pyredex/*
build-cmake/redex-all.exe

Redex is invoked through the Python script nowadays, which will unpack an APK, invoke redex-all, and repack the APK. In a standard command prompt (MSYS is no longer required), run something like

python redex.py ...

[youssef212]

thanks for your replay i deleted my comment even before you reply cuz i felt i was bit unfair and my be tilted XD but i calmed down quickly , i know you probably saw the comment in email notification may be , i deleted it cuz i didnt want to be an idiot who just push false claim on some other dev without knowing all the facts so i am sorry about the earlier comment i amnt sure if you saw it or not actually but still i had to say this .

[youssef212]

btw make -j4 is still gives errors

[agampe]

btw make -j4 is still gives errors

Can you open a new issue with the error you get?

[youssef212]

sure i will, thank you so much for your time

[necros2k7]

trying wtih installation instructions "win10 experimental":

C:DEVVCPKGvcpkg>.bootstrap-vcpkg.bat
Не удается преобразовать значение "System.Object[]" в тип "System.Xml.XmlDocument". Ошибка: "Невозможно вставить указан
ный узел в качестве действительного дочернего узла данного узла, т.к. указанный узел имеет неверный тип."
C:DEVVCPKGvcpkgscriptsbootstrap.ps1:76 знак:9

• [xml]$asXml = $output
• ~~~~~
  
  
  
  • CategoryInfo : InvalidArgument: (:) [], RuntimeException
  
  
  • FullyQualifiedErrorId : InvalidCastToXmlDocument
  
  

Не удается найти свойство "instances" для этого объекта. Убедитесь, что оно существует.
C:DEVVCPKGvcpkgscriptsbootstrap.ps1:78 знак:31

• foreach ($instance in $asXml.instances.instance)
• ~~~~~
  
  
  
  • CategoryInfo : NotSpecified: (:) [], PropertyNotFoundException
  
  
  • FullyQualifiedErrorId : PropertyNotFoundStrict
  
  

Could not find Visual Studio. VS2015, VS2017, or VS2019 (with C++) needs to be installed.
C:DEVVCPKGvcpkgscriptsbootstrap.ps1:143 знак:9

• throw "Could not find Visual Studio. VS2015, VS2017, or VS201 ...
• ~~~~~~~~~~~~~
  
  
  
  • CategoryInfo : OperationStopped: (Could not find ...o be installed.:String) [], RuntimeException
  
  
  • FullyQualifiedErrorId : Could not find Visual Studio. VS2015, VS2017, or VS2019 (with C++) needs to be installed
  
  

IMHO issue should be reopened until easy win compilation can be implemented

[agampe]

The MSYS build is simple/easy, tested on CircleCI and works. If you find a problem with that, please file a new issue, and maybe when I have time I'll get around to fixing it.

The simple truth is: the Windows build will always be an experimental, mostly-unsupported feature, as it is not the main environment Redex is developed & used in. The Visual Studio attempt is old, and nobody is working on that right now, especially with a working alternative. Pull requests are welcome, but without those I will close issues (because nobody on our side will fix it ever).

If you cannot deal with the MSYS instructions and want "easy win compilation," I suggest trying out WSL2, which is closest to the supported environment. If you find that route doable, pull requests to amend our docs are very welcome.

---

FullyQualifiedErrorId : Could not find Visual Studio. VS2015, VS2017, or VS2019 (with C++) needs to be installed

I do not understand what your problem is/why you cannot make the next step yourself. The error message seems pretty clear, and our docs for Visual Studio clearly say so: you must install one of the VS versions first, obviously. Installing core Windows software that isn't automatable will never be part of "easy win compilation."

FullyQualifiedErrorId : InvalidCastToXmlDocument

We also do not own or maintain VCPKG, which would be where you should go with issues directly involved in it. Though it's imaginable that the other error somehow causes this.

---

Notes:

• I will probably rearrange the docs to make sure that it's clear the Visual Studio instructions are not guaranteed to work.
• Pull requests to incorporate things like PyInstaller and NSIS for easier deployment are welcome. Please file an issue, attach a PR, and I will try to get it integrated.

[necros2k7]

agampe, you say "The MSYS build is simple/easy, tested on CircleCI and works." can you please upload somewhere working build for community? not all here are skillful compilers

[agampe]

agampe, you say "The MSYS build is simple/easy, tested on CircleCI and works." can you please upload somewhere working build for community? not all here are skillful compilers

I still feel strongly that the instructions cover a simple setup that is complete (and the CircleCI config gives an alternative starting from chocolatey). In general, in my opinion getting the redex configuration right is a way more complex task than getting the binary.

Anyways, I've submitted a patch that adds Windows to the nightly build on CircleCI and stores the binary as an "artifact."

Notes:

• This is totally inofficial. No guarantees, warrantees, etc, you use it at your own risk, it's nothing but a test artifact for us, you're still supposed to compile that for yourself. That said, if you find problems, file a Github issue.
• CircleCI stores artifacts for 30 days. If you want older binaries (maybe they were more stable or w/e), it's your job to keep copies. There's also only the build for master, no such thing for stable.
• Our CircleCI pipeline is here. You have to select a nightly windows build and look into the artifacts tab. You should find a zip with the binary, the python files and the default config.
• It missed today's "nightly" build. If you cannot wait till midnight GMT, you can check the pull request build (2739).

[agampe]

Obviously not. redex-all has not been the main driver in years, instead use redex.py. It's sketched in the install documentation at the end here. We'll update the main docs.

[agampe]

The comment seems to have been removed, I will still reply:

so what redex-all.exe for?

redex-all is the ReDex optimizer implemented in C++. However, for a long time now redex-all has not been dealing with APKs, but unpacked dex files (as we now support multiple custom compression schemes that are hard to support in C++). redex.py is the "new" wrapper that deals with APKs, alignment, signing. redex.py will unpack the given APK, call redex-all (either with a default path, or the --redex-binary parameter), and repackage afterwards. You need Python3 for it (which is detailed in the installation instructions).