Esy: Windows 10 - Esy is spawning wrong 'curl'

[bryphe]

esy version: Reproduces in both 0.3.4 and 0.4.4-8e07b1
Operating System: Windows 10

Porting over from https://github.com/bryphe/revery/pull/106 , as it seems like a general esy bug.

Issue:

From the linked bug:

---

[email protected] spits out a bunch of:

error unable to fetch @opam/menhir@opam:20181113:
                                                   source: archive:https://gitlab.inria.fr/fpottier/menhir/repository/20181113/archive.tar.gz#md5:69ce441a06ea131cd43e7b44c4303f3c
                                                   error: error: error running subprocess
                                                            fetching dist: archive:https://gitlab.inria.fr/fpottier/menhir/repository/20181113/archive.tar.gz#md5:69ce441a06ea131cd43e7b44c4303f3c
                                                   source: archive:https://opam.ocaml.org/cache/md5/69/69ce441a06ea131cd43e7b44c4303f3c#md5:69ce441a06ea131cd43e7b44c4303f3c
                                                   error: error: error running subprocess
                                                            fetching dist: archive:https://opam.ocaml.org/cache/md5/69/69ce441a06ea131cd43e7b44c4303f3c#md5:69ce441a06ea131cd43e7b44c4303f3c
error unable to fetch @opam/cmdliner@opam:1.0.2:
                                                  source: archive:http://erratique.ch/software/cmdliner/releases/cmdliner-1.0.2.tbz#md5:ab2f0130e88e8dcd723ac6154c98a881
                                                  error: error: error running subprocess
                                                           fetching dist: archive:http://erratique.ch/software/cmdliner/releases/cmdliner-1.0.2.tbz#md5:ab2f0130e88e8dcd723ac6154c98a881
                                                  source: archive:https://opam.ocaml.org/cache/md5/ab/ab2f0130e88e8dcd723ac6154c98a881#md5:ab2f0130e88e8dcd723ac6154c98a881
                                                  error: error: error running subprocess
                                                           fetching dist: archive:https://opam.ocaml.org/cache/md5/ab/ab2f0130e88e8dcd723ac6154c98a881#md5:ab2f0130e88e8dcd723ac6154c98a881
error unable to fetch @opam/ocamlfind@opam:1.8.0:
                                                   source: archive:http://download.camlcity.org/download/findlib-1.8.0.tar.gz#md5:a710c559667672077a93d34eb6a42e5b
                                                   error: error: error running subprocess
                                                            fetching dist: archive:http://download.camlcity.org/download/findlib-1.8.0.tar.gz#md5:a710c559667672077a93d34eb6a42e5b
                                                   source: archive:http://download2.camlcity.org/download/findlib-1.8.0.tar.gz#md5:a710c559667672077a93d34eb6a42e5b
                                                   error: error: error running subprocess
                                                            fetching dist: archive:http://download2.camlcity.org/download/findlib-1.8.0.tar.gz#md5:a710c559667672077a93d34eb6a42e5b
                                                   source: archive:https://opam.ocaml.org/cache/md5/a7/a710c559667672077a93d34eb6a42e5b#md5:a710c559667672077a93d34eb6a42e5b
                                                   error: error: error running subprocess
                                                            fetching dist: archive:https://opam.ocaml.org/cache/md5/a7/a710c559667672077a93d34eb6a42e5b#md5:a710c559667672077a93d34eb6a42e5b

while [email protected] says only:

info install 0.3.4
warn failed to fetch ocaml, retrying (attempts left: 3)
warn failed to fetch ocaml, retrying (attempts left: 2)
warn failed to fetch ocaml, retrying (attempts left: 1)
error: error running subprocess
  resolving ocaml@>=4.6.0 <4.7.0
esy: exiting due to errors above

In process manager I found out that esyCommand.exe spawns curl from C:\Windows\System32. Am I right that it should spawn the one which comes with cygwin?

---

[andreypopp]

hm... Curl module uses Lwt_process now directly and that means it resolves curl from the Path. I'm a bit surprised but it looks like Windows comes with curl.exe installed and we use it in esy!

[bgoscinski]

Yeah https://www.bleepingcomputer.com/news/microsoft/built-in-tar-and-curl-utilities-to-be-included-in-windows/

[andreypopp]

This makes me wonder if we wonder if we should use windoes tar and not the one from cygwin (maybe it's faster?).

@bryphe you can repro this? This looks to me like a network glitch (I've seen errors like these but they disappeared on a second try). Maybe we should try to do more retries?

[bryphe]

@bryphe you can repro this? This looks to me like a network glitch (I've seen errors like these but they disappeared on a second try). Maybe we should try to do more retries?

Will give it a shot - I don't repro it when running esy but there might other ways to hit this. I suspect it's an issue that is sensitive to the layout of PATH. A fix for now would be for us to switch the call to use EsyBashLwt.

This makes me wonder if we wonder if we should use windoes tar and not the one from cygwin (maybe it's faster?).

We could try and see! Note that this would break for earlier Windows versions. Sometimes the semantics of the Windows commands aren't quite the same as the POSIX equivalents, so we'd definitely need to test it.

[bgoscinski]

I can repro this in 100% cases (I tried about 10 times in last 2 hours or so)

[bgoscinski]

And I'm sure that it's not an issue with some firewall because when I paste https://gitlab.inria.fr/fpottier/menhir/repository/20181113/archive.tar.gz#md5:69ce441a06ea131cd43e7b44c4303f3c in my browser the download starts