Cmdstanr: Get appveyor CI for windows working

[jgabry]

@rok-cesnovar I added an .appveyor.yml config file so we can try continuous integration on windows via appveyor (in addition to linux on travis). It's working (running on appveyor) but errorring when running some of the unit tests (https://ci.appveyor.com/project/jgabry/cmdstanr/history).

I also started a branch appveyor-ci-windows for if we need to play around with changing things in the package (other than just the .appveyor.yml file) to get it to work.

Most of the .appveyor.yml file is set to the defaults recommended in https://github.com/krlmlr/r-appveyor#readme, except I added to build_script: the line

  - cmd: bash inst/make_cmdstan.sh -j2 -w

which seems to at least run but may not be successful.

One issue seems to be that the default paths used on appveyor may be different than what we're expecting in various functions (e.g., make_cmdstan.sh, set_cmdstan_path()). But I'm having a bit of trouble figuring out what's going on since Windows confuses the hell out of me!

The nice thing about appveyor though is that you can download the failure logs by going to the "Artifacts" tab when on the page for a particular build.

[jgabry]

Also just in case we need to set paths differently on appveyor I added a function in a helper- file in the tests directory that branches on whether we're on appveyor or not:

https://github.com/stan-dev/cmdstanr/blob/ce2cc6f3794ae4a8608d14ff800fbbbdae1e594f/tests/testthat/helper-envvars-and-paths.R#L14-L20

It does the same thing either way right now but we can change it if we need to.

[rok-cesnovar]

There were some tests failing on Windows even locally. I pushed the fixes on the appveyor branch. Does that trigger any tests to check if there is any other error left now?

[jgabry]

Thanks. Do you mean does appveyor automatically test again when you make a commit? It should. Are you able to access https://ci.appveyor.com/project/jgabry/cmdstanr/history?

[rok-cesnovar]

Yes I can access that. Thanks!

[jgabry]

You can also download failure.zip from https://ci.appveyor.com/project/jgabry/cmdstanr/builds/28837393/artifacts. If you unzip it the you get a directory cmdstanr.Rcheck. There's probably only one file in there that's useful (maybe): cmdstanr.Rcheck/tests/testthat.Rout.fail.

[rok-cesnovar]

Thanks. I can see that it fails with "CmdStan path has not been set yet. See ?set_cmdstan_path.".

cmdstan actually gets installed to C:/MinGW/msys/1.0/ so that is where I set HOME to. Crossing fingers now that that helps :)

[jgabry]

I just pushed a commit to that branch too (so you might want to pull), but what I changed is orthogonal to the appveyor/windows issue shouldn't affect anything (just cleaning up a few old things in testthat.R that shouldn't be necessary anymore)

[jgabry]

cmdstan actually gets installed to C:/MinGW/msys/1.0/ so that is where I set HOME to. Crossing fingers now that that helps :)

Worth a try! Hope it works

[jgabry]

Another thing I just noticed is that on appveyor there's an info message when building cmdstan:

bash inst/make_cmdstan.sh -j2 -w
...
* Building CmdStan binaries.
INFO: Could not find files for the given pattern(s).
...

link to lines in appveyor log: https://ci.appveyor.com/project/jgabry/cmdstanr/builds/28838920#L679

I don't get that message when I install locally. Any idea where that comes from and if that's a problem?

[rok-cesnovar]

I get the same "INFO: Could not find files for the given pattern(s)." message on Windows so that is fine I guess.

The strange thing are the \ and . at the end of files when compiling.

Actual value: "\[1\] "Running mingw32-make\.exe \\\\\\\\" \\n\[2\] " \\\\"C:/MinGW/msys/1\.0/\.cmdstanr/cmdstan/examples/bernoulli/bernoulli\.exe\\\\"" \\n\[3\] "mingw32-make\.exe: Nothing to be done for 'C:/MinGW/msys/1\.0/\.cmdstanr/cmdstan/examples/bernoulli/bernoulli\.exe'\.\\\\r""

I dont see anything wrong in the paths in this case tho:

> test_check("cmdstanr")
Running mingw32-make.exe \
  "C:/projects/cmdstanr/cmdstanr.Rcheck/tests/testthat/resources/stan/logistic.exe"
mingw32-make.exe: *** No rule to make target 'C:/projects/cmdstanr/cmdstanr.Rcheck/tests/testthat/resources/stan/logistic.exe'.  Stop.

I have a few more ideas will do a few more rounds of stabbing in the dark :)

[jgabry]

Thanks for trying!

[jgabry]

@rok-cesnovar just a heads up I did a bunch of work on #68 reorganizing test files so I went ahead and updatged the appveyor branch too to keep it up to date.

[jgabry]

@rok-cesnovar Thanks again for trying to figure this out. I didn’t realize it would be so annoying to get working, sorry!

[rok-cesnovar]

It seems as if its not respecting the wd (working directory) argument. Make spits out "nothing to do for x", which means that it does find the exe file we are trying to make but not the make rule. As if its not being called from the cmdstan folder. Otherwise it would say "is up to date". Very bizzare. Will spin up a windows 2012 server instance later today to see if its something related to that win version.

[rok-cesnovar]

This is a tough one. I am able to run tests manually on Windows server 2019 and 2016, wasn't able to make it work on 2012. But appveyor on 2016 or 2019 still doesn't give in.

I am going to try and make custom scripts that would not install all these tools, just those we really need. I have a gut feeling the issue is that one of the build tools used is not from RTools and then things mix up and you get a headache :)

[jgabry]

Thanks @rok-cesnovar, sounds like a serious headache indeed!

@ahartikainen @mitzimorris it looks like you have an appveyor.yml file for cmdstanpy (and that it’s installing rtools, although maybe a different one than we are). Did you have any of these issues? Also I can’t seem to find the appveyor logs for cmdstanpy online. Is there a url?

[ahartikainen]

https://ci.appveyor.com/project/ahartikainen/cmdstanpy-kkwu4

I would not touch msys on appveyor if not really needed.

What toolchain are you using?

[rok-cesnovar]

We are using Rtools and the g++ that comes with it.

Its not really a toolchain problem now, the external processes execution, specifically make, is behaving weirdly.

[ahartikainen]

So, if you ran make commands in cmdline (in appveyor) everything works? But calling from R make fails?

Just making sure you call mingw32-make?

[rok-cesnovar]

Yeah, I can build and run cmdstan in cmdline. The thing is that everything works on Windows otherwise for me locally, I can run all tests in the package, run models, etc. I mostly do stuff through RStudio tho.

So I think it's a PowerShell/cmd confusion I guess. Will try to be more consistent now.

[rok-cesnovar]

Ok, its finally working: https://ci.appveyor.com/project/jgabry/cmdstanr/builds/28910508

A new version is here https://github.com/stan-dev/cmdstanr/tree/appveyor-win-pt2
Will clean this up tomorrow and make a PR.

[jgabry]

Awesome!!