Cmdstanr: install_cmdstan: use `version` instead of `release_url`?

[jgabry]

Describe the issue

For installation, CmdStanPy has a version argument instead of release_url, which is what we have in CmdStan. It's probably easier for the user to just specify a version number instead of a whole url, so we could deprecate release_url in favor of version.

CmdStanR version number

0.1.3

[rok-cesnovar]

I am fine with adding version, I agree it is convenient.

But I am not in favor of removing the URL argument. With it you can also point to other non-official cmdstan releases (non stan-dev).

[jgabry]

Good point about the url, that didn't occur to me!

@mitzimorris When we talked about this I forgot what @rok-cesnovar just pointed out about the release_url argument. What do you think?

On Mon, Sep 21, 2020 at 10:27 AM Rok Češnovar [e-mail]
wrote:

I am fine with adding version, I agree its convenient.

But I am not in favor of removing the URL arg. With it you can also point
to other non official cmdstan releases (not on stan-dev).

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/stan-dev/cmdstanr/issues/300#issuecomment-696226294,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AB3PQQZAFBH7UATKYNRC7MDSG55I7ANCNFSM4RUXEMTA
.

[mitzimorris]

But I am not in favor of removing the URL argument. With it you can also point to other non-official cmdstan releases (non stan-dev).

how many non-official cmdstan releases are there? somewhat reluctant to implement this in CmdStanPy. the workaround is for the user to download and install the release as they wish and then set cmdstan path.

[bbbales2]

Oh I was just looking today. Is it still possible to check out and build from a custom git/branch combo or has that been removed?

I see the only argument is release_url, and the docs point to the different cmdstan releases.

how many non-official cmdstan releases are there?

I have ~at least two~ (edit: actually just one) I have used for my own stuff. I think we used these in some of the reduce_sum/ode solver development as well. I would like to encourage people to branch math and use custom cmdstans instead of writing custom C++ that links into Rstan because there's a chance that they'll end up with maintainable code if they do the first and I'm skeptical of the second.

[bbbales2]

Here's a recent someone doing this and dealing with difficulties of interfacing their C++ in R: https://discourse.mc-stan.org/t/any-simple-working-example-of-external-c-lpdf-function/18080

[mitzimorris]

check out and build from a custom git/branch combo

I totally get that this is important. the question is, write code to do this as part of the package, or find another way of doing the install - because set_cmdstan_path or setting environment variable CMDSTAN will both let you use/test custom installation.

for R users and anyone on Windows, scripts or functions to do this would be useful. but it all adds up, w/r/t maintenance/testing/documentation - if this is a power-user feature, then would documentation alone suffice?

[bbbales2]

Well the reason I was asking was it used to be part of cmdstanr. It was covenient that it took and put the cmdstan in the .r folder and you didn't have to worry about it.

Is there a way to have cmdstanr do the build at least?

if this is a power-user feature, then would documentation alone suffice?

It was really for users of developers' stuff. I can hack up a cmdstan cause I know what's going on.

But if I want someone else to use my cmdstanr extension, then I gotta tell them how to download and build cmdstan.

I see the place it got removed: https://github.com/stan-dev/cmdstanr/pull/158

[bbbales2]

Anyway the cmdstanX interfaces are already doing the right build things, so I like piggy-backing off them. Writing my own scripts here would just be duplicating stuff that's already here. (Edit: and I probably wouldn't even try to write scripts cause they'd be fragile, I'd just send really long e-mails)

[bbbales2]

Is there a way to have cmdstanr do the build at least?

By this I mean, I assume there is a package I can do git clone --branch branchname giturl foldername with (or I'd do it with a system call) and then I can build with cmdstanr and that's not that bad.

[rok-cesnovar]

Is there a way to have cmdstanr do the build at least?

Yes, once you set the path you can run rebuild_cmdstan(cores = 4)

By this I mean, I assume there is a package I can do

This is what I use when I need to clone:

clone_cmdstan <- function(
  clone_dir,
  cmdstan_url = "http://github.com/stan-dev/cmdstan",
  stan_url = "http://github.com/stan-dev/stan",
  math_url = "http://github.com/stan-dev/math",
  cmdstan_branch = "develop",
  stan_branch = "develop",
  math_branch = "develop",
  cores = 4) {
  if (!dir.exists(clone_dir)) {
    dir.create(clone_dir)  
  }
  git2r::clone(url = cmdstan_url, branch = cmdstan_branch, local_path = file.path(clone_dir))
  git2r::clone(url = stan_url, branch = stan_branch, local_path = file.path(clone_dir, "stan"))
  git2r::clone(url = math_url, branch = math_branch, local_path = file.path(clone_dir, "stan", "lib", "stan_math"))
  cmdstanr::rebuild_cmdstan(dir = clone_dir, cores = cores)
}

Though if you share a lot, it might be better for others if you make a tar.gz, upload it to your clone and link the tar.gz via release_url. This will be much faster for the others as git clone is super slow.

[mitzimorris]

This is what I use when I need to:

very cool. translating to Python should be easy, no?

[rok-cesnovar]

I guess it's possible with https://www.pygit2.org/ but I have no idea good it is.

[rok-cesnovar]

@bbbales2 If you only work with cmdstan branches this might be shorter.

clone_cmdstan <- function(
    clone_dir,
    cmdstan_url = "http://github.com/stan-dev/cmdstan",
    cmdstan_branch = "develop",
    cores = 4) {
  if (!dir.exists(clone_dir)) {
    dir.create(clone_dir)  
  }
  git2r::clone(url = cmdstan_url, branch = cmdstan_branch, local_path = file.path(clone_dir))
  processx::run("make", args = "stan-update", wd = clone_dir, echo =  TRUE, error_on_status = TRUE)
  cmdstanr::rebuild_cmdstan(dir = clone_dir, cores = cores)
}

update_clone <- function(
  clone_dir,
  cmdstan_branch = "develop",
  cores = 4) {
  processx::run("make", args = "clean-all", wd = clone_dir, echo =  TRUE, error_on_status = TRUE)
  git2r::pull(repo = clone_dir)
  processx::run("make", args = "stan-update", wd = clone_dir, echo =  TRUE, error_on_status = TRUE)
  cmdstanr::rebuild_cmdstan(dir = clone_dir, cores = cores)
}

checkout_cmdstan_branch <- function(
  clone_dir,
  cmdstan_branch = "develop",
  cores = 4) {
  processx::run("make", args = "clean-all", wd = clone_dir, echo =  TRUE, error_on_status = TRUE)
  git2r::checkout(object = clone_dir, branch = cmdstan_branch)
  git2r::pull(repo = clone_dir)
  processx::run("make", args = "stan-update", wd = clone_dir, echo =  TRUE, error_on_status = TRUE)
  cmdstanr::rebuild_cmdstan(dir = clone_dir, cores = cores)
}

[bbbales2]

Excellent excellent excellent! This will work great!