Cmdstanr: Regex parameter selection

[bnicenboim]

Describe the bug

Say that I have a model m, and then I store the samples in fit by doing m$sample.
If I have a not too small fit object, fit will show me the summary without problems and super fast, but
fit$summary() or fit$summary(NULL) is really really slow.

The thing is that I need to extract some variable names with regular expressions, and I can't do it directly from fit$summary().

To Reproduce
I think this is quite straightforward, but if files are needed I'll upload them somewhere

Expected behavior
fit$summary() should take the same time to show results as fit.

Or there should be a way for me to extract variables from summary directly without the need to extract all the variables first, e.g,,
fit$summary(c("alpha","beta", regex = TRUE)

Operating system
Ubuntu 20.04.1 LTS

CmdStanR version number
0.2.1

[bnicenboim]

just for completeness, a workaround is:

vars <- fit$metadata()$stan_variables
pars <- fit$summary(vars[grep("(alpha|beta).*", vars)]) 

[rok-cesnovar]

Just using fit calls fit$print() which is optimized to only calculate stuff for anything that is printed (which if I recalled correctly is 10 variables). fit$print() does not return anything.

fit$summary() is meant to return the summaries for all parameters (or for all specified by variables). summary mostly just calls posterior::summarise_draws() for which we know is a bit slow: https://github.com/stan-dev/posterior/issues/98.

So this means that apart from speeding up summarise draws this is more of a feature requst for adding regex to variable selection, right?

Are you variables alpha_something or are they just vectors (alpha[1], alpha[2], ...) ?

Thanks!

[bnicenboim]

yeah, mine are alpha_something and I guess it's a feature request :)

I would do it myself as a PR, but I don't quite get R6, sorry :/

I guess, if there's a regex argument, and pars stores the parameters, what I'm asking is simply:

vars <- fit$metadata()$stan_variables
pars <- fit$summary(vars[grep(pars, vars)],...) 

[rok-cesnovar]

That or we allow regex as
fit$summary(variables = "(alpha|beta).*") but we would need to specify its a regex somehow.

as we have stan_variables inside read_cmdstan_csv() anyways.

[jgabry]

Yeah like @rok-cesnovar said print and summary are designed a bit differently intentionally (print to just compute what gets printed to the screen, summary to compute everything for all the requested variables even if they don't fit in the printed output). I think we want to keep this distinction because changing it would result in print getting slower not summary getting faster. But we should definitely add something to the doc about this.

I'm also ok with adding some regex functionality for selecting parameters. posterior::subset_draws() already has regex for selecting parameters once you have all of them, but in cmdstanr we could do better by only reading in the parameters selected by regex.

[jgabry]

I also just renamed this issue to "regex parameter selection" since that's now the topic being discussed here.

[jgabry]

I just opened a PR that updates the doc to explain the speed difference between print and summary

[mitzimorris]

"pattern" is generally used over "regex".

The thing is that I need to extract some variable names with regular expressions, and I can't do it directly from fit$summary()

strongly suggest features like this be in add-on packages - incurs technical debt and requires both fiddling with I/O and dependency on a regex package.

[rok-cesnovar]

Base R has enough capabilites that this does not require additional packages. However, if we fix #430 we can just provide an example of how someone can easily do this on their own with something like

params <- mod$params() # or whatever will be the API
regexed_params <- params[grepl(pattern = "theta.*", params)]
fit$draws(variables = regexed_params)

its not necessary for us to provide an API for this given the simplicity of the above.