Cmdstanr: Getting error 'Error: Can't find columns `lp__`, `treedepth__`, `divergent__` in `.data`.'

[breckbaldwin]

Describe the bug
Hope this is clear enough, I _think_ this is a cmdstanr issue, I ran with cmdstan-2.20.0 as well and got the same bug.

Have small Stan program that used to work fine. Now I can't get results. I get 'Error: Can't find columns lp__, treedepth__, divergent__ in .data.' after running

cmdstanr::set_cmdstan_path("~/.cmdstanr/cmdstan-2.23.0/") model_0 <- cmdstan_model("bug.stan") fit_0 <- model_0$sample(output_dir = ".")

Stan appears to be converged and output looks ok for the chains. I can read in a the .csv file fine with rstan. See below.

To Reproduce

Calling cmdstanr code
`
library("cmdstanr")
cmdstanr::set_cmdstan_path("~/.cmdstanr/cmdstan-2.23.0/")

model_0 <- cmdstan_model("bug.stan")
fit_0 <- model_0$sample(output_dir=".")

above completes with message:Error: Can't find columns lp__, treedepth__, divergent__ in .data.

print(fit_0$summary()) # fit_0 is not instantiated

library(rstan)
stanfit <- rstan::read_stan_csv("bug-202007091947-1-aecd06.csv") #change to correct file, should be sister to bug.stan
print(stanfit) #works

`

bug.stan

`parameters {
real estimate_me;
}

model {
1 ~ normal(estimate_me,1);
}
`
Expected behavior
'fit_0 <- model_0$sample(output_dir=".") ' should not generate error and fit_0$summary() should work

Operating system

$version.string
[1] "R version 3.6.2 (2019-12-12)"
$platform
[1] "x86_64-apple-darwin15.6.0"

CmdStanR version number
Your CmdStanR version number (e.g. from packageVersion("cmdstanr")).
‘0.0.0.9008’

Additional context

[schliebs]

I today got the exact same error on one of four different machines (fresh install of cmdstanr, CentOS7 RStudio Server Cluster).
A fix that worked for me was re-installing cmdstanr from github via devtools and accepting all proposed packages to be updated (1 and Enter when prompted).

[rok-cesnovar]

Thanks for the report. I am unable to reproduce on my system so need a bit more info.

Does reading csv afterwards works?

Like this:

model_0 <- cmdstan_model("bug.stan")
fit_0 <- model_0$sample(output_dir=".", validate_csv = FALSE)
draws <- read_cmdstan_csv(fit$output_files())

[breckbaldwin]

Still get the error but it happens trying to access fit_0. See below:

model_0 <- cmdstan_model("bug.stan")
Model executable is up to date!
fit_0 <- model_0$sample(output_dir=".", validate_csv = FALSE)
Running MCMC with 4 sequential chains...

converges just fine.

draws <- read_cmdstan_csv(fit_0$output_files())
Error: Can't find columns lp__, estimate_me, accept_stat__, stepsize__, treedepth__, … (and 3 more) in .data.
Run rlang::last_error() to see where the error occurred.

just accessing fit_0 generates same error

fit_0
Error: Can't find columns lp__, estimate_me, accept_stat__, stepsize__, treedepth__, … (and 3 more) in .data.
Run rlang::last_error() to see where the error occurred.

[breckbaldwin]

I can try and install from devtools as suggested by @schliebs but I may loose the ability to generate the bug if that is important.

[schliebs]

Unfortunately, since the re-installing filled up my console I cannot go back to the error message, but it seems like some parts of it were related to vroom. @breckbaldwin can you maybe run rlang::last_error() and paste the full error chain?

[rok-cesnovar]

Yeah, “just accessing” is actually the same as running fit$print().

I am guessing this works:

model_0 <- cmdstan_model("bug.stan")
fit_0 <- model_0$sample(output_dir=".", validate_csv = FALSE)
stanfit <- rstan::read_stan_csv(fit_0$output_files())

If that works can you try upgrading the rlang package first and checking if the bug is gone.
If not, also upgrade vroom.

We might be missing a minimum version requirement on some package.

[schliebs]

That matches my recollection. It seems like rlang was in the list of packages I updated, while vroom was not. (~80-90% confidence)

[rok-cesnovar]

Thanks @schliebs !

If its rlang we will need to bring that up to the posterior package repo.

[breckbaldwin]

Yeah, “just accessing” is actually the same as running fit$print().

I am guessing this works:

model_0 <- cmdstan_model("bug.stan")
fit_0 <- model_0$sample(output_dir=".", validate_csv = FALSE)
stanfit <- rstan::read_stan_csv(fit_0$output_files())

If that works can you try upgrading the rlang package first and checking if the bug is gone.
If not, also upgrade vroom.

We might be missing a minimum version requirement on some package.

No luck. Removed and installed 'rlang', same error.
Just ran package.install('vroom'), still doesn't work.
Restarted RStudio.

@schliebs asked for rlang::last_error()

Can't find columns lp__, estimate_me, accept_stat__, stepsize__, treedepth__, … (and 3 more) in .data.
Backtrace:

1. cmdstanr::read_cmdstan_csv(fit_0$output_files())
2. vroom::vroom(...)
3. vroom:::vroom_select(out, col_select, id)
4. tibble:::[.tbl_df(x, vars)
5. tibble:::check_names_df(i, x)
6. tibble:::check_names_before_after_character(j, names(unclass(x)))
   Run rlang::last_trace() to see the full context.

My current session info.

sessionInfo()
R version 3.6.2 (2019-12-12)
Platform: x86_64-apple-darwin15.6.0 (64-bit)
Running under: macOS High Sierra 10.13.6

Matrix products: default
BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/3.6/Resources/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats graphics grDevices utils datasets methods base

other attached packages:
[1] cmdstanr_0.0.0.9008 rstan_2.19.3 ggplot2_3.2.1
[4] StanHeaders_2.21.0-1

loaded via a namespace (and not attached):
[1] Rcpp_1.0.3 pillar_1.4.3 compiler_3.6.2
[4] prettyunits_1.0.2 tools_3.6.2 bit_1.1-15.2
[7] pkgbuild_1.0.6 checkmate_1.9.4 lifecycle_0.1.0
[10] tibble_2.1.3 gtable_0.3.0 pkgconfig_2.0.3
[13] rlang_0.4.7 cli_2.0.0 rstudioapi_0.10
[16] yaml_2.2.0 parallel_3.6.2 loo_2.2.0
[19] gridExtra_2.3 withr_2.1.2 dplyr_0.8.3
[22] bit64_0.9-7 stats4_3.6.2 grid_3.6.2
[25] tidyselect_0.2.5 glue_1.3.1 inline_0.3.15
[28] R6_2.4.1 processx_3.4.1 fansi_0.4.0
[31] vroom_1.2.1 purrr_0.3.3 callr_3.4.0
[34] magrittr_1.5 backports_1.1.5 matrixStats_0.55.0
[37] scales_1.1.0 ps_1.3.0 assertthat_0.2.1
[40] colorspace_1.4-1 lazyeval_0.2.2 munsell_0.5.0
[43] crayon_1.3.4

[rok-cesnovar]

@breckbaldwin could you just update the cmdstanr to the latest github version without updating other packages and check if that solves the issue?

[saudiwin]

I have the same error and installed cmstanr from Github. Same error. Also have rlang and vroom up to date.

[saudiwin]

Running cmdstanr on Ubuntu 18 with R 3.6.

[jgabry]

Hmm, thanks for sharing and sorry you're getting an error! R 3.6 seems to be the common denominator between this post here and this one from discourse: https://discourse.mc-stan.org/t/subset-error-with-new-cmdstanr-version/16884. But I have no idea why. I don't think we're using anything from vroom or rlang that changed between 3.6 and 4.0 but maybe I'm wrong and we missed something. Also it was working for me with R 3.6 before I updated to 4.0, so it's probably not just R 3.6!

Is the traceback the same as above in https://github.com/stan-dev/cmdstanr/issues/243#issuecomment-656894455? (terminating with tibble:::check_names_before_after_character(j, names(unclass(x))))

[saudiwin]

Here's the traceback:

<error/rlang_error>
Can't find columns `lp__`, `poly1.1`, `poly1.2`, `poly1.3`, `poly1.4`, … (and 468 more) in `.data`.
Backtrace:
  1. (function (x, ...) ...
  2. R6:::print.R6(x)
  3. .subset2(x, "print")(...)
  4. self$metadata()
  5. private$read_csv_()
  6. cmdstanr::read_cmdstan_csv(...)
  9. vroom::vroom(...)
 10. vroom:::vroom_select(out, col_select, id)
 12. tibble:::`[.tbl_df`(x, vars)
 13. tibble:::check_names_df(i, x)
 14. tibble:::check_names_before_after_character(j, names(unclass(x)))

Can't find columns `lp__`, `poly1.1`, `poly1.2`, `poly1.3`, `poly1.4`, … (and 468 more) in `.data`.
Backtrace:
     █
  1. ├─(function (x, ...) ...
  2. └─R6:::print.R6(x)
  3.   └─.subset2(x, "print")(...)
  4.     └─self$metadata()
  5.       └─private$read_csv_()
  6.         └─cmdstanr::read_cmdstan_csv(...)
  7.           ├─base::suppressWarnings(...)
  8.           │ └─base::withCallingHandlers(expr, warning = function(w) invokeRestart("muffleWarning"))
  9.           └─vroom::vroom(...)
 10.             └─vroom:::vroom_select(out, col_select, id)
 11.               ├─x[vars]
 12.               └─tibble:::`[.tbl_df`(x, vars)
 13.                 └─tibble:::check_names_df(i, x)
 14.                   └─tibble:::check_names_before_after_character(j, names(unclass(x)))

[saudiwin]

So the error is definitely in vroom and some issue with tibble.

[jgabry]

Thanks, yeah that appears to be the case. Which version of tibble do you have?

[jgabry]

@breckbaldwin I see you have a very old version of the tibble package (tibble_2.1.3 but they're past 3.0 now). Can you try install.packages("tibble") and then see if you still get this error?

[jgabry]

Unfortunately adding a single dependency on vroom (which seemed like a good idea since it's _much_ faster than read.csv()) is really like adding a _lot_ of dependencies with how intertwined the tidyverse/r-lib/etc packages are with each other. I really wanted to avoid depending on any of those packages but we don't have the person hours to commit to writing our own fast csv reader!

[saudiwin]

I just updated tibble to 3.0.3, still no dice. However, the backtrace did change, now showing an error in the vctrs package which tibble depends on.

Backtrace:
     █
  1. ├─(function (x, ...) ...
  2. └─R6:::print.R6(x)
  3.   └─.subset2(x, "print")(...)
  4.     └─self$metadata()
  5.       └─private$read_csv_()
  6.         └─cmdstanr::read_cmdstan_csv(...)
  7.           ├─base::suppressWarnings(...)
  8.           │ └─base::withCallingHandlers(expr, warning = function(w) invokeRestart("muffleWarning"))
  9.           └─vroom::vroom(...)
 10.             └─vroom:::vroom_select(out, col_select, id)
 11.               ├─x[vars]
 12.               └─tibble:::`[.tbl_df`(x, vars)
 13.                 └─tibble:::vectbl_as_col_location(...)
 14.                   ├─tibble:::subclass_col_index_errors(...)
 15.                   │ └─base::withCallingHandlers(...)
 16.                   └─vctrs::vec_as_location(j, n, names)
 17.                     └─(function () ...
 18.                       └─vctrs:::stop_subscript_oob(...)
 19.                         └─vctrs:::stop_subscript(...)

[saudiwin]

For anyone following along at home, it is easy to work around this by including this option to the $sample method:

output_dir=".", validate_csv = FALSE,

and then running this piece of code to load the CSVs with rstan:

  get_mod <- rstan::sflist2stanfit(lapply(us_fit_scale_mod$output_files(),
                                          rstan::read_stan_csv))

Of course you then have to use rstan functions to continue to parse the model object.

[jgabry]

Thanks for sharing the updated backtrace. I'll dig into it a bit and report back if I figure something out.

get_mod <- rstan::sflist2stanfit(lapply(us_fit_scale_mod$output_files(),
rstan::read_stan_csv))

Yeah this is a good workaround for the time being. I think rstan::read_stan_csv() should read multiple csv files so you can probably just use rstan::read_stan_csv(fit$output_files()) and avoid also calling sflist2stanfit.

Also, I'm 99% sure that upgrading to R >= 4.0 would solve this, but I totally understand if you're hesitant to do that (I usually wait a while before updating).

[saudiwin]

The costs of the workaround are less than the costs of upgrading R :)

[rok-cesnovar]

@saudiwin if you install cmdstanr from Github now

devtools::install_github("stan-dev/cmdstanr")

you no longer need the workaround.

[saudiwin]

Thanks! @rok-cesnovar @jgabry

[jgabry]

@saudiwin If by any chance you haven't updated R yet, can you confirm that installing the latest cmdstanr from github works? That is, does it correctly fall back to using read.csv() with an informative warning message? (If you've already updated your R and it's working with vroom, then no worries).

[saudiwin]

Yep patch works, thanks much!

[jgabry]

Great, thanks for trying it out!

[breckbaldwin]

I tried updating tibble, updating RStudio--running ancient version and still had the same error.
Update to R 4.0.2 and all is well.

I am happy to close the issue. Not clear on etiquette.

thanks for the effort @jgabry, @rok-cesnovar, @schliebs and @saudiwin.

Breck

[rok-cesnovar]

I think we can close it yes as the current version checks if vroom fails and back falls to utils::read.csv. We should find a better way of fixing this issue than "upgrade to R4.x", but since users will not see the error in the title anymore we can close this.

Thanks @breckbaldwin for bringing this to our attention!