Data.table: Package does not run with R version 3.5.0

[benjaminschlegel]

Since I updated to R version 3.5, the package does not run anymore. I found that CRAN chows an error on the status of the package: https://cran.r-project.org/web/checks/check_results_data.table.html. Can you fix that?

[HughParsonage]

Yes we are preparing the next release which should reach CRAN in the first week of May.

[jangorecki]

@benjaminschlegel Please reopen if you are able to reproduce that on latest development version.

[pssguy]

FYI I have tried the new development version without any joy. I also tried referring back to the
CRAN version - again with error

library(data.table, lib.loc = "C:/Users/myinfo/Documents/R/win-library/3.4")
Error: package or namespace load failed for ‘data.table’:
 package ‘data.table’ was installed by an R version with different internals; it needs to be reinstalled for use with this R version

I had hoped that would handle it for next week or so

[MichaelChirico]

ALTREP rears its head

[HughParsonage]

That error seems to suggest you are trying to access the R 3.4 version using R 3.5. That won't work because binary packages aren't compatible across major versions of R.

In terms of the development version, how are you attempting to install it? At the moment it's a bit tricky because the appveyor process that produces the binaries can only produce binaries for R 3.4 and R 3.6. I can install the package from source, would that work for you?

[m0nhawk]

I have updated to R 3.5.0 yesterday and I'm able to install data.table on Windows.

The following works for me: ensure you have the latest Rtools, be sure to have checked "Add rtools to system PATH".

Then start R session (or restart, to read updated PATH) and install from source:

install.packages("data.table", type="source")

With this I'm able to run data.table code.

The only additional thing I've done, is reinstalling all packages before:

package_df <- as.data.frame(installed.packages("~/R/win-library/3.4/"))
package_list <- as.character(package_df$Package)
install.packages(package_list)

Hope this will help someone.

[assaforon]

I'm sorry, data.table has become my hands and eyes to handle data at my work. First week of May is way too late.
As luck has it, yesterday I got a new work Surface and had to install R the very first day 3.5.0 was public. Wasted about an hour on this. Now going to install 3.4.4 so that I can work. It'll take another half-hour. This morning is shot.

By the way, I've been teaching my students (following tips from R Core team) to always skip the x.y.0 versions, because they are the most problematic transitions. Not worth the effort - in the best case it's relatively stable and so those skipping it will be out-of-date for 2-3 months till the x.y.1 version comes out. In the more usual case, you've saved yourself agony and wasted time, and the x.y.1 version will be out within a few weeks.

[assaforon]

Btw, does this mean that if I'm a package author my package will also error out if I haven't recompiled it myself for 3.5.0? I don't recall getting any notification about that from the R core team.
Or did they run a blanket auto process for everyone, but for some reason it cannot handle data.table?

Just checked... my 2 little packages check out ok. I guess you have to have some additional non-R code to make the new system err.

[tungmilan]

Confirm that I was able to install and load CRAN version for R 3.5.0 but loading development version (https://github.com/Rdatatable/data.table/commit/2ed76c346d09fb5d7d9f8231a20571b713b511a9) threw error:

Error: package or namespace load failed for ‘data.table’:
 package ‘data.table’ was installed by an R version with different internals; it needs to be reinstalled for use with this R version

[jangorecki]

@tungmilan are you attempting to install windows binaries from appveyor which was build on 3.4 / 3.6 or you build it in 3.5? This is crucial info. Also important are any issues when installing from source (of course having proper Rtools version installed on Windows) because they are not related to packaging environment but actually to data.table.

[tungmilan]

@jangorecki : I download the zip file from here https://ci.appveyor.com/project/Rdatatable/data-table/build/job/ah7trschpu4onbgg/artifacts. Is it not the right one for R 3.5.0? Thanks!

[jangorecki]

@tungmilan no, appveyor "release" job is still on R 3.4.4. R 3.5.0 was just released, upgrading CI is not instant process. For Ubuntu apt-get does not yet provide R 3.5.0 so Linux jobs has not been upgraded to 3.5 either. If you jump onto R 3.5.0 days after release you should be ready to install R packages (not just data.table) from sources. It was always like that.

[metricsSO]

I was able to install data.table in R 3.5.0 (Windows). What I did is remove all old packages, RStudio, and R 3.4.3 and install R 3.5.0 as if I was installing R for the first time (also install Rtools 3.5 and RStudio.)

[jangorecki]

It shouldn't make such burden. Just ensure to have proper version of Rtools for R 3.5 and it is properly added to PATH (not overwritten by old Rtools also).

[assaforon]

@jangorecki I did that twice and it didn't work.
Bottom line, stick with 3.4.x until package authors resolve this. There's no meaningful material change to either R or data.table, they just changed the way packages are compiled to make them run faster.

[tungmilan]

@metricsSO: that is not necessary. All I did were just put R_LIBS_USER="~/R/win-library/3.4" in .Renviron file in my home directory then run update.packages(checkBuilt = TRUE, ask = FALSE) to update all packages. My previous codes ran flawlessly with the new update. The only trouble I had was the development version of data.table but I can live with the CRAN version for now.

@assaforon : there are some significant changes in R 3.5.0 that potentially make codes run much faster yet use much less memory
http://blog.revolutionanalytics.com/2018/04/r-350.html

[jangorecki]

@tungmilan thanks for detailed and interesting post, but it is out of topic in this issue, link to this content is perfectly enough.

[tungmilan]

@jangorecki : Thanks for the reminder :)

On topic, I was able to install the development version on my Win 10 laptop w/ RTools 3.5

remove.packages("data.table")
install.packages("data.table", type = "source",
    repos = "http://Rdatatable.github.io/data.table")

Ran example(data.table) after without any error