Data.table: Segfault with clang 11 on Fedora or Debian Linux

[boxuancui]

Hi data.table maintainers, I am the author of DataExplorer, and during my latest submission to CRAN, they notified me of the following error. It happens on Debian/Fedora clang 11 only, and only during the vignettes compilation. While I do not know how to re-produce this, since:

1. I do not have a clang 11 Debian/Fedora machine.
2. I did many rounds of testing on RHub, and nothing like this were reported.

I am wondering if you guys have any clues or know what could potentially be a next step for me. I also tried to search on past issues and noticed something similar in #2707. Not sure if it is valid, but looks like something similar. Thank you!

Error(s) in re-building vignettes:
--- re-building ‘dataexplorer-intro.Rmd’ using rmarkdown

* caught segfault *
address (nil), cause 'unknown'

Traceback:
1: fsort(as.numeric(irows))
2: [.data.table(dt, variable %in% feature_names[x])
3: dt[variable %in% feature_names[x]]
4: ggplot(dt[variable %in% feature_names[x]], aes(x = value))
5: FUN(X[[i]], ...)
6: lapply(X = X, FUN = FUN)
7: .lapply(parallel = parallel, X = layout, FUN = function(x) {
ggplot(dt[variable %in% feature_names[x]], aes(x = value)) +
do.call("geom_histogram", c(na.rm = TRUE, geom_histogram_args)) +
do.call(paste0("scale_x_", scale_x), list()) + ylab("Frequency")})
8: plot_histogram(final_data, nrow = 3L, ncol = 3L)

[MichaelChirico]

Hi Boxuan, have you installed against latest master? this was merged last week & I think is relevant:

https://github.com/Rdatatable/data.table/pull/4808

[jangorecki]

Agree with Michael.
Traceback shows fsort clearly, and this is what we observed as well, see https://github.com/Rdatatable/data.table/issues/4786.
You may be able to reproduce the problem using this docker image which uses clang-11

docker run -it registry.gitlab.com/jangorecki/dockerfiles/r-devel-clang /bin/bash

You may need to run it on multiple core (5-6 threads should do) machine.

I submitted question to r-hub about clang-11 job but they seems to be unresponsive on github: https://github.com/r-hub/rhub-linux-builders/issues/53

[boxuancui]

Thank you both! Do you roughly know when v1.13.3 will be released to CRAN, so that I can plan for my submission as well? Looks like I can't release with remote dependencies.

[jangorecki]

@boxuancui It is possible to release to CRAN with a remote dependency, although this is officially supported only for Suggested dependencies and your package do import data.table. The fact that a dependency is on CRAN already could eventually make that possible also for Imported dependency. The change would look like:

Imports:
  data.table (>= 1.13.3)
Additional_repositories: https://Rdatatable.gitlab.io/data.table

Anyway, I recommend not to release to CRAN like that but wait for data.table 1.13.4 to be released. @mattdowle any idea when that could happen?

[boxuancui]

@jangorecki Thanks for the info! However, my package was recently archived because I didn't fix a check results error in time. I really would like to have the package back online as soon as possible. However, if 1.13.4 is not that far away, I can certainly wait.

[mattdowle]

@boxuancui Really sorry for the impact. 1.13.4 should be on CRAN within a few weeks. When you resubmit your package please include data.table (>= 1.13.4) in DESCRIPTION. We saw recently that CRAN removed another package (POUMM) for similar reasons (data.table impacting it), and with only 2 weeks notice too. In your case, just why a package has to be removed from CRAN given that it only fails in the very latest version of clang, seems excessive to me.

[boxuancui]

@mattdowle Totally agreed! And FYI, my package was working fine. It just generated 2 warnings during vignette building! Anyway, thank you all so much for everyone's help! I will wait for 1.13.4.

[mattdowle]

@boxuancui 1.13.4 has been submitted to CRAN today. It can take 3-7 days from here to settle down on CRAN.