Data.table: noLD 'additional issue' on CRAN check page

[mattdowle]

Running test id 1729.9      Test 1729.9 ran without errors but failed check that x equals y:
  > x = binary(DT[[1]]) 
  First 6 of 8 (type 'character'): [1] "0 11111100101 111010011010000100010111101110000100 11110100 00001000"
  [2] "1 11111100101 111010011010000100010111101110000100 11110100 00001000"
  [3] "0 00000001001 110000010110110001011100010100100101 00110101 01110001"
  [4] "0 11111110100 001000111010010100010110111010000010 11011001 10111101"
  [5] "0 00000001010 000011011010011101101010100101111100 10111001 10101010"
  [6] "0 11111110100 010111011111100101001110101100000011 01101011 10110000"
  > y = c("0 11111100101 111010011010000100010111101110000100 11110100 00000100",      "1 11111100101 111010011010000100010111101110000100 11110100 00000100",      "0 00000001001 110000010110110001011100010100100101 00110101 01110101",      "0 11111110100 001000111010010100010110111010000010 11011001 10111010",      "0 00000001010 000011011010011101101010100101111100 10111001 10101101",      "0 11111110100 010111011111100101001110101100000011 01101011 10101100",      "0 00000001010 000101000110010100110011101010000110 00111110 01010001",      "0 11111110100 011001101011100100100011110110110000 01001110 01011101") 
  First 6 of 8 (type 'character'): [1] "0 11111100101 111010011010000100010111101110000100 11110100 00000100"
  [2] "1 11111100101 111010011010000100010111101110000100 11110100 00000100"
  [3] "0 00000001001 110000010110110001011100010100100101 00110101 01110101"
  [4] "0 11111110100 001000111010010100010110111010000010 11011001 10111010"
  [5] "0 00000001010 000011011010011101101010100101111100 10111001 10101101"
  [6] "0 11111110100 010111011111100101001110101100000011 01101011 10101100"
  8 string mismatches

Running test id 1830.7      Test 1830.7 ran without errors but failed check that x equals y:
  > x = identical(fread("F\n1.1\n+1.333333333333333\n5.9e300\n45609E11\n-00890.e-003\n"),      data.table(F = c(1.1, 1.33333333333333, 5.9e+300, 4.5609e+15,          -0.89))) 
  First 6 of 1 (type 'logical'): [1] FALSE
  > y = TRUE 
  First 6 of 1 (type 'logical'): [1] TRUE
  1 element mismatch

  Running test id 1830.8      Test 1830.8 ran without errors but failed check that x equals y:
  > x = identical(fread("G\n0.000000000000000000000000000000000000000000000000000000000000449548\n"),      data.table(G = c(4.49548e-61))) 
  First 6 of 1 (type 'logical'): [1] FALSE
  > y = TRUE 
  First 6 of 1 (type 'logical'): [1] TRUE
  1 element mismatch


Error in eval(exprs[i], envir) : 
    3 errors out of 7860 in 00:01:57 on Thu Dec 20 17:25:23 2018. [endian==little, sizeof(long double)==0, sizeof(pointer)==8, TZ=Europe/London, locale='LC_CTYPE=en_GB.utf8;LC_NUMERIC=C;LC_TIME=en_GB.UTF-8;LC_COLLATE=C;LC_MONETARY=en_GB.UTF-8;LC_MESSAGES=en_GB.UTF-8;LC_PAPER=en_GB.UTF-8;LC_NAME=C;LC_ADDRESS=C;LC_TELEPHONE=C;LC_MEASUREMENT=en_GB.UTF-8;LC_IDENTIFICATION=C', l10n_info()='MBCS=TRUE; UTF-8=TRUE; Latin-1=FALSE']. Search inst/tests/tests.Rraw for test numbers: 1729.9, 1830.7, 1830.8.
  Calls: test.data.table -> sys.source -> eval -> eval
  Execution halted
* checking for unstated dependencies in vignettes ... OK
* checking package vignettes in ‘inst/doc’ ... OK
* checking re-building of vignette outputs ... [18s/20s] OK
* checking PDF version of manual ... OK
* DONE

Status: 1 ERROR
See
  ‘/data/gannet/ripley/R/packages/tests-noLD/data.table.Rcheck/00check.log’
for details.

[MichaelChirico]

:eyes: with interest as I have a noLD problem on another package that I cant seem to reproduce in any of my environments...

[mattdowle]

R can be compiled with --disable-long-double but I've had trouble too in the past.
Extract from https://cran.r-project.org/doc/manuals/r-devel/R-exts.html#Writing-portable-packages

If you must try to establish a tolerance empirically, configure and build R with --disable-long-double and use appropriate compiler flags (such as -ffloat-store and -fexcess-precision=standard for gcc, depending on the CPU type85) to mitigate the effects of extended-precision calculations.

My attempts in the past are here : https://github.com/Rdatatable/data.table/blob/master/CRAN_Release.cmd#L265

[MichaelChirico]

FWIW I managed to set up r-devel and --without-long-double using the r-devel docker image from Dirk:

docker run --rm -it rocker/r-devel /bin/bash

^ then RD --without-long-double is what you want. Hopefully helpful to someone (e.g., future me)

[mattdowle]

@MichaelChirico did you mean --disable-long-double here as I pointed to above, not --without-long-double ?

[MichaelChirico]

yes 😅

[jangorecki]

@MichaelChirico if R has to be compiled with --disable-long-double, how it works on that docker image for you? https://hub.docker.com/r/rocker/r-devel/dockerfile doesn't have such compilation flag

[MichaelChirico]

Yes comment is a bit outdated. actually I don't know of any docker with a good noLD binary and even the build commands in CRAN_release.cmd weren't working for me last time I tried.

I was able to use rhub but IIRC it wasn't great for debugging, only for running the test suite. would be great if Winston's image would add a noLD binary.

Besides noLD I'm using

• rocker/r-ver for version-specific issues
• wch1/r-debug for valgrind/SAN stuff

link about rhub noLD:

https://blog.r-hub.io/2019/05/21/nold/

[jangorecki]

We could add a built including such option.
We already have strict-barrier option in r-devel image, which is re-built daily, not a 9 months(?!) old rocker image.

[mattdowle]

@jangorecki Yep: adding --disable-long-double to the strict build sounds good to me. It feels like R-devel has had some configuration changes in the last few months. It works better now and I was able to simplify the commands in the CRAN_Release.cmd. e.g. don't need to pass CFLAGS= to ./configure anymore. But if you do, as before, then it doesn't work. Or at least, that what it felt like. Once it worked I didn't investigate further.

[jangorecki]

Yes but we want to have r-devel with long double too, right? I think I need to craft another image for that.

[mattdowle]

LD is already tested in r-release builds so that's fine I think. It would be better to do as you suggest. But this is a good opportunity to cut that corner because other things are much higher priority. Especially if it needs a new image. That's more work to setup and extends GLCI time every time, and those are costs to balance against the benefit.

[jangorecki]

@mattdowle
added --disable-long-double to r-devel image: https://gitlab.com/jangorecki/dockerfiles/commit/67e74f723d6be4b4eb91bb1775f4b4b876ad0111
re-run r-devel job on recent master, test script Rout footer:

sizeof(long double)==0

[MichaelChirico]

FWIW i think the "proper" test is:

capabilities('long.double')

[jangorecki]

OK, the other one we already have in Routs of test scripts