Data.table: x.* column not found during join with by=.EACHI

[franknarf1]

Here's where I see it happening:

library(data.table)
DT = data.table(date = as.IDate(Sys.Date()) + 0:2)
eDT = data.table(e_date = as.IDate(Sys.Date()))

DT[eDT, on=.(date >= e_date), {
    target_date = setattr(if (e_date[1L] %in% x.date) e_date[1L] else max(x.date), "class", c("IDate", "Date"))
    target_date
}, by=.EACHI]
# Error in match(x, table, nomatch = 0L) : object 'x.date' not found

DT[eDT, on=.(date >= e_date), {
    x.date = x.date
    target_date = setattr(if (e_date[1L] %in% x.date) e_date[1L] else max(x.date), "class", c("IDate", "Date"))
    target_date
}, by=.EACHI]
#          date         V1
# 1: 2017-01-10 2017-01-10

The first way fails, but the second way (with extra line x.date = x.date) works.

I couldn't repro without setattr ... dunno if that's important to it.

[MichaelChirico]

On current master I get the same output from both & it doesn't match the output from the latter:

Empty data.table (0 rows and 1 cols): date

👀

[MichaelChirico]

Still the following two have different output, not sure why (and also not quite sure why the output has only one row? when I thought it should have 3?)

library(data.table)
DT = data.table(date = as.IDate(Sys.Date()) + 0:2)
eDT = data.table(e_date = as.IDate(Sys.Date()))

DT[eDT, on=.(date >= e_date), {
    target_date = as.IDate(if (e_date[1L] %in% x.date) e_date[1L] else max(x.date))
    target_date
}, by=.EACHI]
#          date         V1
# 1: 2019-09-07 2019-09-07

library(data.table)
DT = data.table(date = as.IDate(Sys.Date()) + 0:2)
eDT = data.table(e_date = as.IDate(Sys.Date()))

DT[eDT, on=.(date >= e_date), {
    target_date = setattr(if (e_date[1L] %in% x.date) e_date[1L] else max(x.date), 'class', c('IDate', 'Date'))
    target_date
}, by=.EACHI]
# Empty data.table (0 rows and 1 cols): date

[MichaelChirico]

Oh... it's because setattr returns invisible():

x = 4
x = setattr(x, 'class', 'foo')
x
# NULL

Proper way is probably with as.IDate. With setattr:

DT[eDT, on=.(date >= e_date), {
    target_date = if (e_date[1L] %in% x.date) e_date[1L] else max(x.date)
    setattr(target_date, 'class', c('IDate', 'Date'))
    target_date
}, by=.EACHI]
#          date         V1
# 1: 2019-09-07 2019-09-07

[MichaelChirico]

I tried git bisecting to find out where the fix was but came up short.

On 1.11.2, the above works as expected already once the setattr bit is fixed.

On 1.11.0, we get a totally different error:

Error in `[.data.table`(DT, eDT, on = .(date >= e_date), { : 
  SETLENGTH() cannot be applied to an ALTVEC object.

Doing the cherry picking etc required to track down the fix now is overkill; closing

[jangorecki]

@MichaelChirico could we close this by including unit test and news entry?

[MichaelChirico]

yes good point