Gpytorch: [Bug] Upstream changes to tensor comparisons breaks things

[Balandat]

🐛 Bug

After https://github.com/pytorch/pytorch/pull/21113 a bunch of tests are failing b/c of the change in tensor comparison behavior (return type from uint8 to bool). Creating this issue to track the fix.

[Balandat]

It appears most of this happens in batch_symeig where we do 1 - mask - will put up a fix shortly.

[izdeby]

@Balandat, we have introduced '~' operator for masks to fix this issue. You should be able to just replace '1-mask' with '~mask'

[Balandat]

Sure that works. There are a few other places in the code where this causes failures, but they should be easy enough to fix.

[Balandat]

Fixed by #823

[Balandat]

Ok of course there are now a bunch more warnings emitted when using uint8 on pytorch master, so all the tests that count the warnings fail there ...

[Balandat]

It's worse than that, the change actually breaks the jit script code for _jit_linear_cg_updates. So the correct thing is to use bool everywhere, and fix the tests that count the warnings...

[Balandat]

Won't be able to fix all this properly right now, deferring to the above issue.

[jacobrgardner]

@Balandat I'll take a look once the nightly build comes out and I can repro the errors without having to build pytorch from source.

[Balandat]

@jacobrgardner sounds good, if you base this on #825 at least you won't have to deal with the warnings.

[Balandat]

Fwiw, here's a notebook that shows the failure (commented out the jit decorators locally).

It only happens when broadcasting. @izdeby, maybe broadcasting semantics for bool tensors are different?

test_broadcasting_issues.ipynb.txt

[izdeby]

@Balandat, what exactly failed in broadcasting for you? there shouldn't be any difference in broadcasting semantics for bool tensors

[Balandat]

@izdeby looks like this is a different issue related to changing sizes in addcmul, see #827

[jacobrgardner]

For now do we think we could define like a gpytorch.bool type to be torch.bool if that will work (i.e. we are on master), and torch.uint8 otherwise?

[jacobrgardner]

We could similarly define gpytorch.not to do ~ and 1 - ... respectively

[Balandat]

So ~ will work even with uint8, so there is no need for that. But we can do the gpytorch.bool thing to avoid the mass of warnings

[Balandat]

made this change in the latest update to #827

[jacobrgardner]

Reasonably fixed for now via #827