Numpyro: BernoulliProbs

[hyperfra]

Hi
I am trying to run a probit. Question: is it that your BernoulliProbs has a constrained on the argument probs which is (0,1) ? I am asking because I am getting as an error

ValueError: The parameter probs has invalid values

In the Bernoulli 0<=p<=1 so 0 and 1 should be valid.

(I am passing as probs cdf(beta*x) where pdf is jax.scipy.stats.norm import cdf)

thanks

[fehiepsi]

Hi @hyperfra, please feel free to open a PR to relax the _Interval constraint to a closed interval: return (x >= self.lower_bound) & (x <= self.upper_bound) (or I can do if you want) if you found that it is helpful to avoid the error. I suspect that the issue comes from something else because we only validate the model when we can't find valid initial parameters.

[hyperfra]

thank you fehiepsi,

so we are both "partially" right. The context is that I am reproducing regressions from Wooldridge textbook to test, Table 15.1 p580.

1. So If I constrain to a open interval the probit model does not run:
   ValueError: The parameter probs has invalid values
2. If I constrain to a closed interval it runs and gives me the correct results.

Notice that is true only if I normalize the variables. (Wooldrige does not normalize as it is classical max likelihood) . If I try to run the model with the un-normalised variables with the open set the issue is again the probs having invalid values and with the closed interval the error becomes what you were referring to: he cannot find initial values. But this is a different issue solved by the normalisation. It looks like checking probs occurs before. So in any case the correct interval is close for the Bernoulli.

is _Interval used in other places where the open interval would be correct ?

[fritzo]

It looks like the original torch.distributions.constraints._Interval is indeed a closed interval.

Note: regarding wither to distinguish OpenInterval vs ClosedInterval vs HalfOpenInterval. The open/closed question has come up repeatedly; we have not found a compelling compromise along the simplicity-expressivity spectrum; and we have therefore mostly avoided creating a proliferation of open/closed classes. The issue gets even more complex in multiple dimensions, e.g. the support of the Stable(alpha,beta,mu) distribution includes a nontrivial subset of its boundary

x ∈ [μ, +∞) if α < 1 and β = 1
x ∈ (-∞, μ] if α < 1 and β = −1
x ∈ R otherwise

Arguments against open/closed distinction include: "it [Lebesgue-]almost never matters", "floating points have finite resolution", and "we clamp for numerical stability anyway". Arguments for open/closed distinction include "constraints should transform faithfully" (e.g. under ExpTransform).

[hyperfra]

@fritzo

I imagine the difficulties. In this particular case you need to correct it as PHI(beta*x) can easily be equal to 0 or 1 (at least numerically) which with the open interval prevents the estimation of the probit model.

in any case let me say how much I appreciate what you are doing : it is really great work

[fehiepsi]

Re open vs closed: I cannot recall the reason why we used open interval. We also got some issues recently with Simplex constraint and relaxed the condition fix the issue. I'll walk over the current constraints to make them match Pyro's ones.