Probability: Feature request: Negative binomial with mean and dispersion parameterization

[gokceneraslan]

For many people working with count data (especially in ecology and genomics), it is very common to parameterize negative binomial with mean (total_count*exp(logits)) and dispersion (1/total_count). Would it be possible to add this reparametrization of NB to tfp?

[brianwa84]

It seems straightforward to do the reparameterization since one can infer
the existing parameters from those you identify. Are there any numerical
stability benefits or downsides to this parameterization? e.g. It seems
like you might lose precision on total_count if it were represented by it's
inverse between 0 and 1.

https://github.com/tensorflow/probability/blob/c6c14ff4e684ebc2613833d71cb476ba5c27e2a8/tensorflow_probability/python/distributions/negative_binomial.py#L148
is where we do sampling.

On Sun, Apr 21, 2019, 7:53 AM Gökçen Eraslan [e-mail]
wrote:

For many people working with count data (especially in ecology and
genomics), it is very common to parameterize negative binomial with mean (
total_count*logits) and dispersion (1/total_count). Would it be possible
to add this reparametrization of NB to tfp?

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/tensorflow/probability/issues/372, or mute the thread
https://github.com/notifications/unsubscribe-auth/AFJFSI53FUTOFD5SA2VTBYLPRRISFANCNFSM4HHLPWGQ
.

[gokceneraslan]

There is no consensus over how dispersion is defined actually, some people use number of failures directly, some people use the inverse. The former approaches +inf in Poisson regime, but I'm not entirely sure if this is as bad as it approaching zero.

R uses both parameterization in following way: rnbinom(n, size, prob, mu) where size is the target for number of successful trials, or dispersion parameter (the shape parameter of the gamma mixing distribution) and mu is the mean. Either prob or mu has to be specified by the user, but not both. Note that they use number of fails given number of successes kind of NB definition. Details are here.

I think having both parameterizations in tfp would be very convenient, too.

[brianwa84]

I agree, if it's a fairly common parameterization, we should support it. It seems like R does support this, but scipy does not.

If you'd like to send a PR to add either alternative constructor args, I think we'd be open to it (make sure to raise if >1 parameterization is actually used). It would be good if you could could put a reference or two in the docstring showing how the new parameterization is used in practice in a couple compbio/eco papers? I found this one: https://stackoverflow.com/questions/40846992/alternative-parametrization-of-the-negative-binomial-in-scipy

We try to be sparing with new arg names, but dispersion seems fairly commonly used for this and is somewhat analagous to concentration, which we use widely. @jvdillon thoughts?

[jvdillon]

We've discussed mean parameterization of distributions in the past. Unfortunately there's a few usability questions which need discussion. Can we discuss this as a team and report back?

[gokceneraslan]

Sure. I think dispersion is the easy one, given that one can use total_count directly as dispersion. But mean is tricker.

[gokceneraslan]

Here is Stan's alternative NB parameterization: https://mc-stan.org/docs/2_19/functions-reference/nbalt.html

[jvdillon]

Hi @gokceneraslan.

Our team has discussed the design possibilities and come up with these five options:

# Idea1:
  tfd.Bernoulli(
     **tfd.Bernoulli.mean_parameterization(          # staticmethod
           mean=0.25, validate_args=True, name="bar"),
     validate_args=True,
     name="foo")
   # ==> returns a tfd.Bernoulli

# Idea2:
   tfd.BernoulliFromMean(         # BernoulliFromMean a class (or a function)
       mean=0.25, validate_args=True, name="foo")  
   # ==> returns a tfd.Bernoulli

# Idea3:
   x = tfd.Bernoulli.from_mean(        # from_mean is a static method
       mean=0.25, validate_args=True, name="foo") 
   # ==> returns a tfd.Bernoulli

# Idea4:
   tfd.from_mean.Bernoulli(          # Bernoulli a function (or a class)
       mean=0.25, validate_args=True, name="foo")
   # ==> returns a tfd.Bernoulli

# Idea5:
   tfd.Bernoulli(
       mean=0.25, validate_args=True, name="foo")  # mean is mutex with, eg, probs
   # ==> returns a tfd.Bernoulli

We'd love to hear your thoughts on which you most prefer. (We're all still chewing on which we most prefer.)

[jvdillon]

It sounds like our team is converging on favoring #3. We'd still like to hear your opinion though. :)

[gokceneraslan]

Looks good. A few questions:

• How would that look for negative binomial? What'd be the way to provide mean and dispersion with this new function?
• What about the other parameters like scale in location-scale family distiburions?
• How about tfd.Normal.from_location_scale(location=0.25, scale=1)?

[brianwa84]

For NB I think it would look like:
tfd.NegativeBinomial.from_mean_dispersion(mean=xx, dispersion=yy)

[brianwa84]

Such function would need a nice docstring to explain precisely which dispersion parameter it's talking about. PR welcome.

@classmethod
def from_mean_dispersion(cls, mean, dispersion, **kwargs):
  ...
  if 'logits' in kwargs or 'total_count' in kwargs: raise
  kwargs['total_count']=...
  kwargs['logits']=...
  ...
  return cls(**kwargs)

[jvdillon]

Brian, would you be ok with just "from_mean"? Or perhaps: "mean_parameterization"? I ask because it'd be nice if we could use names which were the same across all distributions (thus making programmatic construction slightly easier). I also dont think there's diminishing value in repeating the kwargs as part of the name. (Saying "mean" is different because it conveys the semantics.)

Additionally, I think its ok that the mean parametrization takes other args. For example, a normal distribution takes a scale but we pin the scale to a constant in generalized linear regression. (Making it mean parameterized.)

Another thing which I think might be great: we have default values for everything except the mean. This would be a step toward unifying the tfp.glm library. A distribution could be passed in and we'd simply call:
Distribution.from_mean(mean=inverse_link_function(w@X))

Thoughts?

[brianwa84]

I'm ok with from_mean as long as it is unambiguously the mean
parameterization.

On Tue, Apr 30, 2019, 4:34 PM Joshua V. Dillon [e-mail]
wrote:

Brian, would you be ok with just "from_mean"? Or perhaps:
"mean_parameterization"? I ask because it'd be nice if we could use names
which were the same across all distributions. I also dont think there's
diminishing value in repeating the kwargs as part of the name. (Saying
"mean" is different because it conveys the semantics.)

Additionally, I think its ok that the mean parametrization takes other
args. For example, a normal distribution takes a scale but we pin the scale
to a constant in generalized linear regression. (Making it mean
parameterized.)

Thoughts?

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/tensorflow/probability/issues/372#issuecomment-488104991,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AFJFSIZEB4UE4I3SZQQKOBTPTCUNLANCNFSM4HHLPWGQ
.

[jarotter]

Did this parametrization ever get to tfp?

[alexonov]

Is there any update ion this?

[brianwa84]

No updates. Still nice to have, PRs would be welcome. Consider using tfp.util.DeferredTensor to defer any transformations to happen inside a GradientTape.
i.e.

nb = NegativeBinomial.from_mean_dispersion(mu, disp)
with tf.GradientTape() as t:
  t.watch((mu, disp))
  lp = nb.log_prob(3)
t.gradient(lp, (mu, disp))  # not None

[jvdillon]

As another example, I think you could more or less copy this:

https://github.com/tensorflow/probability/blob/master/tensorflow_probability/python/glm/family.py#L512