Pymc3: TypeError: Deterministic() got an unexpected keyword argument 'observed'

[schlichtanders]

Why is not possible to add "observed" keyword to pymc3.Deterministic instance?

My data is just a transformation of several standard normal variables, however I seem to be unable to sample the pymc3 model because deterministic transformation cannot except data via "observed".

Is there a reason for this?
Is there another way to achieve this?
thank you very much for your valued time

[fonnesbeck]

A deterministic node is just a transformation of whatever values it is passed as arguments, so "observed" doesn't mean anything. Moreover, if the value does not change there is no point in saving a trace (which is all that Deterministic really adds to simply applying functions to nodes).

[schlichtanders]

you just closed this issue? with the argument that "it is just a transformation, so observed doesn't mean anything"?
I feel like I missed something here. I thought github issues where intended for discussing language features too and that exact error descriptions are a valid name for issues.

so just to repeat myself because I really feel overlooked and I think that was not your purpose:
Is there another way to achieve this?, i.e. for instance to tell the pymc3 model that my data is a deterministic transformation of normal random variables?

[schlichtanders]

please also note that any deterministic transformation of a random variable is itself a random variable and hence I see in fact no reason that "observed" doesn't mean anything for Deterministic. If this would be the case, then "observed" should also mean nothing for Probabilistics if I see this point right.

[fonnesbeck]

I closed this because the issue as stated is not a bug; the software is behaving exactly as we expect it to in this case.

The issue tracker is for tracking bugs in the PyMC code base. The best place to discuss how to solve particular programming difficulties is either on Gitter or for longer discussions, on StackOverflow with the pymc tag. We need to be fairly strict about this, otherwise the issue tracker becomes very hard to manage. In fact, you are welcome to continue the discussion here -- marking the issue as closed does not prevent you from continuing the discussiuon, it just lets the developers know that this is not a bug.

A Deterministic simply transforms its arguments. The "observed" argument is an attribute for stochastics only, which tells us whether to condition on its value or not. A Deterministic is fixed if its arguments (parents) are fixed, and not fixed if its arguments are not fixed.

In fact:

a_b_sum = a + b

is exactly the same as:

a_b_sum = Deterministic('a_b_sum', a + b)

for nodes a and b except that the latter gets tallied to the trace of the model run and the former does not. That's pretty much all that Determinstic is for!

[schlichtanders]

Thanks for coming back to me. I understand that it is currently implemented
this way such that Deterministic does expect the observed argument. It
seems that this decision was made because Deterministic can be used for
transformations of fixed values as well as for stochastics variables. While
for transformation of stochastic variables it makes sense to have an
"observed" keyword (as they are themselves stochastic) it does not make
sense for transformations of fixed values (as they are themselves fixed
again).

However the documentation introduces Deterministc as something you should
use if you want to sample an deterministic transformation of a random
variable (i.e. referring to transformations of stochastic variables).
Furthermore, if I have a transformation of fixed values, the theano
variable itself is enough to be used everywhere in pymc3/theano as the
reference to the fixed transformation. There seems to be no use at all for
wrapping everything in a Deterministic in this case.

Concluding, the use of Deterministic seems to be only needed if you have a
transformation of a random variable, which itself is again random. A
Deterministic is a Stochastic in this case. Hence it makes sense to
support "observed" here.

However I admit, I have the feeling of slightly repeating myself, and hence
I think a third person who could rephrase some ideas might crucially
benefit this little exchange. If you know someone, please feel incourage to
add a ping.

Thank you very much again

On Fri, Mar 31, 2017 at 3:21 PM Chris Fonnesbeck [e-mail]
wrote:

I closed this because the issue as stated is not a bug; the software is
behaving exactly as we expect it to in this case.

The issue tracker is for tracking bugs in the PyMC code base. The best
place to discuss how to solve particular programming difficulties is either
on Gitter https://gitter.im/pymc-devs/pymc or for longer discussions,
on StackOverflow with the pymc tag. We need to be fairly strict about this,
otherwise the issue tracker becomes very hard to manage. In fact, you are
welcome to continue the discussion here -- marking the issue as closed does
not prevent you from continuing the discussiuon, it just lets the
developers know that this is not a bug.

A Deterministic simply transforms its arguments. The "observed" argument
is an attribute for stochastics only, which tells us whether to condition
on its value or not. A Deterministic is fixed if its arguments (parents)
are fixed, and not fixed if its arguments are not fixed.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/pymc-devs/pymc3/issues/1971#issuecomment-290710147,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AEDu-JEf10Z7QIkYRr0PAJLTDXGWsnzQks5rrP3ZgaJpZM4MuHfR
.

[aseyboldt]

I think I have a general idea about what you are trying to do, but I think this is much harder than you seem to think. Let's look at a pymc model as a way to define a function R^n to R (the log pdf). A statement like

a = pm.Normal('a', mu=0, sd=1)

does two things: tell pymc that it needs to increase n by one, and add a term normal_logp(x[n-1] | 0, 1) to the global logp function. A statement like

a = pm.Normal('a', mu=0, sd=1, observed=1)

does not increase n, it just adds a term normal_logp(1 | 0, 1) to the global logp function. A statement like

b = Deterministic('b', whatever)

leaves the function alone altogether, it just tells pymc3 that the value of whatever should appear in the trace under the name b.

Now assume we allowed something like this:

a = pm.Normal('a', mu=0, sd=1)
b = pm.Deterministic('b', 3 * a, observed=1)

I guess what you would want that to do is leave n alone, but increase logp by normal_logp(1/3 | 0, 1).

I can see two big problems with that. First – more of a software engineering problem – we'd somehow need to delay the decision if we should increase n until the model is finished, and we'd have to figure out what to do if there are two deterministics. But perhaps more important: we need to invert the function that is used to define the deterministic. And if this function is not even bijective, things can get pretty much arbitrarily difficult.

[fonnesbeck]

If that's true, and all you want to do is increment the model log-probability by some amount, then a factor potential (Potential in PyMC3) is what you want.

[schlichtanders]

thank you very much for your comments.
I admit the problem of inverting Deterministic components in order to solve the model appeared to me before. However I thought this problem would also appear if we do

a = pm.Normal('a', mu=0, sd=1)
c = pm.Normal('c', 3 * a, observed=1)

@aseyboldt could you explain shortly why this does not need the inverse of 3*a, but the Deterministic version in fact would need it?

---

the explanation with the model being a function from R^n to R really helped! is there some more documentation like this on how pymc3 in fact works? This might clarify many of the problems and questions I have.
When googling Potential class, I only find pymc references like http://pymc-devs.github.io/pymc/modelbuilding.html#the-potential-class however nothing for pymc3. I guess things stand similar with other documentation?

[aseyboldt]

With your model the prob density is a function from R to R (just one parameter, a). We get $log(P(a)) = normal_lpdf(a | 0, 1) + normal_lpdf(1 | 3 * a, 1)$. Not sure where you would expect to have to invert anything.
By the way, I forgot to add the log det of the transformation in the example above. It's just a constant in this case, so it doesn't change anything here, but it might not be in other cases. You might want to have a look at transformations of probability densities (or change of variables) if you want to know more.

[schlichtanders]

thanks for coming back to me
I think I kind of get your formula, I guess your left hand side should in fact mean log(P(a|c=1))
if this is the case, then only the normalization constant is missing, and no log det, but probably you mean proportionality instead

log(P(a|c=1)) ∝ normal(1|3*a, 1) normal(a|0,1) = likelihood * prior

where would you like to add a log det?

---

I am still a bit confused, that we don't need the inverse of the transformation here. Maybe the bayesian inference is taking this over in this setting when a likelihood is available.

For the deterministic transformation, we seem to be in the need of first computing its respective likelihood (which requires the inverse of the transformation)...
However, I just can set the variance of c to very very very low and get a good approximation of the Deterministic b, which is a way to sample from b without knowing the inverse if the Deterministic transformation.
For my personal intuition this highly suggests that we can get rid of the the inverse of b in the first place, without the need of introducing another dummy noise model around it like we did with c.