Julia: var( [num] ) and std( [num] ) should be zero
The variance and the standard deviation of a single value are well defined and both are zero.
Of course a single number has a variance! It's the mean square deviation from its mean, namely zero.
William A Huber
julia
data = [3];
var(data), std(data)
(NaN, NaN)
JeffreySarnoff
All 8 comments
julia> data = [3];
julia> var(data, corrected=false), std(data, corrected=false)
(0.0, 0.0)
martinholters
on 29 Mar 2017
Yeah. I think the current result is correct. Mr. Huber is specifically distinguishing sample and population and our function computes the bias adjusted sample std/var by default.
andreasnoack
on 29 Mar 2017
We should probably have explicit tests for all of these – I think we don't test all the cases.
StefanKarpinski
on 29 Mar 2017
I understand what you are saying. I see the variance and standard deviation of a sample-of-one as intrinsically unbiased; whilst the variance and standard deviation of samples of two or more need bias correction. ( additional note omitted )
JeffreySarnoff
on 29 Mar 2017
I see the variance and standard deviation of a sample-of-one as intrinsically unbiased
A sample size of one doesn't really carry any information about the variance of the population. You can pick an estimator of the population variance that returns zero when n=1 but you could right as well pick an estimator that returns seven when n=1. They'd have the same properties. If the population has variance zero then the first estimator is unbiased but if the population has variance seven then the second version is unbiased.
In most situations, it is not really that meaningful to use statistics for samples of size one. Do you have an application where it would help to change the behavior when n=1?
andreasnoack
on 29 Mar 2017
Yes. I just wrote v0.0.1 of RollingFunctions.jl. Some ways of rolling taper the span at the start or at the end. To roll std, var that way evaluates once over a single value.
JeffreySarnoff
on 29 Mar 2017
These cases are explicitly tested; see https://github.com/JuliaLang/julia/pull/7502. I agree with @andreasnoack's argument. One of my labmates recently had to rerun his data analysis because he realized that, for some datasets, he was taking the variance of a single element vector and MATLAB was returning 0. The problem was precisely that, when he averaged these values of the variance across many datasets, it was biased downward because some of the datasets contained only a single element. If the average were NaN, the problem would have been apparent immediately. Instead, he discovered the problem only after having made the figures and written up the paper.
simonster
on 29 Mar 2017
ok -- that is the stronger use case
JeffreySarnoff
on 29 Mar 2017
Related issues
omus
·
3Comments
dpsanders
·
3Comments
wilburtownsend
·
3Comments
helgee
·
3Comments
StefanKarpinski
·
3Comments
Most helpful comment
These cases are explicitly tested; see https://github.com/JuliaLang/julia/pull/7502. I agree with @andreasnoack's argument. One of my labmates recently had to rerun his data analysis because he realized that, for some datasets, he was taking the variance of a single element vector and MATLAB was returning 0. The problem was precisely that, when he averaged these values of the variance across many datasets, it was biased downward because some of the datasets contained only a single element. If the average were
NaN, the problem would have been apparent immediately. Instead, he discovered the problem only after having made the figures and written up the paper.