Plots.jl: xlim and ylim defaults are incorrect on 0.6

[dpsanders]

On 0.5, the defaults for xlim and ylim take into account all elements of the plot and try to accommodate them all; on 0.6, they seem to default to (0, 1).

[mkborregaard]

I can't replicate this on 0.6, could you post an MWE? Are you on Plots master?

[mkborregaard]

@dpsanders Do you still have this issue or can it be closed?

[daschw]

I could replicate this with histogram(randn(1000))on 0.6 yesterday

[daschw]

(currently not at my workstation... thus 'yesterday' )

[mkborregaard]

Yup, me too. Ugh.

[mkborregaard]

This only affects the :bar seriestype, and thus, by extension, histogram.

[dpsanders]

Also the :shape seriestype (which I guess is used for :bar).

[mkborregaard]

It appears to be the extrema function pulling tricks

[mkborregaard]

In 0.5:

julia> Base.extrema([1,2,3,NaN])
(1.0,3.0)

In 0.6:

julia> Base.extrema([2,3,4, NaN])
(NaN, NaN)

[dpsanders]

That in turn is due to

julia> max(3, NaN)
NaN

which, as far as I remember, is a documented change in behavior in 0.6.

Probably best to just write Plots' own extrema function:

julia> my_extrema(x) = extrema(filter(!isnan, x))
my_extrema (generic function with 1 method)

julia> my_extrema([1,2,3,NaN])
(1.0, 3.0)

[dpsanders]

https://github.com/JuliaLang/julia/pull/12563

[mkborregaard]

There's apparently a super-lightweight package to deal with such things. I've put in a PR with extrema https://github.com/mlubin/NaNMath.jl/pull/16 - I am guessing we'll need to depend on that package anyway.

[mkborregaard]

This turns out to be a longer ordeal - there is https://github.com/JuliaPlots/Plots.jl/pull/856 but there are so many instances of maximum and minimum in the code, not all compatible with the NaNMath versions.

[mkborregaard]

It's also zlim and clim, meaning that a lot of functionality is currently lost:

z = randn(10,10)
z[2,2] = NaN
heatmap(z)

This is a major issue, as NaNMath has no methods for arrays of higher dimension (e.g. matrices). I don't know if they plan to implement them.
This is very likely the biggest hurdle towards becoming 0.6 compliant on time.

[mkborregaard]

It affects at least minimum, maximum, extrema and mean, perhaps also sum.

[dpsanders]

Maybe there could be an ignore_nan keyword argument in all of those functions? Or just define Plots.jl's own, non-exported versions, and write Plots.extrema etc everywhere?

[dpsanders]

If you just change the type annotation for the argument of NaNmath.extrema to AbstractArray, it will work for any dimension.

[mkborregaard]

Yeah I don't think the want an ignore_nan keyword (I think it would lead to dynamic dispatch of functionality?), but instead e.g. NaNMath.mean – the issue is that these functions are only defined for a narrow subset of types.

If you just change the type annotation for the argument of NaNmath.extrema to AbstractArray, it will work for any dimension.

It's that simple, huh? I thought they'd need generated functions and everything. I'll suggest that to the NaNMath maintainers.

So far I am experimenting with your suggestion above (defining _extrema etc everywhere and filtering with isnan)

[dpsanders]

As far as I can see, it should be that simple, at least for anything that has indexing for which "for i in x" works.

[mkborregaard]

There is a new version of the PR here https://github.com/JuliaPlots/Plots.jl/pull/866 that uses your suggestion above of just defining _ versions locally and using filter(!isnan(.

for fun in (:extrema, :minimum, :maximum, :mean)
       @eval $(Symbol(string("_",fun))){F<:AbstractFloat, N<:Integer}(x::AbstractArray{F, N}) = Base.$(fun)(filter(!isnan(x)))
       @eval $(Symbol(string("_",fun)))(x) = Base.$(fun)(x)
end

Here's the bad news - it hasn't solved the (0,1) limits issue. I will revisit this later.

[mkborregaard]

After working an inordinate number of hours(!) on this, including 3 PRs on the NaNMath package, I know have a solution to this that works: #876 . I am not a big fan of JuliaLang/julia#12563 .
Comments are requested on the PR.

[dpsanders]

I can confirm that the original bug is now fixed on Plots.jl master, many thanks!