Plots.jl: True heat maps

[diegozea]

Hi!
heatmap are actually more like a _2D histogram_ plot than a real heat map.
Would be great to have heatmap to take a single Matrix and plotting its values are colors or to take three vectors: x, y and z (color).
Maybe heatmap with only x and y could be the actual _2D histogram_ or the actual heatmap could be renamed to histogram2D or something similar.
Best,

[tbreloff]

I think I agree with you... But I'd like to review the terminology that other graphics packages use to look for precedence of notation. If packages are split on the usage then I lean towards backwards compatibility. In that case maybe there's another way to distinguish the uses... Checking for matrix input maybe? Could utilize the z argument?

On Mar 1, 2016, at 12:46 PM, Diego Javier Zea [email protected] wrote:

Hi!
heatmap are actually more like a 2D histogram plot than a real heat map.
Would be great to have heatmap to take a single Matrix and plotting its values are colors or to take three vectors: x, y and z (color).
Maybe heatmap with only x and y could be the actual 2D histogram or the actual heatmap could be renamed to histogram2D or something similar.
Best,

—
Reply to this email directly or view it on GitHub.

[diegozea]

Maybe the default value of z, if z is missing, should be counts/frequencies. With a Matrix as input z is defined, because x are the row numbers, y the column numbers and z the matrix values. That shoulds maintain backwards compatibility, I guess.

[tbreloff]

Looking at plotly examples, I think I agree with the proposed change:

• [x] hist2d: new name for current heatmap
• [ ] hist3d: http://matplotlib.org/examples/mplot3d/hist3d_demo.html
• [x] heatmap: https://plot.ly/javascript/heatmaps/ (similar/same as imshow?)

Going along with all this, I want to overhaul the api for 3D data... namely I want to better use context of what is being plotted to determine what a vector/matrix/etc refers to:

z = rand(10,10)

# this should be 10 lines:
plot(z)

# this should be a "surface"
plot(z, linetype = :contour)

[diegozea]

Sounds good :+1:

[diegozea]

hist2d is already exported from Base, maybe will be better histogram2d to avoid the name collision...

[tbreloff]

Yeah... I had the same issue with hist, so I'll probably follow the same convention.

[tbreloff]

Kicking this off with plotly:

[diegozea]

It's working awesome with the matrix!!! :smiley:

But I found this problems using x, y and z vectors:

[tbreloff]

Yes passing in vectors which apply to surfaces is a big (not so easy) item on my todo list.

• Do you fit to a grid?
• Do you zero out positions not given? Interpolate?

I think these questions are tricky in the general case, so there probably needs to be additional interface that the user can specify, but maybe Plots can try to guess the right answer.

If you have any ideas on proper behavior, please post.

[tbreloff]

I suspect you really want to do this in this case:

but this is not always what you'd want to do. How do you generalize?

[diegozea]

I was expecting something like this:

Where the missing areas/points haven't colors.
The most similar plot I get with Plots is:

I believe that, by default, missing point shouldn't be plotted like in the two examples above. If that isn't possible, plot missing points with the background color (and maybe excluding the background color from the color range).

Having a default argument, like in the Base.get function, to indicate the value to be used to replace NaNs, DataFrames' NAs, empty/null Nullables, etc. would be a good option.

My favorite heatmap function from R is heatmap.2. It has the following arguments:

• na.rm : logical indicating whether NA's should be removed.
• na.color : Color to use for missing value (NA). Defaults to the plot background color.

[tbreloff]

I think this is what you're going for?

using Plots; plotly()
x = [1,1,1,2,2,3]
y = [2,3,4,3,4,4]
z = 1:6

# compress to unique values and remap to new indices
function remap(vals)
    compressed = sort(unique(vals))
    remapped = Array(eltype(x), length(vals))
    for (i,x) in enumerate(compressed)
        remapped[find(v -> v == x, vals)] = i
    end
    compressed, remapped
end

# create a matrix from the sparse representation
xs, I = remap(x)
ys, J = remap(y)
mat = full(sparse(I, J, z))

# create a gradient where (normalized) zeros are invisible
grad = ColorGradient(vcat(RGBA(0,0,0,0), Plots._gradients[:bluesreds]), [0,1e-6,0.5,1])

# plot a heatmap
heatmap(xs, ys, mat, c = grad)

I think this belongs (if anything) as a recipe... not as standard behavior (but I could be convinced)

[tbreloff]

If you can think of a good function name, and other optional settings that you'd like, I can turn this into a proper Plots recipe...

[diegozea]

I must admit than the heatmap(mat) is more easy to use than the Vega heatmap and Gadfly rectbin interfaces with x, y and color. However, sometimes I have matrix with a lot of NaN values where x, y and z are a better representation.

If heatmap(mat) is the only option, a good way to deal with NaNs, NAs and Nullables will be useful.

I'm having this problems with the last master:

P.S.: I'm willing to replace the actual protovis function of PairwiseListMatrices by a Plots' recipe ;)

[tbreloff]

Your output looks really wierd. I just started up a notebook and ran:

using PairwiseListMatrices, Plots; plotly()
mat = PairwiseListMatrix([1,2,-1,-3,4,0])
heatmap(mat, yflip=true)

and this is what I got:

Try a Pkg.checkout("Plots","dev")?

Also this is what I see when I add a NaN:

mat[1,2] = NaN
heatmap(mat, yflip=true)

[tbreloff]

And after checking out your current viz... that's pretty straightforward to reproduce:

plotlyjs()
x = 1:4
n = length(x)
scatter(x, zeros(n), m=(10,:orange), leg=false)

zmin,zmax = extrema(mat)
grad = ColorGradient(:bluesreds)
curvecolor(v) = getColorZ(grad, (v-zmin)/(zmax-zmin))

for i=1:n, j=1:i-1
    curve = BezierCurve(P2[(x[i],0), ((x[i]+x[j])/2, i-j), (x[j], 0)])
    plot!(curve_points(curve), line = (curvecolor(mat[i,j]), 0.5, 2))
end

[diegozea]

Thanks! It works fine in the dev branch :)

[diegozea]

Could the arc diagram be a function of Plots?

[tbreloff]

Sure... it would belong in recipes.jl. Any interest in adding it? It seems like you're into graph algos. You can use the example above as a baseline.

[diegozea]

I could try it :) Is there a way to plot real semicircles instead of Bézier curve?

[tbreloff]

Sure... there's always a way.

Function signature is function partialcircle(start_θ, end_θ, n = 20, r=1)

The coordinates might be easier to work with if you unzip:

using Plots; plotly(size=(400,300),leg=false)
circ = Plots.partialcircle(0.25π, π, 50, 1)
x, y = Plots.unzip(circ)
plot(2x+3, 4y-2)

[diegozea]

Two more question:

1. Now, If the mouse is over the line, plotly shows the coordinates. Is there a way to customize that behavior? Would be great to show the edge value and/or the nodes linked by that edge instead of the line coordinates.
2. Is it possible to force the axis to have the same ratio? I want to avoid this:
   

[tbreloff]

Yes it's possible: https://plot.ly/javascript/hover-events/

I expect PlotlyJS to support this eventually. (cc: @spencerlyon2)

[sglyon]

Definitely, julia callbacks to javascript events is one of my next goals.

However, this is possible without needing to use events. See the text and hoverinfo trace attributes

[diegozea]

@tbreloff Is It possible at this moment to access text and hoverinfo plotly attributes from Plots? How can I do that?

[tbreloff]

Well if you do:

using Plots; plotlyjs()
p = plot()

then p.o is a PlotlyJS.SyncPlot. I don't know PlotlyJS well enough to say exactly what the commands would be, but I suspect you can just make a call on that object.

[sglyon]

Hey @diegozea as tbreloff says p.o will be a PlotlyJS.SyncPlot so you can do anything PlotlyJS allows using that object.

In this example I believe you want to alter the text/hoverinfo attribute on a particular trace. To do that you will want a call similar to:

PlotlyJS.restyle!(p.o, trace_number; hoverinfo=my_hover_info, text=my_text)

For more info on what restyle! and it's friends can do, check out this section of the PlotlyJS.jl docs

[diegozea]

p.o works with plotlyjs() but with plotly() p.o is nothing.

[tbreloff]

What were you hoping it would be? plotly() is a much different beast... it's bare-bones access to plotly.js. While both backends share much of the "Plots-to-Plotly" translation code, plotlyjs() has much more in the way of features/interaction.

[diegozea]

Ok... Is it possible to use text and hoverinfo with plotly() ?

[tbreloff]

Not without some hacking. I recommend using PlotlyJS if you can.

[diegozea]

@tbreloff Is there a way to plot a heatmap with categorical axis labels?

[tbreloff]

This works out of the box:

julia> using Plots; plotly()
Plots.PlotlyBackend()

julia> x = ["Mon","Tues","Wed","Thur","Fri"];

julia> y = ["Morning","Afternoon","Night"];

julia> heatmap(x, y, rand(length(x), length(y)))
[Plots.jl] Initializing backend: plotly

[diegozea]

Great, It works!

How can I modify the margins?

[tbreloff]

You can't do it directly through Plots right now (it's hardcoded). You should approach it similarly to how you did text/hover.

I should add back an argument to allow for arbitrary overrides, which get merged into the Plotly dictionary before converting to JSON. Then you could add something like: override = KW(:margin => KW(:l=>35, :b=>30, :r=>8, :t=>20))

[diegozea]

I like override, having a way to pass data to plotly/plotlyjs sounds really useful!