Flux.jl: MethodError with Complex inputs

[ssfrr]

I'm not actually sure whether this is an issue in ForwardDiff or Flux, so LMK if I should re-file over there instead. I'm trying to adapt the linear regression example to use complex numbers and running into some issues:

using Flux

# pick parameters to solve for
W = 1 + (randn(2, 5) + randn(2, 5) * im) * 0.5
b = (randn(2) + randn(2) * im) * 3

N = 1000
# generate input and output data
X = randn(5) .+ 0.2*randn(5) .* im
Y = W * X .+ b

# initialize solver parameters
W2 = param(randn(2, 5) + randn(2, 5) * im)
b2 = param(randn(2) + randn(2) * im)

predict(x) = W2*x .+ b2

# just try running `predict` on the data:
predict(X)

This throws the error:

MethodError: no method matching ForwardDiff.Dual{Void,V,N} where N where V<:Real(::Complex{Float64}, ::Tuple{Bool,Bool})
_collect(::Array{Complex{Float64},1}, ::Base.Generator{Array{Complex{Float64},1},Flux.Tracker.##35#36{Tuple{Bool,Bool}}}, ::Base.EltypeUnknown, ::Base.HasShape) at array.jl:488
map(::Flux.Tracker.##37#39{2}, ::Tuple{TrackedArray{…,Array{Complex{Float64},1}},TrackedArray{…,Array{Complex{Float64},1}}}, ::Tuple{Int64,Int64}) at tuple.jl:178
tracked_broadcast(::Function, ::TrackedArray{…,Array{Complex{Float64},1}}, ::TrackedArray{…,Array{Complex{Float64},1}}) at array.jl:277
broadcast(::Function, ::TrackedArray{…,Array{Complex{Float64},1}}, ::TrackedArray{…,Array{Complex{Float64},1}}) at broadcast.jl:455
predict(::Array{Complex{Float64},1}) at untitled-64aba8eb4b9dba636ecade62690a595b:68
include_string(::String, ::String) at loading.jl:522
...

[MikeInnes]

I think this is quite simple to fix, but we just need to handle complex numbers specially in broadcast; basically we need to create Complex{Dual}s here.

[dsweber2]

It seems that this is working (on 0.7 at least; it's broken on 0.6.4); however, if we go the next step and try to take gradients, Flux seems to think the gradient should never be non-zero. Specifically, adding the following code to your example:

function loss(x, y)
  ŷ = predict(x)
  sum(abs.(y .- ŷ).^2)
end
gs = Tracker.gradient(()-> loss(x,y), Params([W2, b2]))
Δ = gs[W2]

results in Δ being a zero matrix. I've also tried this with norm instead of sum(abs.(... to much the same effect

[lamhoson]

Is Flux able to handle Deep Learning Network with complex number as inputs and weightings ?

[pelagisk]

Hi, is there any progress in getting Flux to play nicely with complex numbers as weightings? It would be a great advantage in various physics applications!

[MikeInnes]

This is basically annoying to do with our current implementation of broadcast. But I'm planning to throw that out in favour of a reverse mode version, so once that's done this should more-or-less just work.