Flux.jl: onecold is very slow

[afternone]

julia> valY |> size
(3755, 128)

julia> @time onecold(valY)
 21.304120 seconds (2.40 M allocations: 95.372 MiB, 0.08% gc time)
128-element Array{Int64,1}:
   1
   2
   3
   ⋮
 126
 127
 128

[colbec]

On my machine with Julia 1.0.3 I get:

julia> using Flux: onecold
julia> valY = rand(3755,128);
julia> valY |> size
(3755, 128)
julia> @time onecold(valY)
  0.838053 seconds (2.47 M allocations: 126.871 MiB, 2.98% gc time)
128-element Array{Int64,1}:
 2885
 2932
  755
...
 3122
 3128
 1780

and with Julia Master I get

  0.001299 seconds (1.06 k allocations: 58.750 KiB)
128-element Array{Int64,1}:
 3659
 1292
...

so maybe there is something else happening on your machine?

[afternone]

What happens when change valY to a CuArray?

[colbec]

No GPU here. Please specify your issue more completely.

[KristofferC]

The problem to do when the input is on the GPU and is a TrackedArray cf:

using Flux: onecold
valY = rand(1000,128);
@time onecold(valY)
#   0.000369 seconds (1.00 k allocations: 36.375 KiB)
@time onecold(gpu(valY))
#   0.025580 seconds (12.65 k allocations: 1.499 MiB)
@time onecold(Flux.param(valY));
#  0.007966 seconds (641.86 k allocations: 22.523 MiB, 49.17% gc time)
@time onecold(Flux.param(gpu(valY)));
#  4.803005 seconds (1.16 M allocations: 42.439 MiB, 0.15% gc time)

[KristofferC]

And running CuArrays.allowscalar(false) we get the error scalar setindex! is disallowed which is why it is slow.

[MikeInnes]

I've fixed the basic issue for vectors, but there's still the problem that we don't support argmax across a dimension in CuArrays. It's probably straightforward to implement that with mapreduce. We'll also need mapslices, I'm not sure if that works already.

[appleparan]

Bump this issue. Now we could rewrite using findmax by CuArray's #500

[h-spiess]

The problem to do when the input is on the GPU and is a TrackedArray cf:

using Flux: onecold
valY = rand(1000,128);
@time onecold(valY)
#   0.000369 seconds (1.00 k allocations: 36.375 KiB)
@time onecold(gpu(valY))
#   0.025580 seconds (12.65 k allocations: 1.499 MiB)
@time onecold(Flux.param(valY));
#  0.007966 seconds (641.86 k allocations: 22.523 MiB, 49.17% gc time)
@time onecold(Flux.param(gpu(valY)));
#  4.803005 seconds (1.16 M allocations: 42.439 MiB, 0.15% gc time)

Hey. I have the same performance problems with the new flux release using zygote. So i assume its not due to being tracked by Tracker.

[piever]

Not sure if this is a good general solution, but the following hack removes the need for onecold when computing the accuracy on a softmax on onehot-encoded data, and is much faster when the model is run on the GPU with CuArrays.

using Flux: OneHotMatrix
compare(y::OneHotMatrix, y′) = maximum(y′, dims = 1) .== maximum(y .* y′, dims = 1)
accuracy(x, y::OneHotMatrix) = mean(compare(y, model(x)))