Julia: unnecessary allocations in `broadcast!`?

[stevengj]

The broadcast! function seems like it allocates memory, e.g. in the following example it apparently allocates about 16 bytes per loop iteration.

julia> function foo(x, n)
           for i = 1:n
               broadcast!(x -> 2x+1, x, x)
           end
           return x
       end
foo (generic function with 1 method)

julia> @time foo([0,0,0], 10^4); # warmup
  0.031009 seconds (26.09 k allocations: 904.224 KB)

julia> @time foo([0,0,0], 10^4);
  0.000213 seconds (10.01 k allocations: 156.531 KB)

It seems like this should be something that can be eliminated.

cc @pabloferz

[Sacha0]

Strangely

julia> VERSION
v"0.6.0-dev.1557"

julia> using BenchmarkTools

julia> q = [0, 0, 0];

julia> @benchmark broadcast!(z -> 2z+1, $q, $q)
BenchmarkTools.Trial:
  samples:          10000
  evals/sample:     997
  time tolerance:   5.00%
  memory tolerance: 1.00%
  memory estimate:  0.00 bytes
  allocs estimate:  0
  minimum time:     21.00 ns (0.00% GC)
  median time:      21.00 ns (0.00% GC)
  mean time:        22.61 ns (0.00% GC)
  maximum time:     193.00 ns (0.00% GC)

Perhaps the allocation in the example above is elsewhere? Best!

[yuyichao]

This is caused by the allocation of (x,) which will likely be a compile time constant if x is a constant.

[stevengj]

So, basically we need some improved loop hoisting for the compiler to recognize that (x,) can be allocated outside the loop?

[stevengj]

(I thought Julia didn't need to do a heap allocation for local tuples?)

[yuyichao]

I doubt hoisting is the most useful optimization in this case (it certainly is useful) since it will breakdown if x is assigned to. The tuple isn't tuple since it is somehow passed to a _broadcast..... function that's not inlined.

[stevengj]

So maybe we just need to add @inline somewhere?

[pabloferz]

So maybe we just need to add @inline somewhere?

That's most likely

[pabloferz]

This might be the cause https://github.com/JuliaLang/julia/blob/d76388f3bf4818fdfeeb72a916c0b6026de99477/base/broadcast.jl#L132.

[stevengj]

Why do we have the noinline there?

[tkelman]

(useful tip - hit y when viewing a file and github will add the sha to the url, so the line of code you point to stays correct even if the file changes later: https://github.com/JuliaLang/julia/blob/d76388f3bf4818fdfeeb72a916c0b6026de99477/base/broadcast.jl#L132)

[pabloferz]

I don't know, but we could ask @timholy (https://github.com/JuliaLang/julia/commit/522547ba3806bd2f0aae24f850f96607c7479000)

@tkelman Thanks for the tip!

[Sacha0]

This comment might be the answer? Best!

[stevengj]

Since that comment is for code that seems to have been deleted, maybe it is no longer relevant? Regardless, I just tried removing the noinline and adding @inline to all the _broadcast! methods, and it doesn't seem to remove the allocation, so this is not sufficient.

[pabloferz]

I think I have a fix. Will submit a PR (#19639)

[stevengj]

Closed by #19639.