Julia: Wishlist: API and documentation for extending broadcast

[timholy]

We have extensive documentation on the array interface, specifically how to extend getindex and setindex! for arbitrary array types. In 0.6, julia's broadcasting is more awesome and central than ever; it would seem appropriate to have some discussion about how one should go about specializing broadcasting behavior. An example of a package that's about to take a do-it-yourself plunge is https://github.com/JuliaImages/ImageMetadata.jl/pull/14; if you follow that link, you'll see that there was at least one very cool surprise lurking for the unwary.

[andreasnoack]

I'm also wondering if we could restructure the broadcast code to detect the right return type for non-base arrays such that custom <:AbstractArray, the developer should mainly implement _broadcast! and some promote_containertype methods. Right now it seems as if you'll have to implemental a broadcast_c method which is very similar to the base version, e.g. https://github.com/JuliaParallel/DistributedArrays.jl/blob/e5b4612b4f0e95ad28359958a34aa1b94f4193ac/src/mapreduce.jl#L26-L36 and https://github.com/JuliaStats/DataArrays.jl/blob/2a83487bb7cbf570bb6679eb5995ce3545c2869c/src/broadcast.jl

[Sacha0]

Agreed, chances are we should restructure broadcast's container type promotion mechanism altogether with various types of extension in mind. Ref. recent discussion in #20007, #20009, #20102, towards the end of #16961 (particularly https://github.com/JuliaLang/julia/pull/16961#issuecomment-269882295, which contains a few other refs), and #19787. Best!

[andreasnoack]

@Sacha0 Thanks for keeping track of the previous discussions. It is really useful to have the references.

[Sacha0]

My pleasure. I hope to write up some thoughts whenever the time is right. Best!

[StefanKarpinski]

I suspect the time is now since we're entering 0.6 feature freeze :)

[andyferris]

Yes, I was hoping to understand this better so I can make the StaticArrays implementation more compatible.

[KristofferC]

As someone who is also a fan of the AbstractArray interface, a guide on what to do to get broadcasting work smoothly with custom types would be very appreciated.

[Sacha0]

Ref. to related discussion https://github.com/JuliaLang/julia/commit/746dbb0c83a6f9a43a19a0328dba3bf150ae0e56#commitcomment-22403175

[Sacha0]

Perhaps should receive a 1.0 tag?

[Sacha0]

(Crossref #22060.)

[tkelman]

the issue of container type promotion seems a bit separate from the fusion issue? may touch similar parts of the base code though

[JeffBezanson]

Similar to #22060, can evolve over 1.x.