Julia: make LinearAlgebra.Transpose objects interchangeable with arrays

[HDictus]

It is somwhat irritating that transposing any array requires explicitly converting it to an array type (and hence losing what I presume is the advantage of implemeting a whole new Transpose object to represent the transposed array) before it can be used in methods that require Arrays. (as an example, they can't directly be used for DSP's conv2). I know that in python, numpy has ways of representing 'views' on arrays as indistinquishable to arrays, and it would be most convenient and efficient if objects like LinearAlgebra.Transpose and LinearAlgebra.Adjoint could be used as easily in julia.

I'm afraid I can't volunteer to work on this myself for the time being, as I have some things queued up already, but given a few (6+) months I might get round to it.

[KlausC]

Which type has your matrix and which function do you want to call?
Many methods support AbstractArray, and as Adjoint<:AbstractArray, these can handle all kinds of matrices, using the getindex interface.
If the underlying matrix type is SparseMatrixCSC, that is not effective ("abstract array fallback"). I am working to install something like a "sparse array fallback" to improve this situation.

[HDictus]

The function with which I ran into this issue is conv2 from DSP.jl, which takes a StridedMatrix. I realise that this particular issue could instead be fixed on the DSP end, but it seems an easy enough trap to fall into to exclude transpose by mistake. I think it would be better to eliminate reliance on the function-definer including it - or the issue will keep popping up.

[KlausC]

I don't see your point. As Transpose<: AbstractMatrix and StridedMatrix<:AbstractMatrix it is already possible to use any transposed array as an array.

What you propose is automatic conversion of call arguments of type Transpose to another abstract matrix type, whichever is supported by the method. I think that was intentionally not implemented (for example because it introduces additional ambiguity in the method dispatching process).

If the provider of DSP and conv2 could easily allow different input argument types by just changing the used StridedMatrix by AbstractMatrix without further consequences.
Please file an issue against DSP, if you want this change.

[HDictus]

This isn't so much about my particular case - I don't currently need to convolve the transpose of a matrix - it's just something I noticed, and thought could save people trouble if addressed.
I'm not sure what the purpose is of StridedMatrix - but I assume there's a reason it is being used rather than AbstractMatrix. Relying on library creators to always use AbstractArray in order to work smoothly with transpose doesn't seem ideal.
In principle, it seems that this should work:
f(x::Array{Int64, 2}) = x; x = [1 2 3; 3 4 5]; f(transpose(x))
Since anything that is defined for all 2-dimensional integer arrays, will also be defined for the transpose of any integer array. And I certainly don't think automatic conversion is the solution, as that implies creating a copy, rather than a view, of the original array, defeating the purpose of having a Transpose type in the first place. What _would_ be a valid solution, I'm not sure. I'd look into it more once I work on it myself, provided no one else gets to it first.

[KlausC]

In this case, I have not further ideas. As Transpose provides a view and also provides access to the transposend matrix via the getindex - setindex "AbstractArray" interface, that is already what you want. You have just to change the argument types to AbstractArray and use this interface in the implementation.

[mbauman]

I really appreciate the enthusiasm, but this isn't really a direction we want to go with the language — we're "all in" on many, many, many different subtypes of AbstractArray. Why should Transpose support get folded into Array but not those other array types (which are likely just as important to others as Transpose is to you)? It's perhaps worth noting that Numpy can only represent _some_ views in an ndarray while others get left out in the cold (e.g., advanced indexing).

It's not uncommon for folks to need to "widen" signatures to exploit their full potential, but that's something we strongly recommend from the get-go.

[StefanKarpinski]

Note that the process of generalization of library code usually goes something like this:

1. Package code dispatches on something fairly specific like Array{Float64}.
2. Someone wants to use it on something more general, they file an issue or pull request.
3. The package gets generalized to an appropriate abstraction level, usually just by broadening method signatures to the appropriate level of generality (do you need dense layout? probably not).

That's usually it. After that the code is often sufficiently generic not just for the one different case that was needed in 2 but for many other types as well. Occasionally an additional round of generalization is needed to get the right level of abstraction, but this process converges very quickly. If we made transpose and adjoint type-indistinguishable from arrays they wrap (which is already technically problematic since we can wrap many different kinds of arrays), we'd actually be making step 2 happen less often and be reducing the overall genericness of Julia packages. In other words, by adding a special-case solution to some specific cases of generalization, it would undermine the process by which Julia packages become more broadly generic.

If you want to help, no hard work is required: just file an issue when you encounter an API that is more tightly typed than it should be.

[HDictus]

Alright, thanks, i guess I'll just keep an eye out.