Julia: qr(A; full=false) no longer available?

[PetrKryslUCSD]

How is the economy factorization supported in 0.7-alpha? There doesn't seem to be a good retrieval method for getting just the economy Q (or have I missed it?).

[Sacha0]

In 0.7, qr returns a decomposition object with an efficient internal representation

julia> using LinearAlgebra

julia> foo = rand(4, 4)
4×4 Array{Float64,2}:
 0.126194  0.945395  0.424594  0.680391
 0.429456  0.398015  0.673802  0.923965
 0.489852  0.590296  0.290908  0.695528
 0.502459  0.517047  0.652069  0.727883

julia> qr(foo)
LinearAlgebra.QRCompactWY{Float64,Array{Float64,2}}
Q factor:
4×4 LinearAlgebra.QRCompactWYQ{Float64,Array{Float64,2}}:
 -0.151615   0.981928     0.112784  -0.0104818
 -0.515967  -0.151403     0.566512  -0.624436
 -0.588529  -0.00381495  -0.778279  -0.218861
 -0.603676  -0.113489     0.246224   0.749713
R factor:
4×4 Array{Float64,2}:
 -0.832332  -1.00823   -0.97688   -1.42864
  0.0        0.807119   0.239793   0.442943
  0.0        0.0        0.363752   0.238082
  0.0        0.0        0.0       -0.190609

from which you can extract the orthogonal/unitary factors with the same efficient internal representation

julia> qr(foo).Q
4×4 LinearAlgebra.QRCompactWYQ{Float64,Array{Float64,2}}:
 -0.151615   0.981928     0.112784  -0.0104818
 -0.515967  -0.151403     0.566512  -0.624436
 -0.588529  -0.00381495  -0.778279  -0.218861
 -0.603676  -0.113489     0.246224   0.749713

Close if this answer satisfies you? Best! :)

[PetrKryslUCSD]

But what if A is rectangular and I want Q to be the same shape? Right now I am getting as Q a square matrix...

[andreasnoack]

Right now I am getting as Q a square matrix...

You can get that with

julia> A = randn(4,2);

julia> F = qr(A);

julia> Array(F.Q)
4×2 Array{Float64,2}:
 -0.70317    0.295658
 -0.45671   -0.879036
 -0.445754   0.357912
 -0.313483   0.108542

but I think I'd actaully like that method to return the square version eventually. My preferred version to get the rectangular version would be

julia> F.Q*Matrix(I, size(A)...)
4×2 Array{Float64,2}:
 -0.70317    0.295658
 -0.45671   -0.879036
 -0.445754   0.357912
 -0.313483   0.108542

since it is how it is constructed anyway from the elementary reflectors. Do you really need the densely stored version though? It might be more efficient to apply the compact form first and then only afterward extract the part you interested in, e.g. least squares

julia> b = randn(4);

julia> UpperTriangular(F.R)\(F.Q'*b)[1:size(F, 2)]
2-element Array{Float64,1}:
 0.39458420793912025
 0.8375702802780137

where extracting the rectangular first would effectively be

julia> UpperTriangular(F.R)\((F.Q*Matrix(I, size(F)...))'*b)
2-element Array{Float64,1}:
 0.39458420793912025
 0.8375702802780137

which is much less efficient.

[PetrKryslUCSD]

I use the qr to orthogonalize a set of vectors (columns of the input matrix). Would you expect the procedure you described to be efficient for this purpose?

[andreasnoack]

I use the qr to orthogonalize a set of vectors (columns of the input matrix). Would you expect the procedure you described to be efficient for this purpose?

The compact Q type we use is an efficient representation of such an orthogonal set of vectors so it might but in the end, it depends on the application of the orthogonal vectors. I suspect that the orthogonal representation would be preferable in many cases, but of course, there are cases where is better, easier, or only feasible to work with the explicitly stored version of Q.

[PetrKryslUCSD]

I see: the step Array(F.Q) is crucial in converting the factorization into an actual rectangular matrix!

I think the documentation mentions this now that I look at it again (... A Q matrix can be converted into a regular matrix with Matrix. ...), but it might be worthwhile to explicitly spell out that the resulting matrix might be an economy matrix.

[garrison]

qr() returned a thin decomposition by default on julia 0.6. Now it no longer does, making it a breaking change without deprecation. I believe this should be mentioned in NEWS.

[andreasnoack]

It is mention. See https://github.com/JuliaLang/julia/blob/master/NEWS.md#breaking-changes and search for qr. If people have specific suggestions that would improve the documentation or the NEWS entry then please open a PR.

[garrison]

It is mention. See https://github.com/JuliaLang/julia/blob/master/NEWS.md#breaking-changes and search for qr.

No, it is not mentioned. The only mention of qr in that section is

• qr methods now return decomposition objects such as QR, QRPivoted,
  and QRCompactWY rather than tuples of arrays ([#26997], [#27159], [#27212]).

After reading this, one would reasonably expect code that worked on julia 0.6 to either continue working or actually break when used as a tuple of arrays. Instead, there is no error but just different behavior, without any mention of the thin default changing. (thin is mentioned in a different section, which says full replaces it, but full does not work either with qr any longer.)

[andreasnoack]

The point is that it now returns a factorization object.

without any mention of the thin default changing

That is not really the case. The Q in the factorization object both thin and thick depending on the context

julia> using LinearAlgebra

julia> Q, R = qr(randn(5,3));

julia> Q*Matrix(I, 5, 3)
5×3 Array{Float64,2}:
 -0.126594   -0.162614    0.836955
  0.0265404  -0.400726    0.128851
 -0.640374    0.0083259  -0.431216
  0.447749   -0.730629   -0.30548
  0.610501    0.528287   -0.0603235

julia> Q*Matrix(I, 5, 5)
5×5 Array{Float64,2}:
 -0.126594   -0.162614    0.836955    0.38462    0.330311
  0.0265404  -0.400726    0.128851   -0.802883   0.421297
 -0.640374    0.0083259  -0.431216    0.218607   0.596754
  0.447749   -0.730629   -0.30548     0.395966   0.124877
  0.610501    0.528287   -0.0603235   0.0535566  0.584546

and Array(F.Q) is still thin.

[garrison]

Perhaps I should have been more clear from the beginning. It has been a longstanding goal in julia development, for as long as I remember, that code written for julia 0.x should behave identically on julia 0.(x+1). Yes, one should expect that doing so will result in a ton of deprecation warnings, and it may even run slower until these warnings are fixed. And sure, there have been a few occasions where this principle has been violated (though typically with good reason). But overall, the goal has been for the upgrade path to be smooth.

This change has broken with this tradition. size(qr(rand(4,2))[1]) results in (4, 2) on julia 0.6 and (4, 4) on julia 0.7.

The Q in the factorization object both thin and thick depending on the context

As mentioned above, in an identical context (i.e. when treating it as a tuple), it results in a thin decomposition under julia 0.6 and a thick decomposition under julia 0.7.

I would consider this a bug in the deprecations, but it may be a bit late to "fix" this, so I think it would be reasonable to mention this explicitly in NEWS. I will work on phrasing for a pull request.

[StephenVavasis]

I have created a pull-request to improve the documentation of this issue: https://github.com/JuliaLang/julia/pull/28446