Homebrew-cask: URL audit issues

[commitay]

https://github.com/caskroom/homebrew-cask/pull/41733

Downgrades to HTTP mirrors.

audit for qt-creator: failed
 - The URL http://download.qt.io/official_releases/qtcreator/4.5/4.5.0/qt-creator-opensource-mac-x86_64-4.5.0.dmg should use HTTPS rather than HTTP
Error: audit failed for 1 cask: qt-creator

Link: <http://download.qt.io/official_releases/qtcreator/4.5/4.5.0/qt-creator-opensource-mac-x86_64-4.5.0.dmg.meta4>; rel=describedby; type="application/metalink4+xml"
Link: <http://download.qt.io/official_releases/qtcreator/4.5/4.5.0/qt-creator-opensource-mac-x86_64-4.5.0.dmg.torrent>; rel=describedby; type="application/x-bittorrent"
Link: <http://master.qt.io/official_releases/qtcreator/4.5/4.5.0/qt-creator-opensource-mac-x86_64-4.5.0.dmg>; rel=duplicate; pri=1; geo=fi
Link: <http://ftp.fau.de/qtproject/official_releases/qtcreator/4.5/4.5.0/qt-creator-opensource-mac-x86_64-4.5.0.dmg>; rel=duplicate; pri=2; geo=de
Link: <http://qt-mirror.dannhauer.de/official_releases/qtcreator/4.5/4.5.0/qt-creator-opensource-mac-x86_64-4.5.0.dmg>; rel=duplicate; pri=3; geo=de
Link: <http://mirrors.dotsrc.org/qtproject/official_releases/qtcreator/4.5/4.5.0/qt-creator-opensource-mac-x86_64-4.5.0.dmg>; rel=duplicate; pri=4; geo=dk
Link: <http://mirror.netcologne.de/qtproject/official_releases/qtcreator/4.5/4.5.0/qt-creator-opensource-mac-x86_64-4.5.0.dmg>; rel=duplicate; pri=5; geo=de
Location: http://master.qt.io/official_releases/qtcreator/4.5/4.5.0/qt-creator-opensource-mac-x86_64-4.5.0.dmg



md5-ca59772670873c95e783f5bef23775d9



URL: https://download.qt.io/official_releases/qtcreator/4.5/4.5.0/qt-creator-opensource-mac-x86_64-4.5.0.dmg
Redirected: http://master.qt.io/official_releases/qtcreator/4.5/4.5.0/qt-creator-opensource-mac-x86_64-4.5.0.dmg



md5-ad4dd91d7ca6852b68fbd16fe7fec410



==> 1 modified cask: anka-flow
==> Downloading https://d1efqjhnhbvc57.cloudfront.net/AnkaFlow-1.1.1.79.pkg
==> Verifying checksum for Cask anka-flow
audit for anka-flow: failed
 - The URL https://d1efqjhnhbvc57.cloudfront.net/AnkaFlow-1.1.1.79.pkg is not reachable (HTTP status code 403)
Error: audit failed for 1 cask: anka-flow



md5-18dd17c58524915a218e237296ecb21c



==> Verifying checksum for Cask opera-developer
audit for opera-developer: failed
 - The URL https://www.opera.com/computer/beta is not reachable
Error: audit failed for 1 cask: opera-developer



md5-e6dd1d016a112819e4eac345e3af150c



audit for font-abeezee: failed
 - The URL https://github.com/google/fonts/trunk/ofl/abeezee is not reachable (HTTP status code 404)
audit for font-advent-pro: failed
 - The URL https://github.com/google/fonts/trunk/ofl/adventpro is not reachable (HTTP status code 404)
audit for font-allan: failed
 - The URL https://github.com/google/fonts/trunk/ofl/allan is not reachable (HTTP status code 404)

---

brew cask audit --debug 0ad and adobe-photoshop441 both hang at ==> Auditing URL format with no further output. Both return HTTP 200.

[claui]

@commitay In the part where it _downloads then fails_ (#41724), the server returns a 403 because the Referer header is missing from the HTTP request.

Digging a bit deeper reveals a possible issue in this line, which does consider the user_agent field from the cask but not the referer field.

[vitorgalvao]

Pinging @reitermarkus @Git-Jiro.

[commitay]

audit only seems to check for a HTTPS upgrade on the Cask url itself.

Sharing the URL audit code with brew may not be suitable for us unless it is changed to follow the proposal in https://github.com/caskroom/homebrew-cask/issues/25403#issue-182329709.

[Git-Jiro]

I see that my latest addition is causing some trouble.
I will help with coding a solution once it is found.
I am not sure if I can give any valuable insight which helps finding a solution ...

[commitay]

I'm leaning toward reverting the URL audit.

While the other issues should be fixable without to much trouble (referer, etc), we really need the full implementation of https://github.com/caskroom/homebrew-cask/issues/25403#issue-182329709 for the HTTP/HTTPS upgrade/downgrade to be reliable and useful.

Keeping the audit means that we would have to deal with:

• Merging Casks with audit warnings.
  
  
  
  • Ignoring the audit means we lose the rest of the CI checks (style and install)
  
  
• Manually checking HTTP→HTTPS upgrades for possible downgrades after the first hop.
• Reverting changes in contributor PRs that have been made because of audit recommendation.
  
  
  
  • https://github.com/caskroom/homebrew-cask/pull/41733#issuecomment-350230355
  
  

---

Some figures on HTTP:

• 1354 Casks have a HTTP url

• 1326 Casks have a HTTP homepage

• 405 Casks have a HTTP appcast

[vitorgalvao]

If this is causing more problems than it solves and is not an easy fix, reverting might be best.

[commitay]

It makes it difficult to run audit locally, some Casks seem to hang indefinitely (hours), others take 20 minutes to pass.

If this is causing more problems than it solves and is not an easy fix, reverting might be best.

If it was only a couple of minor issues I'd be in favour of trying to fix it but all of the issues encountered so far makes audit unusable for some Casks.

PR to revert the Cask changes only: https://github.com/Homebrew/brew/pull/3564

[claui]

I feel it’s worth mentioning that notwithstanding any unintended side effects, @Git-Jiro did an amazing job with the additions, and it would have been a useful feature. Thanks @Git-Jiro for that!

Even though the partial revert seems to be the best thing to do right now, I hope that one day, someone with lots of free time is going to revisit the whole thing and re-add it.

[vitorgalvao]

I feel it’s worth mentioning

It is! Thank you for bringing that up, @claui, I agree.

Thank you, @Git-Jiro.

[commitay]

it would have been a useful feature

We will still benefit from having had this available to us, if only briefly. I'm running audit on all repos and I'll start doing PRs for HTTPS upgrades after the sorting though false positives.

Thank you, @Git-Jiro!

[commitay]

An interesting problem is that some Casks (e.g. all libreoffice variants) have a mix of HTTP and HTTPS mirrors that they redirect to.

To account for the possibility that a user can downgraded to HTTP the Cask should be HTTP, but it's possible (maybe even likely) that the majority of users would be redirected to a HTTPS link.

This is kind of a non-issue for checksummed Casks* but anything with :no_check might need to have _all_ possible redirects checked.

*If the checksum is obtained from a HTTP link it is an issue.

[commitay]

@vitorgalvao @reitermarkus

It's only occurred a few times so far but checking HTTPS upgrades and downgrading them if required is somewhat inconvenient. Can we merge https://github.com/Homebrew/brew/pull/3564?

[vitorgalvao]

Reverted.