Https-everywhere: Research broken bbc.co.uk / player.fm audio podcast feed

[ldexterldesign]

Type: ruleset issue

Domain: http://bbc.co.uk

Hi all,

Thanks for software

Hope you're well

This[s] audio podcast's episodes won't play with http everywhere installed. Although the .rss feed is served as https the .mp3 episodes are served as http.

It's actually an issue with all the BBC podcasts I subscribe to

Before I attempt to contact the BBC, prompting their adoption of universal https, please confirm whether:

a) BBCs lack of https support is the root of the problem, or;
b) it's a https everywhere ruleset issue, or;
c) both?

NB I attempted to fix this issue by disabling some https everywhere features but no luck which makes me suspicious this isn't just a)

It would sure be nice to have my podcasts back without having to disable https everywhere extension

If you have any issues (e.g. questions/queries) happy to help

Hope this is clear/useful and to hear back

Keep up great work!

Reference:

https://twitter.com/TrustifyCyber/status/1050745889806008321
https://twitter.com/getify/status/1043970491743113216
https://twitter.com/CydeWeys/status/1050795094931062785
https://twitter.com/alexcomninos/status/1049923164803911680

Related:

https://github.com/EFForg/https-everywhere/issues/275
https://github.com/EFForg/https-everywhere/issues/15218

Yours faithfully

PS couldn't really make sense of https://www.eff.org/https-everywhere/atlas/domains/bbci.co.uk.html so if it's relevant I'd love to know how to interpret this resource in the future

[s] https://player.fm/series/kermode-and-mayos-film-review-1301420/with-jamie-lee-curtis

[numismatika]

In the past only a few things on the bbc websites could be fetched via TLS without being redirected to plain.
This made the rules quite ugly. Recently the BBC began to secure their websites and announced that on their blog.
So before we go over the bbc rules again, it would make sense to wait until they are done converting their website to fully support TLS.

[ldexterldesign]

Hi @numismatika,

Thanks for reply

Recently the BBC began to secure their websites and announced that on their blog

What's the source please?

Hope to hear back

Yours faithfully

[numismatika]

http://www.bbc.co.uk/blogs/internet/entries/f6f50d1f-a879-4999-bc6d-6634a71e2e60
http://www.bbc.co.uk/blogs/internet/entries/a6604322-99a9-4272-860c-f78e667e18e3
It is older than i remembered.
Maybe we should try again with a clean file with no exceptions and go from there.

[numismatika]

It is work in progress in : https://github.com/numismatika/https-everywhere/tree/bbc_TLS
Right now i am just testing the base urls for the most obvious redirects to plain.
Then i will browse the page and see which subfolders cause problems.

[ldexterldesign]

BUMP

[pipboy96]

@ldexterldesign Is this issue still relevant?

[ldexterldesign]

@pipboy96 yes 😞, I disable this extension on https://player.fm now so my podcasts stream back-to-back without stopping

[pipboy96]

@ldexterldesign Can it be tested without an active BBC UK account?

[ldexterldesign]

@pipboy96 yea, totally, use the URL in my OP or just go [h]ere and hit play

h: https://player.fm/series/kermode-and-mayos-film-review-1301420

[pipboy96]

@ldexterldesign Confirmed!

[pipboy96]

The reason is the URL the <audio> tag on the page is pointing at is not available over an encrypted connection. Until the website owners will fix that, we would have to disable stitcher.acast.com domain.

[pipboy96]

High priority since a very high-traffic website is broken. If this is added by mistake, please remove.

[ldexterldesign]

For convenience...

Not using podcast app':

1. https://www.bbc.co.uk/programmes/m0007dn0
2. https://www.bbc.co.uk/sounds/play/m0007dn0

Using player.fm and RSS (broken with http-everywhere):

1. https://player.fm/series/kermode-and-mayos-film-review-1301420/episode-239547342
2. http://open.live.bbc.co.uk/mediaselector/6/redir/version/2.0/mediaset/audio-nondrm-download/proto/http/vpid/p07k7lxh.mp3 redirect...
   2.1 ... redirect https://stitcher.acast.com/livestitches/3a59ac07-148b-4599-83ed-cd9156da14b7/fbdd750548fd668632dcb54f95fd2b52.mp3?aid=3a59ac07-148b-4599-83ed-cd9156da14b7&chid=761083cd-3c59-4246-b8d7-a5a9288d43aa&ci=303cdae4-5cae-4868-989f-3813d1862c00&pf=rss&uid=fbae889f7cad7b5f46a97b5d40d4d60c&Expires=1578534047&Signature=GRPDKDaEcPX3VdJKuXOyUkg-T-M6G3iqf6NhOmFnPwhmAQNz6VxBVoOobGPlxUzMMQu5tg~Ymqex01uj1ciZ98~AEL6rgcRTkzOlGEM~1x0HmiUC7gYkbebx2wm1WOoan2nDcc9h2XO3VMPrpi3P5no-Svo4~cEH42zRyb8r~NSKv3vKoRDnpK6yRccxFmhTB-LJnuYcLOy-SY5EQUwAPx-8wKMFdOprVrQpc353gNLhE20m61tjPQTmvc-xVNWqy~jJb76fV23aNJPTWCxD0WscH5~SVkP6Q58lcDEskAI4~F-SAA~J8sWNrpw4w86Kxu5VGddqsGEv-1jA6mfXvg__&Key-Pair-Id=APKAJXAFARUOTJQ3BLOQ

So [t]his will explain more

[A]cast's business model is paywalling content and I gather they're using https to enforce this so this situation ain't likely to change any time soon from a Player FM / Acast / BBC perspective

Regards

a: https://en.wikipedia.org/wiki/Acast
t: https://techcrunch.com/2018/05/01/bbc-acast/

[pipboy96]

@ldexterldesign Thanks! This should allow fixing the issue instead of disabling the domain.

[pipboy96]

@ldexterldesign I confirmed a fix works correctly. I will create a PR soon!

[pipboy96]

@ldexterldesign #18310.

[ldexterldesign]

FYI Encrypt All Sites Eligible is ON (i.e. Unencrypted requests are currently blocked) seems to fix this issue, which I believe is OFF (i.e. Unencrypted requests are currently allowed) by default because I've only just discovered it

This seems to be the total opposite of what I think it does?!

If I wanted any chance of playing my blocked podcast then I would set this feature to off and allow unencrypted requests - please someone explain what's going on here and why setting it to on fixes my issue?

Yours hopefully

[pipboy96]

@ldexterldesign EASE fixes it since it makes the first request (to flex.acast.com) to be sent over HTTPS, which causes it to return a HTTPS link to stitcher.acast.com that has a correct hash (which is not reproducible in any other way other than having that first request sent over HTTPS).

[ldexterldesign]

@pipboy96 when/where in the UX does that flex.acast.com request occur - I've missed that?

[pipboy96]

flex.acast.com redirects to the actual sound file that's stored on stitcher.acast.com domain (the link is unique and also depends on http/https).

[pipboy96]

The fix for this issue has been merged and will be shipped as a part of next ruleset update!