Mastodon: Remote media is not refetched if the original media URL no longer works

[rtucker]

Expected behaviour

Media attachments that can be retrieved from the remote instance should be retrieved and displayed by the local instance.

Actual behaviour

Media attachments with broken URLs are not fetched, even though they are retrievable from the remote instance.

A test case for this, which reproduces this issue on mastodon.social, is: https://mastodon.art/@gtccreations/100551804492070507

Steps to reproduce the problem

1. Verify that the media is retrievable from the remote instance, by going to https://mastodon.art/@gtccreations/100551804492070507.
2. Note that the URL for the first media attachment is https://cf.mastohost.com/v1/AUTH_91eb37814936490c95da7b85993cc2ff/mastodonart/media_attachments/files/001/135/101/original/573a094d763ab0a7.jpg.
3. Load the status on mastodon.social: https://mastodon.social/web/statuses/100551804485369913
4. The media attachment fails to load. The URL that mastodon.social is looking for is https://curate.mastodon.art/gallery/media_attachments/files/001/135/101/original/573a094d763ab0a7.jpg, which is broken.

Specifications

Found on vulpineclub/mastodon@prod-20181224-01, verified and reproduced on mastodon.social today.

[nightpool]

I don't think this is a bug in mastodon—there is no provision for remote server "updating" their image urls in mastodon's current implementation, and if you change image urls (as mastodon.art seems to have) without notifying the remote servers, then they're obviously still going to use the old ones. whoever did mastodon.art's migration needs to serve HTTP redirects from curate.mastodon.art to cf.mastohost.com

[nightpool]

cc @hugogameiro since this seems to concern a mastohost instance.

[hugogameiro]

Mastodon.art moved servers recently and the old media server is now down. I will work with the admin to create a redirect from the old subdomain to the new URL.

[hugogameiro]

I implemented a redirect for curate.mastodon.art and the old urls are now redirecting correctly.

[rtucker]

I'm not sure how this isn't a Mastodon bug? mastodon.art runs Mastodon, and failed to provide a persistent URL on the media. mastodon.social runs Mastodon, and failed to refresh the status when the media fetch failed. This has happened many times on many different instances, and the folks using the service do get confused by it.

I'm sorry but this is a bug, it's a Mastodon bug, and it's real.

[barzamin]

i think @nightpool's point is that from an AP standpoint, statuses are immutable after receipt and an "invalid" media url shouldn't force an update of an already-published status.

please note that you could actually use a behavior like that to edit past statuses as an adversarial remote server, by changing the presented remote post and 404ing media on it, causing a refetch. also note that this "refetching" behavior would be impossible when statuses are 'followers only', since those aren't remotely fetchable after receipt (that is, there is not a working /user/:user/status/:status AP endpoint to grab them, since we don't have remote user authorization).

[Gargron]

mastodon.art runs Mastodon, and failed to provide a persistent URL on the media

Operator error, not software error. Mastodon never refreshes statuses as they are considered immutable.

[rtucker]

Okay then why does it use ephemeral URLs for media in the first place? We do have a /media/ route, don't we?

Also I don't recall any documentation about this being a Bad Idea, and I'm gonna be boned if I ever switch away from AWS for media...

[rtucker]

In other words: this is a significant UX problem, and is also an undocumented pitfall for admins. I'll accept that we're fucked in the past, but I believe it can be fixed for future statuses by using persistent URLs.