Mastodon: Uploading media fails when using OpenStack Swift at master

[mayaeh]

My test instance is currently running with master 0a4739c73 .

And my test instance is using OpenStack Swift as the Mastodon Media Storage.

Then "500 error" is outputted when uploading the media and it is now to fail.

Also, when receiving toot with media from the outside, Sidekiq leaves "Fog :: Service :: NotFound: openstack has no storage service" error.

I think that it is related to #8877 .
Because it is output as "WARN:Fog::Service::NotFound: openstack has no storage service".

Full log is here ( https://gist.github.com/mayaeh/8f7b3381f39fa27e37b4fdc7bfc82d8e ).

---

• [X] I searched or browsed the repo’s other issues to ensure this is not a duplicate.
• [ ] This bug happens on a tagged release and not on master (If you're a user, don't worry about this).

[Gargron]

I don't know how to debug this, should we just revert the fog-openstack gem? They don't have a readable changelog and their commit log is full of merge commits so it's hard to read.

[mayaeh]

I would like to that wait for someone who receives the same error for a while.
I am also checking their commit, but it is hard...

[lo48576]

I had same problem.
Reverting da8abef56d5af3b3c706c939fc9f1595d6c215f2 and building web container, I can upload images to the object storage.

[Gargron]

@lo48576 Is downgrading both necessary, or only fog-openstack?

[lo48576]

Trying by this commit https://github.com/lo48576/mastodon/commit/bdf2e889433d3193343b640b71413fc94b99b659, fog-core 2.1.0 and fog-openstack 0.3.7 seems to work expectedly.

I think fog-core can be updated to 2.1.0.
However fog-openstack 0.3.* depends on fog-core <=2.1.0, so latest fog-core (currently 2.1.2) cannot be used.

[mayaeh]

I guess this will happen because Paperclip does not support the new name format.
( reference: https://github.com/fog/fog-openstack/pull/437/commits/391cda23afad6c0aa1ccd878e71250419a727edc )

I tested replacing ::Fog::Storage.new of Paperclip to ::Fog::OpenStack::Storage.new.

Then it appears to be able to upload media successfully with Mastodon.

However, it is declared "Deprecating Paperclip" .

By the way, since fog-openstack has backported version 0.3.7 has been released, I think that we can expect security corrections as well.
( For the history of Backport see here . )

But I could not find it in fog-core.

I think that the fog-openstack 0.3.7 can not handle the latest version of fog-core.

I think it is difficult to decide which choice to do.

[Gargron]

However, it is declared "Deprecating Paperclip" .

Even if paperclip is deprecated, Mastodon will not switch from it soon. ActiveStorage is not very good in terms of SQL joins necessary

I tested replacing ::Fog::Storage.new of Paperclip to ::Fog::OpenStack::Storage.new.

It might be possible to monkeypatch this, if that's the only cause of the issue...

[mayaeh]

Yes, I read this issue about ActiveStorage.

I wrote a modified commit of the paperclip, but this is only for OpenStack.
This causes problems if someone wants to support to fog-azure etc.

Also, if other problems are found in paperclip, I can not say that I can support it.

If you do not mind, I think I can use this but ...

[mayaeh]

Umm, this fix alone seems to be incomplete.
A warning such as "[fog][WARNING] Unrecognized arguments: provider" is output to syslog.
I will investigate a little more.

[hugogameiro]

I have been investigating this issue and don't think it's related to PaperClip.

Using fog-openstack (1.0.3), fog-core (2.1.2) I get "Fog :: Service :: NotFound: openstack has no storage service" but it also includes a warn:

[fog][DEPRECATION] Unable to require fog/openstack/storage
[fog][DEPRECATION] The format fog/storage/openstack is deprecated

This is exactly the same issue https://github.com/fog/fog/issues/4014 that happens using CarrierWave. It looks like it's an issue with fog-core.

I tested it on Mastodon the suggestion of overriding fog-core Fog::ServicesMixin#require_service_provider_library (to do nothing) and it works, no need to touch PaperClip.

So, let's hope fog-core fixes this. I am following the fog-core issue and if something new is posted there, I will update here.

[hugogameiro]

It looks like fog-openstack 1.0.8 solves this compatibility issue with fog-core: https://github.com/fog/fog/issues/4014#issuecomment-456759238