Mastodon: OpenSSL - SSLError : certificate verify failed

[valentinp72]

Hello,

Since the 18 April, around 19h30 (07:30 pm) my instance isn't federating. I tried to restart mastodon, apache, even the server, and nothing work.
At first, I thought it was a proxy error, because I had error in my apache errors file about proxy. Now it seems to work, there are no more errors, but it's still not federating.

On sidekiq, there are a lot of failed and "retries" jobs. It all says :
OpenSSL::SSL::SSLError: SSL_connect returned=1 errno=0 state=error: certificate verify failed

My openssl version is OpenSSL 1.1.0f-dev xx XXX xxxx.

I renewed my certificates, but this doesn't solve the problem...

I have updated to v1.2 yesterday, and now I'm on v1.2.2.

I'm lost, what could I do to solve this ?

---

• [x] I searched or browsed the repo’s other issues to ensure this is not a duplicate.
• [x] This bug happens on a tagged release and not on master (If you're a user, don't worry about this).

[lutoma]

Do you have the ca-certificates bundle installed? Which Linux distribution do you use?

I'm not sure at which paths the ruby OpenSSL module looks by default, but maybe you'll have to set it to the correct location for your distribution.

[celesteh]

I had a similar-looking problem with let's encrypt certificates. This pointed me in the right direction: https://github.com/certbot/certbot/issues/2026

[valentinp72]

I'm on Debian (Jessie). I installed ca-certificates.

ruby -ropenssl -e 'p OpenSSL::X509::DEFAULT_CERT_FILE' tells me "/usr/lib/ssl/cert.pem".

I'm using let's encrypt, so I have put the let's encrypt chain.pem + ca-certifcates.crt inside /usr/lib/ssl/cert.pem, but it's still doesn't work...
Is this what you have done @celesteh ?

Thanks!

[halest]

I have the same problem since Gargron bumped the Ruby-Version.
I don't know if my instance is still federating, I've stumbled upon this problem realizing my instance doesn't send mails any more because of the error. I've tried three E-Mail-Servers with various certificates and setting the SSL-Options to OpenSSL::SSL::VERIFY_NONE but the problem still persists.

[ToroNZ]

Have you guys checked #1496 or #1486 ?

[halest]

Hey @ToroNZ . Thanks for pointing me in that direction. I patched it manually and now it is sending mails again.

[valentinp72]

Hey @ToroNZ: I tried to add the cert file like here, but it only concerns SMTP, that changed nothing about federation... :|

[bnjbvr]

I hit the same issue and I think I've got a lead. See the logs containing the Arguments to this job.

One URL there that appears a lot is bernard.tcit.fr. This is weird, because the mastodon instance hosted there is called social.tcit.fr. I wonder if the admin hasn't moved domain names and forgot to change something in their configuration. When trying to access bernard.tcit.fr with my browser, I get a SSL exception saying that the certificate is valid for social.tcit.fr. Since this triggers a SSL error in Ruby code, I assume this could be the same cause. I've contacted the admin and asked them to see if there was something wrong in their configuration.

However, Sidekiq could probably be more resilient here and just 1. avoid fetching data on a domain that has a SSL issue, with a way to warn the administrator ideally, or 2. just have a way to ignore SSL errors with all/some domains.

[bnjbvr]

The following is fixing the issue locally, but I am very unsure about my ruby, so I'll defer to somebody who can make a better patch out of this.

commit 68c39fc917479b9f1eb7a6525b1b6f0160d1afac
Author: Benjamin Bouvier <[email protected]>
Date:   Mon Apr 24 02:18:33 2017 +0200

    Fix fetch of avatar/header if SSL is incorrectly set on the instance;

diff --git a/app/models/account.rb b/app/models/account.rb
index 084b17f..b263060 100644
--- a/app/models/account.rb
+++ b/app/models/account.rb
@@ -188,6 +188,8 @@ class Account < ApplicationRecord

     self.avatar              = parsed_url
     self[:avatar_remote_url] = url
+  rescue OpenSSL::SSL::SSLError => e
+    Rails.logger.debug "Error fetching remote avatar: #{e}"
   rescue OpenURI::HTTPError => e
     Rails.logger.debug "Error fetching remote avatar: #{e}"
   end
@@ -199,6 +201,8 @@ class Account < ApplicationRecord

     self.header              = parsed_url
     self[:header_remote_url] = url
+  rescue OpenSSL::SSL::SSLError => e
+    Rails.logger.debug "Error fetching remote avatar: #{e}"
   rescue OpenURI::HTTPError => e
     Rails.logger.debug "Error fetching remote header: #{e}"
   end