Mattermost-server: Tab in Firefox constantly signals notifications when there is none

[alerque]

Any Mattermost tab left open in Firefox perpetually signals that it has notifications or activity even when there are none.

This is 100% reproducable for me across several computers with different Firefox configurations. Open Mattermost (at least the last 2 releases have shown this, not sure when it started) in Firefox (56, & 57 have both show this, not sure before that) into some team chat (any team will do, even a sandbox with nobody else in it and no activity). Switch over to another tab. In about 60 seconds the tab will sight up signaling it has new activity.

Click onto the MM tab and the alert goes away, but there wasn't any reason for it to be there. Click away again and it will be back inside of a minute, again for no reason.

The web app should only tell the browser it has activity when there is really something to report: whether that is a ping or just any chat activity I don't know, but _no activity_ should not trigger it constantly.

Other web chat apps (Whatsapp, Stack Exchange chat, etc.) seem to trigger this at expected times.

[lindy65]

Hi @alerque,

Thanks for your feedback,

I tried to reproduce your issue in Firefox 55 but couldn't.

I then upgraded to the latest Firefox version, 57, where I have no messages in any channels and no activity and could not reproduce. Sending a message from another account correctly triggers activity and at-mentions also trigger correctly.

Which server version of Mattermost are you running?

[alerque]

I'm currently running Mattermost 4.4.5, but this has been a problem for several release cycles. I keep hoping the latest release will fix this but have been too lazy to file a bug report.

I suppose one thing to look into is the way I have Mattermost deployed behind a proxy server. The actual chat server runs on it's own (non public) port and the host that people access is serving it up via Apache 2.4 with SSL and proxing to the internal port.

Here is the entire Apache config for my setup:

<VirtualHost *:443>
        ServerName mattermost.alerque.com

        DocumentRoot /usr/share/webapps/mattermost/webapp/dist

        SSLEngine on
        SSLCertificateFile      /etc/letsencrypt/live/alerque.com/cert.pem
        SSLCertificateKeyFile   /etc/letsencrypt/live/alerque.com/privkey.pem
        SSLCertificateChainFile /etc/letsencrypt/live/alerque.com/chain.pem

        ProxyPreserveHost On

        RewriteEngine On

        RewriteCond %{REQUEST_URI}  ^/api/v1/websocket    [NC,OR]
        RewriteCond %{HTTP:UPGRADE} ^WebSocket$           [NC,OR]
        RewriteCond %{HTTP:CONNECTION} ^Upgrade$          [NC]
        RewriteRule .* ws://127.0.0.1:8065%{REQUEST_URI}  [P,QSA,L]

        RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f
        RewriteRule .* http://127.0.0.1:8065%{REQUEST_URI} [P,QSA,L]
        RequestHeader set X-Forwarded-Proto "https"

        <Location /api/v4/websocket>
                Require all granted
                ProxyPassReverse ws://127.0.0.1:8065/api/v4/websocket
                ProxyPassReverseCookieDomain 127.0.0.1 mattermost.alerque.com
        </Location>

        <Location />
                Require all granted
                ProxyPassReverse http://127.0.0.1:8065/
                ProxyPassReverseCookieDomain 127.0.0.1 mattermost.alerque.com
        </Location>

        LogFormat "%{X-Forwarded-For}i %l %u %t \"%r\" %>s %b" common_forwarded
        ErrorLog  /var/log/httpd/mattermost.alerque.com_error.log
        CustomLog /var/log/httpd/mattermost.alerque.com_forwarded.log common_forwarded
        CustomLog /var/log/httpd/mattermost.alerque.com_access.log combined env=!dontlog
        CustomLog /var/log/httpd/mattermost.alerque.com.log combined
</VirtualHost>

[alerque]

I guess I should also note that the icon change that lights up the favicon when I get pinged does work properly as expected, this activity indicator is independent of that.

[lindy65]

Thanks @alerque,

Mattermost 4.5 released on December 16th and 4.6 goes out on January 16th... I'm wondering whether you should try upgrading to the latest version because I am not able to reproduce your issue on our nightly build server running off the latest master.

Could be there was an issue that was fixed in 4.5 or on master

[alerque]

Thanks @lindy65 . I was unaware that I wasn't on the latest release. It looks like my distro might be held up over problems with the mozjpeg dependency. I will retest this and confirm or close as soon as I can get upgraded. It will be apparent if something changed in just under 60 seconds after upgrade.

[lindy65]

Perfect, thanks @alerque!

Please do let us know how it goes after upgrade...

[alerque]

@lindy65 I just got upgraded to 4.5.0 and this errant behavior is still happening for me like clockwork.

[lindy65]

Hi @alerque,

Thanks for letting us know... I'll ask one of my team-mates to weigh in on your issue as it is not reproducing for me but I am testing on Windows so it could be something in your Apache set-up...

[lindy65]

Hi @alerque,

Apologies for the delay in responding... We have been trying to reproduce your issue and are still investigating.

@sousapro feels it might be a proxy issue of some kind and is investigating further.

[alerque]

If there is anything I can provide to debug this further please let me know.

[lindy65]

Thanks @alerque,

@sousapro is going to do a local deploy on his home server, set up apache as a proxy and see if he can reproduce.

Hope to have some feedback for you soon...

[SmartHoneybee]

@alerque, can you reproduce the issue with a different browser?
Any relevant output in the developer console of your browser?
(Did you try to follow the "Unofficial, community-maintained guides for configuring Apache as a proxy instead of NGINX"? Your configuration looks slightly different, hence the question.)

[sousapro]

also, if you upgrade to version 4.5.1 or 4.6.0 does the issue still persist? (there were other firefox v52+ issues that were resolved in these versions)

[alerque]

@SmartHoneybee I do remember checking out Chrome and _not_ seeing this behavior but I didn't do a very exhaustive test.

I did try to follow that guide originally, but there was some reason I had to implement a modified version of the Apache config. It has been some time and I don't remember exactly what the issue was.

@sousapro I just updated to 4.6 and I guess we'll see. As I type this I'm waiting for my MM tab to light up but it hasn't yet and I think were past 60 seconds, so something did change. I'll keep an eye on it today and give a more definitive report later.

Edit: Nope we're back to the same behavior as afar as lighting up as if there was activity wheen there isn't.

[alerque]

This seems to have been caused by my Apache config being outdated and not matching current API versions. I've submitted and update to the documentation so others don't run into this problem. It's been a whole 10 minutes now and I haven't seen this recurr, so I think we're good here.

Thanks for all the help everybody. For the record, you guys make Mattermost one of the easiest open source projects to participate in that I've even run across.

[lindy65]

Glad your issue is resolved @alerque :)

Thanks for the doc PR - we'll get that through the review process as soon as we can.

[alerque]

I'm afraid I spoke too soon. This behavior seems to have gotten a bit more slippery with the latest changes to my config, but it hasn't actually gone away.

I'm now running Mattermost 4.6.0, Apache 2.4.29, and almost exactly the proxy configuration from my docs PR. Testing on Firefox 57.0.4 on Linux.

I finally had the bright idea to run the developer tools in a separate window instead of as a pane so that I can watch the logs while the tab is inactive, and have noticed that the tab status lighting up exactly corresponds to what appears to be an XHR poll request sent via POST to and API endpoint apparently looking for current user statuses.

[alerque]

Okay more information! This only happens when this Firefox addon is active. Whew, now we're getting somewhere.

The question now is why is a URL change event being triggered every time Mattermost polls for user statuses? The URL (at least as visible in the address bar) doesn't seem to be changing. Lots of other apps also sit around polling servers using similar XHR requests and I don't my tab bar isn't lighting up like Christmas, only Mattermost keeps coming on.

[saturninoabril]

I've posted a comment to your PR like if it should have / before websocket. Let me know if that makes sense, and try if it still reproduces your issue?

[alerque]

@saturninoabril Thanks for the suggestion, but as I commented there (with docs) I don't think that change makes any sense at all.

[saturninoabril]

Thanks for clarifying! Since we're not able to repro the issue using browsers (including FF without the said addon), feel free to close this issue at the moment and reopen if necessary.

[alerque]

@saturninoabril I'm sorry but that seems rather hasty to me. This still seems very much like Mattermost is touching some browser API when it shouldn't be. Just because you don't see this problem in plain FF doesn't mean Mattermost is doing everything properly. There are now two users who have noticed this bad interaction, but this plugin isn't causing this on any other sites (Slack, Gitter, Whatsapp Web, Hangouts chat, Facebook chat, Stack Exchange chat, and others come to mind). Mattermost is doing something fairly unusual here given that nobody else's web apps are triggering this issue. Also the source code for that plugin is a grand total of 44 lines of JS, of which only about 6 is remotely relevant.

browser.tabs.onUpdated.addListener(updateUrl);

The tab seems to be lighting up because it thinks the title changed, but it thinks the title change because something is telling the browser that it's been updated even when it hasn't. The plugin isn't telling the browser the tab has been updated, the browser is telling the plugin that. And somehow, Mattermost is triggering the browser doing that.

[saturninoabril]

@alerque I've tried to install the said plugin and still it didn't repro on me. We have built-in notification which are configurable by the user and it works fine as far as I can see.
We would certainly welcome fix on this matter.

[alerque]

@cloutierjo Was your reproduction of this issue also on a Mattermost instance behind an Apache proxy?

[cloutierjo]

I'm not in charge of the mattermost installation but from our internal doc I believe it's behind an nginx proxy.

[lindy65]

@alerque,

I'm going to close this issue off for now as we have not been able to reproduce it. We don't have an Apache proxy though so this could be where the issue lies.

v4.7 releases on Feb 16th so perhaps you could try reproducing the issue after you upgrade to see if it's still occurring?