Meteor-files: File download not possible in IE11 (Internet Explorer) - VeliovGroup/Meteor-Cookies problem?

[derwok]

meteor 1.4.2.3
ostrio:[email protected]
ostrio:[email protected]

Unfortunately we have to support IE11 :-( (I know, I know... - sad world).
VeliovGroup/Meteor-Files works great with all other browsers - so we where hit by surprise when we found out the infamous IE11 makes trouble again, here... :-(

So what happens?
With IE11 it is possible to upload and remove files. But it is not possible to download files with an URL like this, while having a downloadCallback that checks for a logged in user:
http://www.myserver.com:3100/cdn/storage/AttachmentsCollection/oDYpTaG7k7xQGNEW3/original/oDYpTaG7k7xQGNEW3.txt?download=true

* IE11 gives a "HTTP 404" error.*

We have to protect files against not logged in users. Looking on the server the 404 problem seems to be as our FilesCollection/downloadCallback returns a "false" here:

    if (! this.userId) {
        console.log("Attachment download prohibited. User not logged in.");
        return false;
    }

If we dump the "this" in the above context we see that there is actually no userId available:

[...]
I20170113-07:56:19.481(1)?   params:
I20170113-07:56:19.483(1)?    { query: { download: 'true' },
I20170113-07:56:19.484(1)?      _id: 'oDYpTaG7k7xQGNEW3',
I20170113-07:56:19.484(1)?      version: 'original',
I20170113-07:56:19.484(1)?      name: 'oDYpTaG7k7xQGNEW3.txt?download=true' },
I20170113-07:56:19.494(1)?   user: [Function: user],
I20170113-07:56:19.494(1)?   userId: null }

Taking a closer look at the server log we see that the problem seems to arise way before the actual download. If the IE11 user signs in to our app with the meteor accounts package we see this in the error log:

W20170113-07:37:52.782(1)? (STDERR) Error: `value` required in setHeader("Set-Cookie", value).
W20170113-07:37:52.782(1)? (STDERR)     at ServerResponse.OutgoingMessage.setHeader (_http_outgoing.js:342:11)
W20170113-07:37:52.782(1)? (STDERR)     at ServerResponse.appendHeader (/Users/wok/.meteor/packages/webapp/.1.3.12.10jgpt3++os+web.browser+web.cordova/npm/node_modules/connect/lib/patch.js:95:33)
W20170113-07:37:52.782(1)? (STDERR)     at ServerResponse.res.setHeader (/Users/wok/.meteor/packages/webapp/.1.3.12.10jgpt3++os+web.browser+web.cordova/npm/node_modules/connect/lib/patch.js:126:19)
W20170113-07:37:52.783(1)? (STDERR)     at Object.handle (packages/ostrio_cookies.js:509:21)
W20170113-07:37:52.784(1)? (STDERR)     at next (/Users/wok/.meteor/packages/webapp/.1.3.12.10jgpt3++os+web.browser+web.cordova/npm/node_modules/connect/lib/proto.js:174:15)
W20170113-07:37:52.784(1)? (STDERR)     at Function.app.handle (/Users/wok/.meteor/packages/webapp/.1.3.12.10jgpt3++os+web.browser+web.cordova/npm/node_modules/connect/lib/proto.js:182:3)
W20170113-07:37:52.784(1)? (STDERR)     at Object.fn [as handle] (/Users/wok/.meteor/packages/webapp/.1.3.12.10jgpt3++os+web.browser+web.cordova/npm/node_modules/connect/lib/proto.js:79:14)
W20170113-07:37:52.785(1)? (STDERR)     at next (/Users/wok/.meteor/packages/webapp/.1.3.12.10jgpt3++os+web.browser+web.cordova/npm/node_modules/connect/lib/proto.js:174:15)
W20170113-07:37:52.785(1)? (STDERR)     at Object.WebAppInternals.staticFilesMiddleware (packages/webapp/webapp_server.js:369:5)
W20170113-07:37:52.785(1)? (STDERR)     at packages/webapp/webapp_server.js:642:22

In the context of meteors "staticFilesMiddleware" here this line of interest: at Object.handle (packages/ostrio_cookies.js:509:21)
So, the root cause seems to be a problem with VeliovGroup/Meteor-Cookies here.
(By the way: should we file the issue over there?!)

Would be great if you could have look into this issue?

Thanks for all your great work!
We hobe you have a clue what's going on here...

Thanks in advance.
Wolfram.

[derwok]

Added version numbers at top of issue text...

[dr-dimitru]

Hello @derwok

So, issue is in login/authentication process, not in download, right?

1. What cookies do you have?
2. What in localStorage?
3. What value of Meteor.connection._lastSessionId in browser?
4. Is user 100% logged in? Meteor.user()
5. What limits of IE 11? Does it supports WebSockets?

[derwok]

Hi @dr-dimitru ,
Thanks for asking.

Yes - the problem seems to be during the login phase. But it seems to be in the code of your Meteor-Cookies sub-package, as the above call stack shows.
see line at Object.handle (packages/ostrio_cookies.js:509:21).

Funny: the complete rest of the application with all Meteor.user() Meteor.userId checks works 100% perfect... So - metoer login and session handling works and all checks in frontend and backend work also correctly. So the problem seems to be during login phase where your code tries to store the cookie for Meteor Files...
To your questions:
IE11 supports WebSockets according to here: http://caniuse.com/#feat=websockets
The user is completely logged in when he clicks on the download link (which does not work). At the moment when the above exception occurs - I have no clue, if the user is 100% logged in, because the exception call stack is somewhere beyond our control (meaning: not our code).

The problem with question 1-4 above is: The exception happens on server during login. So, as your questions are all client/browser questions: you want me to get this information client side after login - right? At this time point the server exception already happended. So, its too late...

I will try to collect the data soon. And comment below.

In the mean time - if you can get your hands on a Windows with IE11 on it somewhere... you can either:

1. Head over to https://www.4minitz.com
2. Login with demo/demo
3. Click on the meeting series "Test - Meteor-Files"
4. Click the date of today's meeting 2017-01-13
5. Scroll to attachments section and try to download the file

Or - if you may investigate client and server side:

git clone https://github.com/4minitz/4minitz.git
cd 4minitz
meteor run --settings ./settings_sample.json

And then login with IE11 browser and user: demo and password: demo.
You will immediately see the above server side exception on login.
And its not your code somewhere in the exception call stack...

[dr-dimitru]

@derwok Thank you for details.
I need to find a PC or deploy VM to investigate this.

Please provide Windows version and IE 11 full version including build (if exists).

As investigation may took some time, here is temporary solution - https://github.com/thomasspiesser/ostrio-files-bug/pull/1/files
Where we attempted to fix similar issue

Please read this thread - https://github.com/VeliovGroup/Meteor-Files/issues/159
I believe it has same roots.

What you're having now is Server issue not client.
Here at Cookies package empty value is passed to header. That means it's not sent from client.

Crated issue on Cookies package - https://github.com/VeliovGroup/Meteor-Cookies/issues/6

[derwok]

Hi,

Smallest meteor app to reproduce

I just managed to build the smallest meteor app to reproduce the error...
I was not able to reproduce the problem with the Meteor-Cookies package alone. I needed to add the Meteor-Files package. So, instead of cloning our full blown 4minitz app you can also clone this repo:

https://github.com/derwok/cookietest

After a simple meteor create I just added these packages:

meteor add accounts-ui 
meteor add accounts-password 
meteor add ecmascript
meteor add ostrio:[email protected]

There are only 3 commits. Commit #2 everithing is fine:
https://github.com/derwok/cookietest/commit/8ab49dd979116731c173924d4485d12f7988d843

After commit #3 you can see the server side exception after login/register with IE11. Klick link to see the code diff of commit #3:
https://github.com/derwok/cookietest/commit/84dd0a7816abf47ec57e135a64eb0752cd40e32c

When you cloned & run the cookiestest meteor app above:
Then open webapp in the IE11 browser, and simply register a new user with arbitrary email (no validation happening!) and password of at least 6 chars. As soon as the user is registered, the user is is logged in and you will see the server exception triggered by the cookies package:

I20170114-09:27:14.948(1)? Login of user: ELJB4aou57pADMAAA
W20170114-09:27:15.001(1)? (STDERR) Error: `value` required in setHeader("Set-Cookie", value).
W20170114-09:27:15.001(1)? (STDERR)     at ServerResponse.OutgoingMessage.setHeader (_http_outgoing.js:342:11)
W20170114-09:27:15.001(1)? (STDERR)     at ServerResponse.appendHeader (/Users/wok/.meteor/packages/webapp/.1.3.12.10jgpt3++os+web.browser+web.cordova/npm/node_modules/connect/lib/patch.js:95:33)
W20170114-09:27:15.002(1)? (STDERR)     at ServerResponse.res.setHeader (/Users/wok/.meteor/packages/webapp/.1.3.12.10jgpt3++os+web.browser+web.cordova/npm/node_modules/connect/lib/patch.js:126:19)
W20170114-09:27:15.002(1)? (STDERR)     at Object.handle (packages/ostrio_cookies.js:509:21)
W20170114-09:27:15.002(1)? (STDERR)     at next (/Users/wok/.meteor/packages/webapp/.1.3.12.10jgpt3++os+web.browser+web.cordova/npm/node_modules/connect/lib/proto.js:174:15)
W20170114-09:27:15.003(1)? (STDERR)     at Function.app.handle (/Users/wok/.meteor/packages/webapp/.1.3.12.10jgpt3++os+web.browser+web.cordova/npm/node_modules/connect/lib/proto.js:182:3)
W20170114-09:27:15.003(1)? (STDERR)     at Object.fn [as handle] (/Users/wok/.meteor/packages/webapp/.1.3.12.10jgpt3++os+web.browser+web.cordova/npm/node_modules/connect/lib/proto.js:79:14)
W20170114-09:27:15.003(1)? (STDERR)     at next (/Users/wok/.meteor/packages/webapp/.1.3.12.10jgpt3++os+web.browser+web.cordova/npm/node_modules/connect/lib/proto.js:174:15)
W20170114-09:27:15.003(1)? (STDERR)     at Object.WebAppInternals.staticFilesMiddleware (packages/webapp/webapp_server.js:369:5)
W20170114-09:27:15.004(1)? (STDERR)     at packages/webapp/webapp_server.js:642:22

Here are some answers to your questions:

0. Im am testing on
Serverside: MacOS with meteor 1.4.2.3

Clientside:

   "Windows 7 Professional" with ServicePack 1 and current patches
   "Internet Explorer" v11.0.9600.18537

According to here:
https://www.netmarketshare.com/browser-market-share.aspx?qprid=2&qpcustomd=0

IE11 still holds a share of >10% of the global browser market. Though it is known for
handling some things not 100% as the rest of the big browsers... I think it is worth a
consideration in our apps, as especially (large) companies currently still stick to Windows 7 or Windows 8 and IE11 is the latest Internet Explorer that runs on these systems. In these companies staff is not allowed to install FF / Chrome and MS Edge wont run on these legacy windows versions. Edge need Windows 10.

1.What cookies do you have?

In IE11 you can show the cookies as descibed here:
http://stackoverflow.com/questions/19941029/how-do-i-view-cookies-in-internet-explorer-11-using-developer-tools

When I do as described above, and I reload my page and login, I see the server side exception.
But there show no cookies up on the client. The windows that should show the cookies is simply empty.

2.What in localStorage?
After server side login exception, I am nevertheless logged in on the client.
When opening the F12 dev console, and I enter "localStorage" I get the following output:

{
   [functions]: ,
   __proto__: { },
   constructor: { },
   length: 3,
   Meteor.loginToken: "6XOUVJ1p-0nSR4YNv-zGJyyeQjZ79ZcP8EK0NjB1be4",
   Meteor.loginTokenExpires: "Fri Apr 14 2017 10:08:00 GMT+0200 (Mitteleuropäische Sommerzeit)",
   Meteor.userId: "ELJB4aou57pADMAAA",
   remainingSpace: 4999822
}

3.What value of Meteor.connection._lastSessionId in browser?

"34PBFFQRbfA8cMKqA"

4.Is user 100% logged in? Meteor.user()

On the client:
{
   [functions]: ,
   __proto__: { },
   _id: "ELJB4aou57pADMAAA",
   emails: [ ]
}

5.What limits of IE 11? Does it supports WebSockets?

Yes IE11 is known to handle things a bit strange sometimes...
Nevertheless here is stated websockets and localStorage work just fine in IE11:
http://caniuse.com/#feat=websockets
http://caniuse.com/#search=localstorage

I hope, this helps in tracking the problem down.
Thanks for taking care!
Your commitment is greatly appreciated!

Kind regards.
Wolfram.

[dr-dimitru]

Hello @derwok ,

Could you try v1.7.7 from dev branch? (have no access to IE now)

[derwok]

Hey! Great News! Yes, sure! I'll try! Stay tuned.

[dr-dimitru]

@derwok pushed more critical changes related to this issue into dev

[derwok]

Hi @dr-dimitru,

The good news: I do not see the Cookie-Exception on user login any more.

But unfortunately I have also sad news... (at least for me) ;-)
IE11 download still does not work - no cookie gets set.
(Firefox, Chrome, Safari - all work fine!)

What did I do?
I removed the official ostrio:files 1.7.6 from my project, cloned the current Meteor-Files /dev/ branch in my local /packages/ directory in the root of my project. I checked the cloned package.js - yes! I got version 1.7.7.
Then I did:

$ meteor add ostrio:files
ostrio:files: updating npm dependencies -- fs-extra, request, throttle, file-type...

Changes to your project's package version selections:

ostrio:cookies  added, version 2.1.1
ostrio:files    added, version 1.7.7

I rebuild the meteor app with ostrio:files 1.7.7.
Uploaded file with IE11 - this still works fine (as before).
Then tried to download a file - I get a 404 Error with Text "File Not Found :(".
I checked the IE11 Developer Tools (F12 key), I recorded the network traffic from start & login - but NO COOKIES SET in IE11.
I double checked with a different website with heavy cookie usage - these cookies all show up.

I think, because no cookies arrive at IE11, the user check in my downloadCallback fails...
If inside the downloadCallback I do a console.log(this) I get a userId==null:

I20170119-23:34:58.146(1)?   params:
I20170119-23:34:58.147(1)?    { query: { download: 'true' },
I20170119-23:34:58.147(1)?      _id: 'CGijPAyfmB6mfqiQf',
I20170119-23:34:58.147(1)?      version: 'original',
I20170119-23:34:58.148(1)?      name: 'CGijPAyfmB6mfqiQf.jpg?download=true' },
I20170119-23:34:58.148(1)?   user: [Function: user],
I20170119-23:34:58.148(1)?   userId: null }

Any clue what's going on here?!

[derwok]

Maybe this is of help?
http://stackoverflow.com/questions/22690114/internet-explorer-11-wont-set-cookies-on-a-site

[dr-dimitru]

@derwok thank you, helped a lot.

Updated cookies package to v2.1.2

Please see updated v1.7.7 in dev
Let me know if this issue still persists for you.

[macrozone]

can also reproduce on IE11, update on 1.7.7 fixes the error about the cookies on file download,
but file can't be accessed. The mtok-cookie is set, but it's not send with the image-request. Not sure why this is happening

[macrozone]

ok here is the problem:

the cookie value has to be written in quotes:

document.cookie = 'x_mtok="abc1234"; path=/';

(Edit: that is not the problem, see below)

Btw. why re-implementing the whole cookie handling and not using an already existing cookie library that has been tested properly on all browsers to set and get cookies? e.g. this looks good https://github.com/js-cookie/js-cookie . Sounds like the "not-inveted-here"-syndrom.

My workaround is now:

downgraded back to 1.7.5 and...

if (Meteor.isClient) {
  const setTokenCookie = () => {
    document.cookie = 'x_mtok="' + Meteor.connection._lastSessionId + '"; path=/';
  };
  const unsetTokenCookie = () => {
    document.cookie = 'x_mtok=""; path=/';
  };
  Accounts.onLogin(setTokenCookie);
  Accounts.onLogout(unsetTokenCookie);
}

Edit: actually, this solution makes not sense, but it seems to work, at least on development...
IE seems to behave absolut weird concerning cookies. I would definitly use a bullet-proove battle-tested cookie-library to set cookies

[dr-dimitru]

Hello @macrozone ,

Btw. why re-implementing the whole cookie handling and not using an already existing cookie library that has been tested properly on all browsers to set and get cookies?

100% agree. Thats why Meteor-Cookies is based on top of jshttp/cookie package.

I haven't found any reference, which tells to wrap cookie value into quotes. jshttp/cookie package does not wraps value, and js-cookie package leaves value unwrapped

Could you please run Meteor-Cookies package tests? meteor test-packages ./

Need to confirm I'm subject of NIH syndrome... can't do anything about it... :(

[macrozone]

Hi @dr-dimitru

Meteor-cookies does its own version of cookie-parsing (https://github.com/VeliovGroup/Meteor-Cookies/blob/master/cookies.js#L56) and setting (https://github.com/VeliovGroup/Meteor-Cookies/blob/master/cookies.js#L95). consider using an an npm package here

Yeah, That thing with wrapping the values in values in quotes is complete bs (its strange that it works...).

I just was very confused when debugging this on the IE. I tried to set an arbitrary cookie by pasting this in the IE11-console:

document.cookie = 'thecookie=value; path=/';

and checked the network tab on the IE to see if this cookie is present, and it wasnt.

Then i tried document.cookie = 'thecookie="value; path=/'; and it worked...

But i really don't trust the dev-tools in IE... sometimes cookies are there, sometimes not... its a mess

I forgot to write above that i wrapped the document.cookie in a Meteor.setTimeout(() => {...}, 300)

In your code, the timeout is 25. I did not investigate much on this, but maybe Meteor.connection._lastSessionId is not defined sometimes, maybe a timing issue?

[dr-dimitru]

I'm asking to run tests, as I got access to MS Win 8 IE 11. All Meteor-Cookies package tests is passed on my end. Tests for protected files (cookie validation) is passed too. So, if you have a chance could you please run tests and/or try to access protected files at https://files.veliov.com

To do so:

1. Go to https://files.veliov.com
2. Sing up via Meteor Account / Twitter / Facebook / GitHub
3. Upload a file
4. Make sure it's protected (lock icon)
5. Click on download button

If file is downloaded in IE - you're authorised via cookies

[macrozone]

I opend it on IE11 windows 8 and could not login, because the info button is above the login button...

I removed it know through dev tools... and yeah, it works in IE.

I try now to checkout the develop-branch again and see if it works... I still think we have a weird timing issue somewhere

[dr-dimitru]

dev and master currently at the same state.
files.veliov.com is up to date too

[dr-dimitru]

because the info button is above the login button...

PR is welcome here

[macrozone]

oh i see... i think i checked out master not develop before, my bad!

[macrozone]

Ok i checked it out again and it still did not work. But i think i found now the real problem:

The cookie path was not set so i guess ie has here a different behavior.

I sent a PR: https://github.com/VeliovGroup/Meteor-Files/pull/333

[dr-dimitru]

@macrozone

RFC6265 does not indicate any of cookie attributes as required.

Each cookie begins with a name-value-pair, followed by __zero__ or more attribute-value pairs.

Need to confirm IE can't set cookie without path attribute

[macrozone]

ie can set cookies without path.

if you omit path, it sets the path to the current path. In my case this was "/de".
Since the image-urls did not start with that path, IE did refuse to send a cookie there.

I am not sure, why this worked in the other browsers though.

path = "/" makes it available for the whole domain. I think this is exactly what you want here.

[dr-dimitru]

   If the cookie-attribute-list contains an attribute with an
   attribute-name of "Path", set the cookie's path to attribute-
   value of the last attribute in the cookie-attribute-list with an
   attribute-name of "Path".  Otherwise, set the cookie's path to
   the default-path of the request-url.

What is default-path is not explained, or I didn't found an explanation.
So, I guess IE uses request-url here, and your PR is reasonable in this case. But I still would like to see some test results...

[derwok]

Hi @dr-dimitru, @macrozone!
I cloned in the change of @macrozone 's PR (138a0133ebcb7f828c92dec09829ce080ca45fee) - in the local packages dir - and what should I say?! ;-)

For the first time in my life I could see stupid IE11 perform some download

(Of course I first had to logout/login with IE11 before it worked.)

HOORAY! Thank you very much to both of you!

I then checked upload and download functionality with these browsers:

• Safari 10.0.2 - OK
• Chrome 55.0.2883.95 (64-bit) - OK
• FireFox 50.1.0 - OK
• IE11 11.0.9600 - Finally: OK!!!

Just to ~doublecheck~ out of curiosity: I removed the patched line of @macrozone of commit 138a0133ebcb7f828c92dec09829ce080ca45fee - so I went back to:

     cookie.set 'x_mtok', Meteor.connection._lastSessionId

With every browser I did a logout / login then download try:

• Safari 10.0.2 - FAIL - HTTP-404
• Chrome 55.0.2883.95 (64-bit) - FAIL - HTTP-404
• FireFox 50.1.0 - FAIL - HTTP-404
• IE11 11.0.9600 - Still: OK!!! - surprise!

So, yes: I would vote for @macrozone 's PR.
;-)

Thanks for taking care.

Wolfram.

[dr-dimitru]

Hi @derwok ,

Thank you for tests, with surprising results.
Looking forward do merge @macrozone 's PR, minor changes is requested.