Signal-desktop: Symlinks can prevent start or display of attachments

[protist]

• [X] I have searched open and closed issues for duplicates
  

---

Bug description

Image attachments fail, and are now replaced with IMAGE FAILED TO LOAD. This worked perfectly in my previous version of Signal, 1.14.4, but now fails after upgrading to 1.15.4. I checked an image that I remember displaying fine in the previous version (on the same system), but this image now also fails to load.

Steps to reproduce

1. Open a conversation that has an attached image

Actual result: image fail to show, and is now replaced with IMAGE FAILED TO LOAD.

Expected result: image should be displayed

Platform info

Signal version: 1.14.4-1 -> 1.15.4-1

Operating System: up-to-date Arch Linux

Linked device version: Android Signal 4.24.8

Link to debug log

INFO  2018-08-13T01:31:18.173Z Warning: denying request to path '/home/protist/.config/Signal/attachments.noindex/f3/f3cdfedd23e36f8d67d778b50bc352fa70d98ae54f9ba098049e3683c9b591a5'
ERROR 2018-08-13T01:31:18.174Z getImageDimensions error [object Event]
ERROR 2018-08-13T01:31:18.174Z captureDimensionsAndScreenshot: error processing image; skipping screenshot generation [object Event]

[scottnonnenberg-signal]

If you send a new image, what happens? What if you receive a new image?

Looking at that 'denying request to path' entry, I'm also wondering if you have any interesting symlinks in place for your home directory. Did your home directory setup perhaps change recently?

[protist]

Thank you for the quick reply.

If you send a new image, what happens?

I get the same error on the desktop GUI if I send an image from my phone.

What if you receive a new image?

I also see the same error in the GUI.

Looking at that 'denying request to path' entry, I'm also wondering if you have any interesting symlinks in place for your home directory. Did your home directory setup perhaps change recently?

Ah, yes, /home/protist/.config/Signal is a symlink to a decrypted encfs file system. However, this is not a recent change; it's been like that for a year, and previous versions of Signal have worked fine with this.

I also had a look through /home/protist/.config/Signal/attachments.noindex, and it looks like new attachments (sent and received) are still being stored here. They are added to my file system, and I can view them, so Signal Desktop is able to access these directories on some level.

[akutuzov]

I confirm I have the same problem after upgrading. In my case the Signal config directory is also a symlink, and I see Image failed to load instead of the actual image. If I remove the symlink and make it a real directory, everything works fine. I run up-to-date Linux Mint 18.3.

I also confirm that this problem didn't exist before version 1.15 or something like that.

[protist]

More information: I just checked on another computer that also runs Arch Linux, the same version of Signal, and is linked to the same account. This computer doesn't use a symlink, and the images load fine.

[scottnonnenberg-signal]

@protist @akutuzov I just released v1.15.5 which has additional logging to help us track this down. As I mentioned previously we do try to follow the symlinks in question. Now we'll be able to see, very clearly, why the security check isn't succeeding.

[protist]

Thanks @scottnonnenberg. After observing the bug in v1.15.5, I get this log now:

INFO  2018-08-16T01:25:24.403Z SQL channel job 51 (getMessagesByConversation) succeeded in 56ms
INFO  2018-08-16T01:25:24.406Z SQL channel job 52 (getMessagesByConversation) succeeded in 7ms
INFO  2018-08-16T01:25:24.648Z Warning: denying request to path '/home/protist/.config/Signal/attachments.noindex/77/77dadcee2798975de2871d9c6ddd9fb99a8553b542fe2d4d91d03f7798909ef3' (userDataPath: '/home/protist/.decrypt/Signal', installPath: '[REDACTED]/app.asar')
INFO  2018-08-16T01:25:24.648Z Warning: denying request to path '/home/protist/.config/Signal/attachments.noindex/e1/e1a2667ecdadd838837c3a38760b18d31391dff81e36abb3e4ad943225552a6a' (userDataPath: '/home/protist/.decrypt/Signal', installPath: '[REDACTED]/app.asar')
INFO  2018-08-16T01:25:24.648Z Warning: denying request to path '/home/protist/.config/Signal/attachments.noindex/c3/c324f293ca1c3d491cbc728aa664eeecdeee9070b57f55b60d158d4c9756400f' (userDataPath: '/home/protist/.decrypt/Signal', installPath: '[REDACTED]/app.asar')
INFO  2018-08-16T01:25:24.648Z Warning: denying request to path '/home/protist/.config/Signal/attachments.noindex/80/80114a784e19ef61392790c4f5eb351aa331fc7c23daed8d6983717e113e902f' (userDataPath: '/home/protist/.decrypt/Signal', installPath: '[REDACTED]/app.asar')
INFO  2018-08-16T01:25:24.648Z Conversation group([REDACTED] ?­) took 304 milliseconds to load
INFO  2018-08-16T01:25:24.672Z Message: Image failed to load; failing over to placeholder
INFO  2018-08-16T01:25:24.672Z Message: Image failed to load; failing over to placeholder
INFO  2018-08-16T01:25:24.673Z Message: Image failed to load; failing over to placeholder
INFO  2018-08-16T01:25:24.673Z Message: Image failed to load; failing over to placeholder
INFO  2018-08-16T01:25:24.700Z SQL channel job 53 (getUnreadByConversation) succeeded in 6ms

[scottnonnenberg-signal]

@protist Well that's surprising. Is it a hard link you're using? We explicitly use fs.realpath() to try to get the underlying location of the config directory: https://github.com/signalapp/Signal-Desktop/blob/15751f35213afa98659e52c97a0768c81c8a2683/main.js#L584-L585

It seems that Electron/Chromium uses a deeper form of 'real path following' than we're able to with our fs.realpath() call, in any case.

[protist]

No, it's definitely a symlink.

$ ls -ld /home/protist/.config/Signal
lrwxrwxrwx 1 protist protist 25 Feb 19 11:17 /home/protist/.config/Signal -> /home/protist/.decrypt/Signal

Also, it seems that Signal is able to deposit the attachments in the symlinked directory, as per my earlier comment, so I guess there's some inconsistency in how the code is operating. (And as mentioned, it worked fine in a previous version, too.)

[akutuzov]

Same here. As soon as the Signal config directory becomes a symlink, images stop loading.
And I again confirm @protist observation: it did work correctly before 1.15.

{"name":"log","hostname":"wonderland","pid":26167,"level":30,"msg":"SQL channel job 78 (getMessagesByConversation) succeeded in 16ms","time":"2018-08-16T18:07:21.309Z","v":0}
{"name":"log","hostname":"wonderland","pid":26167,"level":30,"msg":"Warning: denying request to path '/home/user/.config/Signal/attachments.noindex/18/188d9784c7e972c0a650c037fbf50aafc18be05bbbb3d5bf7e4734dbe1704285' (userDataPath: '/home/user/tmp/Signal', installPath: '/opt/Signal/resources/app.asar')","time":"2018-08-16T18:07:22.164Z","v":0}
{"name":"log","hostname":"wonderland","pid":26167,"level":30,"msg":"Message: Image failed to load; failing over to placeholder","time":"2018-08-16T18:07:22.198Z","v":0}

[avinson]

fyi, i also see this problem on MacOS and have a symlink for my ~/Library directory. As with the other users, this used to work fine until recent versions.

INFO  2018-08-23T05:23:04.476Z Warning: denying request to path '/Volumes/850_Pro/Users/avinson/Library/Application Support/Signal/attachments.noindex/8a/8a66c3e3d3dfb3d84591fdf643742c13f079cf89fd541765446404b7fb920df2' (userDataPath: '/Users/avinson/Library/Application Support/Signal', installPath: '[REDACTED]/app.asar')

[azrinyi]

I can confirm this problem on Windows 10 too. I symlinked my Signal folder (which resides in AppData) to another location. (Since I'm not on the machine at the moment, I can't provide any logs)

[protist]

FWIW I just upgraded from Signal 1.15.5 to 1.16.0, and this issue is still present. Would it be useful for me to git bisect to identify the regression? (I'm a little worried about what downgrading would do to my config, but would it be safe if I backup and restore those directories?)

[protist]

Another update to Signal 1.16.1. This issue is still present, but manifests itself in a slightly different way. Instead of IMAGE FAILED TO LOAD, I now see an icon for the image file (e.g. a generic icon with JPEG on it). I can also see the size of the file, so Signal can clearly access it in some way. I can attempt to download the file, but when the file selector appears, I see the following in the terminal:

{"name":"log","hostname":"my_hostname","pid":5592,"level":30,"msg":"Warning: denying request to path '/home/protist/.config/Signal/attachments.noindex/a5/<alphanumeric_string>' (userDataPath: '/home/protist/.decrypt/Signal', installPath: '/usr/lib/signal/resources/app.asar')","time":"2018-09-18T02:52:20.553Z","v":0}

I then select the save location, and Signal fails silently. There are no further errors reported, and the file is unsaved. Again, my offer to git bisect is there, but please tell me if it's useful, because it will likely take a few hours, and I'm also worried about downgrading and the effects on my config, as above.

[scottnonnenberg-signal]

@protist Before we do the git bisect, let's take another look at this more detailed log entry we've had lately. From your message:

• denying request to path /home/protist/.config/Signal/attachments.noindex/a5/<alphanumeric_string>
• userDataPath: /home/protist/.decrypt/Signal
• installPath: /usr/lib/signal/resources/app.asar

Am I right that the first path would turn into /home/protist/.decrypt/... if we followed the symlink?

If that's the case, then a call to follow the symlink here would solve the problem: https://github.com/signalapp/Signal-Desktop/blob/366401f77aac57ce036eb379f376acbdce8746f7/app/protocol_filter.js#L26

[protist]

Am I right that the first path would turn into /home/protist/.decrypt/... if we followed the symlink?

That is correct.

$ ls -l ~/.config/Signal
lrwxrwxrwx 1 protist protist 25 Feb 19  2018 /home/protist/.config/Signal -> /home/protist/.decrypt/Signal

If that's the case, then a call to follow the symlink here would solve the problem:

Thank you, but I'm not exactly sure what I should be testing here. Could you please explain your suggestion explicitly? Should I test modifying this line?

[scottnonnenberg-signal]

Basically this, right after the line I pointed out above:

var fs = require('fs');
var realPath = fs.realpathSync(target);
// use realPath where target is used today

It should work with that change. It would be great if someone could verify that it works for them.

[bowmasters]

I can verify that this works for me, applying the change to the master branch

[protist]

I'm not 100% sure if I'm doing it right, but I can't seem to get Signal to launch after the patch. First I tried to patch the latest stable release (1.16.1), but Signal just opened to a blank window with the following error:

Uncaught error or unhandled promise rejection: Error: ENOENT, js/views/standalone_registration_view.js not found in /usr/lib/signal/resources/app.asar
    at notFoundError (ELECTRON_ASAR.js:108:19)
    at Object.fs.realpathSync (ELECTRON_ASAR.js:344:9)
    at Function.<anonymous> (/usr/lib/signal/resources/app.asar/app/protocol_filter.js:28:23)

Just in case, I tested building the stable without the patch, which worked. I then realised that the master branch was ahead of the development branch (and hence ahead of the stable release), so I attempted patching that too, as per @bowmasters. However, I still got the same blank window, with the same error. Just to be explicit, this is the correct change, right?

--- orig/src/signal-git-repo/app/protocol_filter.js 2018-09-20 19:36:31.767516123 +1000
+++ patched/signal-git-repo/app/protocol_filter.js  2018-09-20 19:27:45.773166436 +1000
@@ -24,6 +24,9 @@
   return (request, callback) => {
     // normalize() is primarily useful here for switching / to \ on windows
     const target = path.normalize(_urlToPath(request.url, { isWindows }));
+    var fs = require('fs');
+    var realPath = fs.realpathSync(target);
+    // use realPath where target is used today

     if (!path.isAbsolute(target)) {
       console.log(`Warning: denying request to non-absolute path '${target}'`);

[scottnonnenberg-signal]

@protist It looks like fs.realpathSync() throws an error if the target path does not exist. So we'll need an fs.existsSync check first to ensure that we don't call it for nonexistent paths. I'll be sure to include that protection in my change.

[bowmasters]

@protist: I also changed all the subsequent instances of "target" with "realPath" like so

    const target = path.normalize(_urlToPath(request.url, { isWindows }));
    var fs = require('fs');
    var realPath = fs.realpathSync(target);
    // use realPath where target is used today

    if (!path.isAbsolute(realPath)) {
      console.log(`Warning: denying request to non-absolute path '${realPath}'`);
      return callback();
    }

    if (!realPath.startsWith(userDataPath) && !realPath.startsWith(installPath)) {
      console.log(
        `Warning: denying request to path '${realPath}' (userDataPath: '${userDataPath}', installPath: '${installPath}')`
      );
      return callback();
    }

    return callback({
      path: realPath,
    });

[protist]

@scottnonnenberg I'm not exactly sure what that all means. If you could possibly post explicit changes required, I'm happy to test them.

@bowmasters Thanks for that, but it still failed with the same error message when patching the stable or the master.

--- orig/app/protocol_filter.js     2018-09-20 19:36:31.767516123 +1000
+++ patched/app/protocol_filter.js  2018-09-21 10:28:48.431926065 +1000
@@ -24,21 +24,24 @@
   return (request, callback) => {
     // normalize() is primarily useful here for switching / to \ on windows
     const target = path.normalize(_urlToPath(request.url, { isWindows }));
+    var fs = require('fs');
+    var realPath = fs.realpathSync(target);
+    // use realPath where target is used today

-    if (!path.isAbsolute(target)) {
-      console.log(`Warning: denying request to non-absolute path '${target}'`);
+    if (!path.isAbsolute(realPath)) {
+      console.log(`Warning: denying request to non-absolute path '${realPath}'`);
       return callback();
     }

-    if (!target.startsWith(userDataPath) && !target.startsWith(installPath)) {
+    if (!realPath.startsWith(userDataPath) && !realPath.startsWith(installPath)) {
       console.log(
-        `Warning: denying request to path '${target}' (userDataPath: '${userDataPath}', installPath: '${installPath}')`
+        `Warning: denying request to path '${realPath}' (userDataPath: '${userDataPath}', installPath: '${installPath}')`
       );
       return callback();
     }

     return callback({
-      path: target,
+      path: realPath,
     });
   };
 }

[scottnonnenberg-signal]

Hey all - v1.16.2 and v1.16.2-beta.2 were both released today, and should fix this issue. Please let me know!

[protist]

Hey all - v1.16.2 and v1.16.2-beta.2 were both released today, and should fix this issue. Please let me know!

Yes! v1.16.2 works perfectly for me! Thank you!

[breversa]

I have the same problem as @protist on 2018-09-18, when running Signal Desktop 1.17.1 on Windows 10 (no problem on Linux though) : I see a generic image icon, along with its size, but not the actual content.

[scottnonnenberg-signal]

@breversa Does that happen for all images? What is the file type? Please provide a log and we can start looking at what might be causing that.

[breversa]

Here's the log at a time it happened :

Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Signal/1.17.1 Chrome/61.0.3163.100 Electron/2.0.8 Safari/537.36 node/8.9.3 env/production
INFO  2018-10-29T16:05:07.113Z Worker job 531 (arrayBufferToStringBase64) succeeded in 0ms
INFO  2018-10-29T16:05:07.176Z SQL channel job 5001 (saveUnprocessed) succeeded in 63ms
INFO  2018-10-29T16:05:07.176Z queueing envelope +[REDACTED]630.1 1540829103119
INFO  2018-10-29T16:05:07.176Z message from +[REDACTED]630.1 1540829103119
INFO  2018-10-29T16:05:07.224Z SQL channel job 5002 (getUnprocessedById) succeeded in 16ms
INFO  2018-10-29T16:05:07.224Z Worker job 532 (arrayBufferToStringBase64) succeeded in 0ms
INFO  2018-10-29T16:05:07.286Z SQL channel job 5003 (saveUnprocessed) succeeded in 62ms
INFO  2018-10-29T16:05:07.286Z data message from +[REDACTED]630.1 1540829103119
INFO  2018-10-29T16:05:07.286Z GET https://textsecure-service.whispersystems.org/v1/attachments/1694399359872857248
INFO  2018-10-29T16:05:07.832Z GET https://textsecure-service.whispersystems.org/v1/attachments/1694399359872857248 200 Success
INFO  2018-10-29T16:05:07.832Z GET https://whispersystems-textsecure-attachments.s3-accelerate.amazonaws.com/1694399359872857248?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20181029T160507Z&X-Amz-SignedHeaders=content-type%3Bhost&X-Amz-Expires=3600&X-Amz-Credential=AKIAJHWS3AOTJTASHBDA%2F20181029%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Signature=1087ae1257507e08ebe6277cbcffab1535f01ba9f118dd8f6359127caea231fb
INFO  2018-10-29T16:05:08.878Z GET https://whispersystems-textsecure-attachments.s3-accelerate.amazonaws.com/1694399359872857248?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Date=20181029T160507Z&X-Amz-SignedHeaders=content-type%3Bhost&X-Amz-Expires=3600&X-Amz-Credential=AKIAJHWS3AOTJTASHBDA%2F20181029%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Signature=1087ae1257507e08ebe6277cbcffab1535f01ba9f118dd8f6359127caea231fb 200 Success
INFO  2018-10-29T16:05:08.893Z SQL channel job 5004 (getMessageBySender) succeeded in 0ms
INFO  2018-10-29T16:05:09.018Z Warning: denying request to non-absolute path 'xxx.net\DFS\servicefile\Redirect\[user]\AppData\Roaming\Signal\attachments.noindex\61\61e8dcfe001e7218818fb7ef4252e8b48f4f093b50282bdd67819b86df5a8877'
ERROR 2018-10-29T16:05:09.034Z getImageDimensions error [object Event]
ERROR 2018-10-29T16:05:09.034Z captureDimensionsAndScreenshot: error processing image; skipping screenshot generation [object Event]
INFO  2018-10-29T16:05:09.146Z SQL channel job 5005 (saveMessage) succeeded in 112ms
INFO  2018-10-29T16:05:09.146Z SQL channel job 5006 (getNextExpiringMessage) succeeded in 0ms
INFO  2018-10-29T16:05:09.146Z SQL channel job 5007 (getConversationById) succeeded in 0ms
INFO  2018-10-29T16:05:09.224Z SQL channel job 5008 (updateConversation) succeeded in 78ms
INFO  2018-10-29T16:05:09.224Z Warning: denying request to non-absolute path 'xxx.net\DFS\servicefile\Redirect\[user]\AppData\Roaming\Signal\attachments.noindex\61\61e8dcfe001e7218818fb7ef4252e8b48f4f093b50282bdd67819b86df5a8877'
INFO  2018-10-29T16:05:09.224Z SQL channel job 5009 (getMessagesByConversation) succeeded in 0ms
INFO  2018-10-29T16:05:09.239Z SQL channel job 5010 (getMessageById) succeeded in 16ms
INFO  2018-10-29T16:05:09.239Z Add notification {"conversationId":"+[REDACTED]630","isExpiringMessage":false,"messageSentAt":1540829103119}
INFO  2018-10-29T16:05:09.239Z Update notifications: {"shouldClearNotifications":false,"shouldPlayNotificationSound":false,"shouldShowNotifications":true,"type":"ok","isNotificationGroupingSupported":false}
INFO  2018-10-29T16:05:09.239Z draw attention
INFO  2018-10-29T16:05:09.260Z Message: Image failed to load; failing over to placeholder
INFO  2018-10-29T16:05:09.293Z Approving request for permission 'notifications'
INFO  2018-10-29T16:05:09.330Z SQL channel job 5011 (removeUnprocessed) succeeded in 91ms
INFO  2018-10-29T16:05:09.430Z SQL channel job 5012 (getMessagesByConversation) succeeded in 4ms

File type was GIF, size 833.39KiB

It happened previsouly with JPG and PNG too.

[scottnonnenberg-signal]

What is going on here?

INFO 2018-10-29T16:05:09.018Z Warning: denying request to non-absolute path 'xxx.netDFSservicefileRedirect[user]AppDataRoamingSignalattachments.noindex6161e8dcfe001e7218818fb7ef4252e8b48f4f093b50282bdd67819b86df5a8877'

What did the xxx.net replace? A domain for your company or internet provier?

[breversa]

Yes, it's my company domain.

[scottnonnenberg-signal]

Yep, something really strange is going on there. It looks as if your local profile is hosted on some other machine. For security reasons, we check for anything that isn't an absolute path to the local filesystem, and prevent access to it.

[breversa]

You're correct : company users' profiles are stored on a remote, backuped server.
Is there anything that can be done (on Signal Desktop's side) to support this setup ?

[scottnonnenberg-signal]

@breversa No, we likely won't open the door to remote files like that. But we might start producing better error messages for that scenario.

[joao456]

So, I’m apparently having a similar issue. I have my .config/Signal folder symlinked; the app apparently cannot access parts of the directory and this makes some attachments appear as generic image icons.

Besides that, toh, not every image attachment behaves this way and some are displayed ‘correctly’. The issue also happens with some audio attachments that appear as non-playable audio messages in the in-app audio player, but not all of then.

ls -ld ~/.config/Signal/
drwx------ 11 user user Feb 27 02:44 ~/.config/Signal/ → ~/unlocked/Signal

INFO 2019-02-26T16:53:06.573Z Warning: denying request to path '~/unlocked/Signal/attachments.noindex/be/be2df34b1debe6c1bc4cb17a89452c89aacc5309030de46641f1ef57a3d340a1' (userDataPath: '~/.config/Signal', installPath: '[REDACTED]/app.asar')

OS: Debian 9
signal-desktop version: 1.22.0

[joao456]

So, I’m apparently having a similar issue. I have my .config/Signal folder symlinked; the app apparently cannot access parts of the directory and this makes some attachments appear as generic image icons.

Besides that, toh, not every image attachment behaves this way and some are displayed ‘correctly’. The issue also happens with some audio attachments that appear as non-playable audio messages in the in-app audio player, but not all of then.

ls -ld ~/.config/Signal/
drwx------ 11 user user Feb 27 02:44 ~/.config/Signal/ → ~/unlocked/Signal

INFO 2019-02-26T16:53:06.573Z Warning: denying request to path '~/unlocked/Signal/attachments.noindex/be/be2df34b1debe6c1bc4cb17a89452c89aacc5309030de46641f1ef57a3d340a1' (userDataPath: '~/.config/Signal', installPath: '[REDACTED]/app.asar')

OS: Debian 9
signal-desktop version: 1.22.0

If you believe this is a separate/another issue, I can open a new one...

[joao456]

Some more info that I forgot:
I cannot download the placeholder image files/messages to disk (the download symbol/option don't even shows up);

In the case of the "audio files in-app player placeholder message" the download symbol/option appears, I can click it, it prompts me to choose the location to save the file (and the file name, etc.), but when I click to save it fails, silently;

Saving the "correctly" appearing attachments to disk is no problem.

[scottnonnenberg-signal]

@joao456 Have you noticed any pattern to which attachments appear correctly and which don't? That would help us track down what's going on here.

The thing that's disappointing is that in main.js we're attempting to follow any links for the config and install directories: https://github.com/signalapp/Signal-Desktop/blob/development/main.js#L617-L618

Is there anything strange about the way you've linked those directories?

Also, yes, it's probably a good idea to open a new bug for this.

[freed00m]

I've created a symbolic link for attatchments to /tmp/signal and this makes me very dissatisfied.

Because floading ~/.config/ with many useless megabytes makes complete sense :/

[scottnonnenberg-signal]

@freed00m Could you explain your perspective a little bit more? What would you like for Signal Desktop to do regarding the filesystem, and why?

[freed00m]

@scottnonnenberg-signal well when I use programs I often consider privacy a bit more, so I make symlinks to different partitions that are either encrypted or temporary nature.
In case of Signal I would like to have attatchments and logs in tmpfs so they would be gone the next time I boot.
Also I prevent unecessary writes on my ssd, so my drive lives a little bit longer.

No programs have troubles writting somewhere else but Signal looks where the link is targeting and resolves the target with absolute path and decides it is not permitted :(