Gallery-dl: Instagram and .netrc ([instagram][error] netrc: [Errno 13] Permission denied)

[blotsome]

I'm getting the following error when netrc is set to true:
[instagram][error] netrc: [Errno 13] Permission denied: '/home/[user]/.netrc'
I have tried placing the "netrc": true in a half a dozen different places in the .conf and I don't believe that is a valid troubleshooting step. One other thing to note, extractor.instragram.highlights is set to true. I've also tried changing the permission of the .netrc file from 600, to 400, to 777, to 666, and changing the user from [user] to root and back. Here is the output of verbose:

[gallery-dl][debug] Version 1.13.6
[gallery-dl][debug] Python 3.5.2 - Linux-4.15.0-99-generic-x86_64-with-Ubuntu-18.04-bionic
[gallery-dl][debug] requests 2.23.0 - urllib3 1.25.9
[gallery-dl][debug] Starting DownloadJob for 'https://www.instagram.com/test/'
[instagram][debug] Using InstagramUserExtractor for 'https://www.instagram.com/test/'
[instagram][error] netrc: [Errno 13] Permission denied: '/home/[user]/.netrc'

What is the default UID of the gallery-dl process? Does that have to match the UID of the .netrc file?

[blotsome]

I tried with the username and password flags and got this:

[gallery-dl][debug] Version 1.13.6
[gallery-dl][debug] Python 3.6.9 - Linux-4.15.0-99-generic-x86_64-with-Ubuntu-18.04-bionic
[gallery-dl][debug] requests 2.23.0 - urllib3 1.25.9
[gallery-dl][debug] Starting DownloadJob for 'https://www.instagram.com/test/'
[instagram][debug] Using InstagramUserExtractor for 'https://www.instagram.com/test/'
[instagram][info] Logging in as [username]
[urllib3.connectionpool][debug] Starting new HTTPS connection (1): www.instagram.com:443
[urllib3.connectionpool][debug] https://www.instagram.com:443 "GET /accounts/login/ HTTP/1.1" 200 11081
[urllib3.connectionpool][debug] https://www.instagram.com:443 "GET /web/__mid/ HTTP/1.1" 200 28
[urllib3.connectionpool][debug] https://www.instagram.com:443 "POST /accounts/login/ajax/ HTTP/1.1" 400 11
[instagram][error] HttpError: '400 Bad Request' for 'https://www.instagram.com/accounts/login/ajax/'

[lihuelworks]

Same issue here, may be in Instagram's side: https://github.com/mikf/gallery-dl/issues/756
Edit: mikf solved it in that same issue.

[mikf]

Your .netrc file needs to have permissions 600 and be owned by the same user that starts the Python process running gallery-dl (more details in the Python docs)

[mike ~ ]$ ls -l .netrc
-rw------- 1 mike mike 207 16. Nov 2018  .netrc
[mike ~ ]$ gallery-dl --netrc instagram.com/instagram
[instagram][warning] netrc: No authentication info
# (it was able to open and parse my netrc file, but there is no username & password for instagram in there)

And yes, login with username & password is currently broken for Instagram and you have to use your browser cookies: https://github.com/mikf/gallery-dl/issues/756#issuecomment-629679148

[blotsome]

Ok, I guess I don't know what user snap instance is. How would I find out? I installed just now via pip and yeah, I get the same thing as you "no authentication info" which I assume is related to #756 The gallery-dl alias must be tied to snap which is not root nor my username apparently.

[mikf]

I also don't know what happens under the hood when running a snap, but you can (ab)use --filter to basically run arbitrary code inside the gallery-dl snap and print the info you need. For example

# print the current username
$ gallery-dl --filter 'print(__import__("getpass").getuser())' http://localhost/foo.jpg

# print the home directory of the user running gallery-dl
$ gallery-dl --filter 'print(__import__("os").path.expanduser("~"))' http://localhost/foo.jpg

[blotsome]

Hmm.. says my user is running it and same home directory with the .netrc, but clearly, when I do python3 /home/xrintrahx/.local/bin/gallery-dl instead of gallery-dl, the former appears to work ([instagram][warning] netrc: No authentication info), while the latter does not ([instagram][error] netrc: [Errno 13] Permission denied: '/home/username/.netrc'). Seems fishy, but possibly unrelated to the core code, but instead how I installed it (or how snap behaves, which may or may not be of your concern). I'd like to figure it out, but I'm also willing to close and move on. Thanks for your time and dedication.

[mikf]

Yeah, that's probably the Snap environment restricting access to most files in your home directory. It does have "special" permissions to read the config files on their designated paths, but everything else seems to be off limits.

In my opinion, don't use the Snap package if you can help it ...

[Lin-Buo-Ren]

FYI due to the snap runtime's security confinement, hidden files _right under_ the user's home directory isn't accessible by the application by default.

It is possible to request an exception for $HOME/.netrc like the same for $HOME/.gallery-dl.conf though... @mikf what do you think?

Update: A request seems to be feasible: Question: Is personal-files interface allowed for $HOME/.netrc? - store-requests - snapcraft.io

[Lin-Buo-Ren]

@mikf Re-ping, would you let me request the snap's netrc access on behalf of you?

[mikf]

Sure, that would be much appreciated.

I'm guessing simply adding a line for .netrc here won't work until they've granted permission specifically for gallery-dl?
https://github.com/mikf/gallery-dl/blob/ba083b30b26eeea43f1ffc71459ee159b84a2447/snap/snapcraft.yaml#L46-L48

And sorry for not responding the first time ...

[Lin-Buo-Ren]

I've filed a request here: Interface auto-connect request for the gallery-dl snap (personal-files; netrc) - store-requests - snapcraft.io

I'm guessing simply adding a line for .netrc here won't work until they've granted permission specifically for gallery-dl?

Yep. Since the purpose is different I suppose a separate plug declaration would be necessary instead of expanding config-gallery-dl:

  dot-netrc:
    interface: personal-files
    read:
      - $HOME/.netrc

[Lin-Buo-Ren]

I can verify that since the 1.17.0 snap release the user's .netrc file can be accessed by gallery-dl after connecting to the new dot-netrc personal-files interface:

sudo snap connect gallery-dl:dot-netrc core:personal-files

Cheers!