Yay: Searching in AUR via SOCKS proxy does not work

[muffl0n]

Affected Version

yay v8.1173.0 - libalpm v11.0.1

Issue

Updating the "package database" via SOCKS proxy works, searching for package (in AUR) doesn't.

Steps to reproduce

SOCKS proxy

export http_proxy=socks5h://localhost:9191
export https_proxy=socks5h://localhost:9191

yay -Syy
:: Synchronizing package databases...
 core                           135.7 KiB  4.91M/s 00:00 [#############################] 100%
 extra                         1642.9 KiB  8.02M/s 00:00 [#############################] 100%
 community                        4.6 MiB  8.16M/s 00:00 [#############################] 100%
 archlinuxfr                     11.4 KiB  1142K/s 00:00 [#############################] 100%```

yay -Ss mattermost
community/matterbridge 1.11.1-1 (4.6 MiB 25.3 MiB)
    Multi-protocols (IRC/XMPP/Mattermost/Slack/Matrix/etc) bridge
Error during AUR search: Get https://aur.archlinux.org/rpc.php?arg=mattermost&type=search&v=5: proxyconnect tcp: dial tcp: lookup socks5h: Temporary failure in name resolution
Showing Repo packages only



md5-7397a472d05bf47af64b0357ba2d0e8e



yaourt -Ss mattermost                                                     
community/matterbridge 1.11.3-1
    Multi-protocols (IRC/XMPP/Mattermost/Slack/Matrix/etc) bridge
[...]
aur/python-mattermostdriver 2.1.1-2 (Out of Date) (1) (0.00)
    A Python Mattermost Driver



md5-18d89fc453354c5861b0f4c547679730



export http_proxy=http://foo:bar@myproxy:8080
export https_proxy=http://foo:bar@myproxy:8080



md5-68c854179d063c9c970cfae51ed16b86



yay -Syy
:: Synchronizing package databases...
 core                           135.7 KiB  18.9M/s 00:00 [#############################] 100%
 extra                         1642.9 KiB   134K/s 00:00 [#############################] 100%
 community                        4.6 MiB  7.54M/s 00:00 [#############################] 100%
 archlinuxfr                     11.4 KiB  1359K/s 00:00 [#############################] 100%```



md5-68c854179d063c9c970cfae51ed16b86



yay -Ss mattermost
aur/purple-mattermost-spectrum2 v1.2.r1.daf5d6b-1 (+0 0.00%)
    A purple-based (Pidgin) plugin for Mattermost. (spectrum2 work)
[...]
aur/mattermost-desktop 4.1.2-2 (+40 2.91%) 
    Mattermost Desktop application for Linux (Beta)
community/matterbridge 1.11.3-1 (5.4 MiB 27.4 MiB) 
    Multi-protocols (IRC/XMPP/Mattermost/Slack/Matrix/etc) bridge



md5-79fbd2656edb0182e0b9969ed0a84163



{
        "aururl": "https://aur.archlinux.org",
        "buildDir": "/home/svs/.cache/yay",
        "editor": "",
        "editorflags": "",
        "makepkgbin": "makepkg",
        "makepkgconf": "",
        "pacmanbin": "pacman",
        "pacmanconf": "/etc/pacman.conf",
        "tarbin": "bsdtar",
        "redownload": "no",
        "rebuild": "no",
        "answerclean": "",
        "answerdiff": "",
        "answeredit": "",
        "answerupgrade": "",
        "gitbin": "git",
        "gpgbin": "gpg",
        "gpgflags": "",
        "mflags": "",
        "sortby": "votes",
        "gitflags": "",
        "removemake": "ask",
        "requestsplitn": 150,
        "sortmode": 0,
        "completionrefreshtime": 7,
        "sudoloop": false,
        "timeupdate": false,
        "devel": false,
        "cleanAfter": false,
        "gitclone": true,
        "provides": true,
        "pgpfetch": true,
        "upgrademenu": true,
        "cleanmenu": true,
        "diffmenu": true,
        "editmenu": false,
        "combinedupgrade": false,
        "useask": false
}

[Jguer]

This seems like an upstream limitation of net/http. I'll still investigate it, have you tried yay compiled with gcc-go?

As a workaround you could always try https://github.com/Jguer/yay/issues/429 proxychains.

[muffl0n]

I've set up a docker container and tried gcc-go:

$ go version
go version go1.10.3 gccgo (GCC) 8.2.1 20180831 linux/amd64
$ gccgo --version
gccgo (GCC) 8.2.1 20180831
Copyright (C) 2018 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

$ export | grep proxy
declare -x http_proxy="socks5h://localhost:9191"
declare -x https_proxy="socks5h://localhost:9191"
$ yay -Ss mattermost
community/matterbridge 1.11.3-1 (5.4 MiB 27.4 MiB) 
    Multi-protocols (IRC/XMPP/Mattermost/Slack/Matrix/etc) bridge
Error during AUR search: Get https://aur.archlinux.org/rpc.php?arg=mattermost&type=search&v=5: proxyconnect tcp: dial tcp: lookup socks5h: Temporary failure in name resolution
Showing Repo packages only

[Jguer]

Leaving here the only reference I can find in the language https://github.com/golang/go/issues/13454

[Jguer]

I linked that bug report but forgot to ask you, have you tried another protocol like socks5://?

[muffl0n]

That's weird, now it fails the other way around. :D

export http_proxy=socks5://localhost:9191
export https_proxy=socks5://localhost:9191

-> % yay -Syy          
:: Synchronizing package databases...
error: failed retrieving file 'core.db' from ftp.gwdg.de : Could not resolve host: ftp.gwdg.de
error: failed to update core (download library error)
error: failed retrieving file 'extra.db' from ftp.gwdg.de : Could not resolve host: ftp.gwdg.de
error: failed to update extra (download library error)
error: failed retrieving file 'community.db' from ftp.gwdg.de : Could not resolve host: ftp.gwdg.de
error: failed to update community (download library error)
error: failed retrieving file 'archlinuxfr.db' from repo.archlinux.fr : Could not resolve host: repo.archlinux.fr
error: failed to update archlinuxfr (download library error)
error: failed to synchronize all databases



md5-68c854179d063c9c970cfae51ed16b86



$ yay -Ss mattermost
aur/purple-mattermost-spectrum2 v1.2.r1.daf5d6b-1 (+0 0.00%) 
    A purple-based (Pidgin) plugin for Mattermost. (spectrum2 work)
[...]
aur/mattermost-desktop 4.1.2-2 (+40 2.69%) 
    Mattermost Desktop application for Linux (Beta)
community/matterbridge 1.11.3-1 (5.4 MiB 27.4 MiB) 
    Multi-protocols (IRC/XMPP/Mattermost/Slack/Matrix/etc) bridge



md5-e3b1d4e980c30800eab95c13a4189c48



% yaourt -Syy                                                     
:: Synchronizing package databases...
error: failed retrieving file 'core.db' from ftp.gwdg.de : Could not resolve host: ftp.gwdg.de
error: failed to update core (download library error)
error: failed retrieving file 'extra.db' from ftp.gwdg.de : Could not resolve host: ftp.gwdg.de
error: failed to update extra (download library error)
error: failed retrieving file 'community.db' from ftp.gwdg.de : Could not resolve host: ftp.gwdg.de
error: failed to update community (download library error)
error: failed retrieving file 'archlinuxfr.db' from repo.archlinux.fr : Could not resolve host: repo.archlinux.fr
error: failed to update archlinuxfr (download library error)
error: failed to synchronize all databases



md5-68c854179d063c9c970cfae51ed16b86



$ yaourt -Ss mattermost  
community/matterbridge 1.11.3-1
    Multi-protocols (IRC/XMPP/Mattermost/Slack/Matrix/etc) bridge
curl error: Couldn't resolve host name



md5-c8915b9106fdec4189d7e850dd626cb6



-> % curl -I repo.archlinux.fr
HTTP/1.1 200 OK
Date: Mon, 12 Nov 2018 07:36:47 GMT
Server: nginx/1.14.0
Content-Type: text/html
Proxy-Connection: Keep-Alive



md5-7e222dcf7b63985d7b713e51f392ff82



export http_proxy=socks5://localhost:9191
export https_proxy=socks5://localhost:9191

[Jguer]

The proxy string may be prefixed with [scheme]:// to specify which kind of proxy is used. Use socks4://, socks4a://, socks5:// or socks5h:// (the last one to enable socks5 and asking the proxy to do the resolving, also known as CURLPROXY_SOCKS5_HOSTNAME type) to request the specific SOCKS version to be used. Otherwise SOCKS4 is used as default. 

Ok, so here is where I figure the problem is, curl supports all of these schemes, including socks5h which does the hostname resolution on the proxy side. Just from what I'm seeing I deduce:

• go resolves hostname even if it is just socks5, pacman does not (this is why your problem is reverted)
• go doesn't recognize socks5h and so fails proxying and pacman does recognize. (original problem)

If you're only running a localhost proxy server I would use the loopback address instead

export http_proxy=socks5://127.0.0.1:9191
export https_proxy=socks5://127.0.0.1:9191

If this still fails, We should take this opportunity to consider other vocations such as farming, mountain hiking or mule train wagoner so we can enjoy a simpler life where environment variables doesn't cause headaches every day. Either that or use proxychains-ng configured with only one proxy and yay compiled with gcc-go which should work because proxychains respects all protocols.

EDIT: For socks5h to work apparently you do need a functional DNS server on the proxy server

[muffl0n]

I don't see any sense in changing "localhost" to "127.0.0.1" since "localhost" resolves to "127.0.0.1". It doesn't change anything in the behaviour I see.

Since I can reproduce this behaviour with this (my first :yum:) small go-Script I will just raise an issue upstream.

package main

import "net/http"
import "fmt"


func main() {
    resp, err := http.Get("http://example.com/")
    fmt.Println(resp, err)
}

<nil> Get http://example.com/: proxyconnect tcp: dial tcp: lookup socks5h: Temporary failure in name resolution

But regarding your ideas for other activities apart from IT: count me in! Though I prefer mountain biking. :heart_eyes:

Thank you very much for your time!

[muffl0n]

Nevermind, the upstream issue you linked is exactly the issue I was going to create.

[Jguer]

I'm closing this since there's nothing I can do from here, this should be working out of the box and it is not. I will keep tabs on the upstream issue to see if it gets fixed.