Elixir: mix can't resolve any URLs

[ghost]

I am using Elixir 1.4.2 and Erlang 19.3 and when trying to install hex I get:

mix local.hex --force
** (Mix) httpc request failed with: {:failed_connect, [{:to_address, {'repo.hex.pm', 443}}, {:inet, [:inet], :nxdomain}]}
Could not install Hex because Mix could not download metadata at https://repo.hex.pm/installs/hex-1.x.csv.

Or while trying to do deps.get

mix deps.get
Failed to fetch record for 'gettext' from registry (using cache)
{:failed_connect, [{:to_address, {'repo.hex.pm', 443}}, {:inet, [:inet], :nxdomain}]}
Failed to fetch record for 'phoenix_ecto' from registry (using cache)
{:failed_connect, [{:to_address, {'repo.hex.pm', 443}}, {:inet, [:inet], :nxdomain}]}
{:failed_connect, [{:to_address, {'repo.hex.pm', 443}}, {:inet, [:inet], :nxdomain}]}
{:failed_connect, [{:to_address, {'repo.hex.pm', 443}}, {:inet, [:inet], :nxdomain}]}
Failed to fetch record for 'phoenix_html' from registry (using cache)
Failed to fetch record for 'phoenix' from registry (using cache)
Failed to fetch record for 'cowboy' from registry (using cache)
** (Mix) No package with name phoenix (from: mix.exs) in registry
{:failed_connect, [{:to_address, {'repo.hex.pm', 443}}, {:inet, [:inet], :nxdomain}]}

I found issues #3956 and #5731 , but neither of those helped. When opening the URLs in my browser they do work.

OpenSSL is up to date and I tried mix local.hex HEX_UNSAFE_HTTPS=1 which produced the same results. I am totally new to Elixir so I don't know what else to try.

Through a vagrant box on the same machine everything works like it's supposed to.

[isaksky]

I get the same problem. Mostly the first, but also saw the second today.

[schrockwell]

@mgrandl What happens if you run iex and do :inet.getaddr('repo.hex.pm', :inet)?

[ghost]

@schrockwell

Erlang/OTP 19 [erts-8.3] [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false]

Interactive Elixir (1.4.2) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)> :inet.getaddr('repo.hex.pm', :inet)
{:error, :nxdomain}

I know :nxdomain means that DNS lookup failed, but I can't explain why. The issues persists across different networks and DNS servers.

It's also not specific to repo.hex.pm. If I am :inet.getaddr('google.com', :inet) it produces the same.

[OvermindDL1]

@mgrandl Are you able to from your normal system terminal to just do nslookup repo.hex.pm or nslookup google.com or so and if so what are the responses?

[ghost]

@OvermindDL1

nslookup  repo.hex.pm
Server:     8.8.8.8
Address:    8.8.8.8#53

Non-authoritative answer:
repo.hex.pm canonical name = n.global-ssl.fastly.net.
Name:   n.global-ssl.fastly.net
Address: 151.101.1.5
Name:   n.global-ssl.fastly.net
Address: 151.101.65.5
Name:   n.global-ssl.fastly.net
Address: 151.101.129.5
Name:   n.global-ssl.fastly.net
Address: 151.101.193.5

[OvermindDL1]

n.global-ssl.fastly.net

@mgrandl And there is your problem. You are probably on Level3, and Fastly and Level3 have issues talking (which is why I really hate that services like hex use Fastly).

Can you do a mtr/traceroute/tracert to repo.hex.pm? See if 'level3' appears anywhere in your early names to confirm?

[isaksky]

@OvermindDL1 I get this:

C:\>nslookup repo.hex.pm
Server:  x
Address:  192.168.8.11

DNS request timed out.
    timeout was 2 seconds.
DNS request timed out.
    timeout was 2 seconds.
*** Request to x timed-out

C:\>nslookup repo.hex.pm
Server:  x
Address:  192.168.8.11

DNS request timed out.
    timeout was 2 seconds.
DNS request timed out.
    timeout was 2 seconds.
*** Request to x timed-out

C:\>nslookup repo.hex.pm
Server:  x
Address:  192.168.8.11

Non-authoritative answer:
Name:    n.global-ssl.fastly.net
Address:  151.101.69.5
Aliases:  repo.hex.pm

C:\>tracert repo.hex.pm

Tracing route to n.global-ssl.fastly.net [151.101.85.5]
over a maximum of 30 hops:

  1    <1 ms    <1 ms    <1 ms  192.168.8.1
  2    <1 ms    <1 ms    <1 ms  X
  3     3 ms     2 ms     3 ms  Y
  4     *       10 ms     3 ms  lag-16.ear2.denver1.level3.net [4.68.63.165]
  5     *        *        *     Request timed out.
  6   158 ms   158 ms   158 ms  ae-14-3512.ear1.stockholm2.level3.net [4.69.203.46]
  7   158 ms   158 ms   158 ms  149.6.168.50
  8   151 ms   151 ms   151 ms  151.101.85.5

Trace complete.

[ghost]

@OvermindDL1

 traceroute to n.global-ssl.fastly.net (151.101.1.5), 64 hops max
  1   192.168.1.1  1,385ms  10,561ms  2,601ms 
  2   *  *  * 
  3   187.100.86.240  6,562ms  5,626ms  5,960ms 
  4   187.100.196.118  8,149ms  3,632ms  5,839ms 
  5   187.100.53.174  95,739ms  87,492ms  8,413ms 
  6   *  *  * 

[OvermindDL1]

lag-16.ear2.denver1.level3.net [4.68.63.165]

Yep, that is the same issue I had.

If you can contact Fastly (you will not be able to contact Level3, they are one of the main Internet backbones) then have them diagnose (they will want access to a shell so they can test themselves) then maybe they can figure it out.

I worked with them for weeks a couple years ago with the same issue and they never could figure out why their packets vanished on 'some' routes.

Note the 'some', if you keep trying repeatedly then it can randomly work. If you clear the paths in your network stack each time that can help.

I ended up getting sick of it and just set up a proxy from that server to another on the other side of the country and redirected everything to the fastly IP's to go over it instead, yes it was a pain to set up, but it 'worked'. >.>

Good luck, please keep me updated, I hope you have better luck than I did! :-)

[ghost]

@OvermindDL1 sadly it doesn't seem to be the issue for me. I even tried doing it with different VPNs all over the world and the issue persisted so I am very confused by now...

[OvermindDL1]

@OvermindDL1 sadly it doesn't seem to be the issue for me. I even tried doing it with different VPNs all over the world and the issue persisted so I am very confused by now...

Was it talking from level3 -> fastly? It can happen all over as long as it is that kind of set up. Remember it is random too based on the path it decides to take over level3's network too.

[ghost]

@OvermindDL1 I didn't see any communication from level3 to fastly...

[ericmj]

Unfortunately this issue is not actionable so I will close it. Please contact Fastly support if you continue to have issues, their support is very good. For future issues with Hex please open them on https://github.com/hexpm/hex.