Crystal: HTTP Server very slow on MacOS High Sierra

[mamantoha]

After upgrade to MacOS High Sierra, I faced an issue with HTTP Server speed.

I just use an example from the docs:

require "http/server"

server = HTTP::Server.new(8080) do |context|
  puts context.inspect
  context.response.content_type = "text/plain"
  context.response.print "hello world, got #{context.request.path}!"
end

puts "listening on http://127.0.0.1:8080"
server.listen

And use a simple client to make 10 requests:

require "http"

url = "http://localhost:8080/"

t = Time.now

10.times do |i|
  res = HTTP::Client.get(url)

  puts res.status_code
  puts res.body
end

puts Time.now - t

Result:

crystal get_request.cr
200
Hello world, got /!
200
Hello world, got /!
200
Hello world, got /!
200
Hello world, got /!
200
Hello world, got /!
200
Hello world, got /!
200
Hello world, got /!
200
Hello world, got /!
200
Hello world, got /!
200
Hello world, got /!
00:00:18.1139610

crystal -v
Crystal 0.23.1 (2017-10-12) LLVM 4.0.1

[jwoertink]

Yeah, I've been getting slow client use for a long time. My guess is maybe related to https://github.com/crystal-lang/crystal/issues/2660 ? I haven't tried this on linux, but it probably runs fine and related to a macOS issue.

[mamantoha]

Ubuntu 16.04

crystal client.cr 
200
hello world, got /!
200
hello world, got /!
200
hello world, got /!
200
hello world, got /!
200
hello world, got /!
200
hello world, got /!
200
hello world, got /!
200
hello world, got /!
200
hello world, got /!
200
hello world, got /!
00:00:00.0053150

crystal -v
Crystal 0.23.1 [e2a1389] (2017-07-13) LLVM 3.8.1

[asterite]

On my mac it works fast too. But I'm just on Sierra. Maybe it's only related to High Sierra. Or it's the DNS issue, which of course will vary from machine to machine.

[hugoabonizio]

You can try with 127.0.0.1:

require "http"

url = "http://127.0.0.1:8080/"

t = Time.now

10.times do |i|
  res = HTTP::Client.get(url)

  puts res.status_code
  puts res.body
end

puts Time.now - t

[jwoertink]

Switching to the IP speeds it up for me as well. That's definitely a pretty strong indicator of a DNS issue. I ran in to the same issue when I was trying to convert my site of to crystal. If I used the IP addresses of sites, it was fine, but using the domain name made it really slow. Looks like @ysbaddaden started a fix for this.

Then again, I haven't tried this out on the master branch to see if it's been fixed already.... Maybe after next release we can all try this again and see how it works.

[mamantoha]

Yep, switching to the IP in client fixes this problem. Probably this issue is related to DNS resolver.
Also, https://github.com/crystal-lang/crystal/releases/tag/0.24.0 have this issue.

[rdp]

does ruby have the same issue? is it an ipv4 vs ipv6 related maybe?

[Sija]

@rdp more like a DNS resolver, as stated before multiple times (+ there are several open PRs to address that).

[ericproulx]

Same issue with ruby, switching to 127.0.0.1 solved my problem.

[rdp]

if changing the client to url = "http://127.0.0.1:8080/" fixes the problem, and ruby has the same problem, my guess is it's not a crystal bug? (hint: tweaking /etc/hosts might help too...)

[jwoertink]

Looks like this is still an issue, though not really a crystal specific issue. It does this with ruby as well.

#server.cr
require "http/server"

server = HTTP::Server.new do |context|
  puts context.inspect
  context.response.content_type = "text/plain"
  context.response.print "hello world, got #{context.request.path}!"
end

puts "listening on http://127.0.0.1:8080"
server.listen(8080)

#client.cr
require "http"
#url = "http://127.0.0.1:8080/"
#url = "http://localhost:8080/"
url = "http://example.local:8080/"

elapsed = Time.measure do
  10.times do |i|
    res = HTTP::Client.get(url)

    puts res.status_code
    puts res.body
  end
end

puts elapsed

Running above examples against http://127.0.0.1:8080
00:00:00.012332426

Running above examples against http://localhost:8080
00:00:00.019486830

Adding example.local to /etc/hosts and running
00:00:50.076109462



md5-26436a42a58a34ad206d171eb584a7b9



[14:57PM] sandbox$ crystal -v
Crystal 0.27.0 (2018-11-04)

LLVM: 6.0.1
Default target: x86_64-apple-macosx



md5-56f15b0851192832f13acb6a476bc5f3



Running against http://127.0.0.1:8080
0.009027000050991774

Running against http://localhost:8080
0.012924999929964542

Running against http://example.local:8080
50.06058300007135

Lastly, came across this issue which shows to minify your /etc/hosts file. I tried that and then ran the example against example.local and my result was00:00:00.018588815.

So for anyone else that comes to this. Minify your /etc/hosts file, and that should help.

[RX14]

Closing as not our issue (though async DNS would probably help?)