Crystal: HTTP::Client OpenSSL socket leaks

[ghost]

Summary

If the HTTP::Client request fails, it will occupy FileDescriptor.

This happens only with invalid https URLs.
(throw Error reading socket: Connection reset by peer).

These failures (CLOSED) will cause FileDescriptor to be occupied.
(wasting system resources).

Excessive use of FileDescriptor will result in too many open files and crash.
This issue only occurs under https (OpenSSL)
This error can be triggered by an invalid virtual network card.

Lsof

ps ax | grep {{name}}
lsof -p {{pid}}

Example

require "http/client"

# Must throw an error (i.e. `Error reading socket: Connection reset by peer`).
uri = URI.parse "https://something.af"

begin
  HTTP::Client.get url: uri do |y|
  end

  # The same problem.
  # HTTP::Client.new uri do |y|
  #  y.connect_timeout = 30.second
  #  y.read_timeout = 30.second
  #  y.get("/")
  # end
rescue ex : Errno
  # Error reading socket: Connection reset by peer
  puts [ex.message]
end

loop do
end

Verbose

$ lsof -p 6272
COMMAND  PID USER   FD     TYPE             DEVICE  SIZE/OFF    NODE NAME
mod     6272 User  cwd      DIR                1,2      1156  443611 /Users/User/Downloads
mod     6272 User  txt      REG                1,2   2122596 6470086 /Users/User/Downloads/mod
mod     6272 User  txt      REG                1,2    381200 5486131 /usr/local/Cellar/openssl/1.0.2r/lib/libssl.1.0.0.dylib
mod     6272 User  txt      REG                1,2   1999820 5486128 /usr/local/Cellar/openssl/1.0.2r/lib/libcrypto.1.0.0.dylib
mod     6272 User  txt      REG                1,2    416976  422220 /usr/lib/libpcre.0.dylib
mod     6272 User  txt      REG                1,2    243708  640350 /usr/local/Cellar/libevent/2.1.8/lib/libevent-2.1.6.dylib
mod     6272 User  txt      REG                1,2    698896  422030 /usr/lib/dyld
mod     6272 User  txt      REG                1,2 662274048 3565842 /private/var/db/dyld/dyld_shared_cache_x86_64h
mod     6272 User    0u     CHR               16,3    0t1158    1141 /dev/ttys003
mod     6272 User    1u     CHR               16,3    0t1158    1141 /dev/ttys003
mod     6272 User    2u     CHR               16,3    0t1158    1141 /dev/ttys003
mod     6272 User    3u  KQUEUE                                      count=1, state=0x8
mod     6272 User    4r     CHR               14,1      0t32     574 /dev/urandom
mod     6272 User    5u     CHR               16,3      0t51    1141 /dev/ttys003
mod     6272 User    6u     CHR               16,3       0t0    1141 /dev/ttys003
mod     6272 User    7     PIPE 0x89aae7a93be7f26f     16384         ->0x89aae7a93be7f3ef
mod     6272 User    8     PIPE 0x89aae7a93be7f3ef     16384         ->0x89aae7a93be7f26f
mod     6272 User    9u   systm                          0t0         
mod     6272 User   10u    unix 0x89aae7a94bfaa93f       0t0         ->0x89aae7a94bfaa7af
> mod     6272 User   11u    IPv4 0x89aae7a94565f527       0t0     TCP 198.18.0.1:61414->198.18.5.54:https (CLOSED)

[ghost]

Update

After some debugging, I finally found out that it caused problems.
crystal/src/http/client.cr#L742

• Repair solution
  
  
  
  • Add begin..rescue and _socket = socket and _socket.close fixes(CLOSED).
  
  

    {% if !flag?(:without_openssl) %}
      if tls = @tls
        _socket = socket
        begin
          socket = OpenSSL::SSL::Socket::Client.new(socket, context: tls, sync_close: true, hostname: @host)
        rescue
          _socket.close
        end
      end
    {% end %}

[jhass]

Good find! PR welcome :)