Serverless-offline: Calling multiple endpoints on 6.0.0-alpha.55 with --useDocker causes port bind error

[gauntface]

Bug Report

Current Behavior
I have multiple lambda functions and the first URL I access will work as expected. If I then access I second URL, i.e. a different function, I get the following error:

offline: GET /dev/ok (λ: ok)
Lambda API listening on port 9002...

2019/12/20 18:35:11 listen tcp 127.0.0.1:54321: bind: address already in use

Sample Code

• file: serverless.yml

service: my-service

plugins:
  - serverless-offline

provider:
  name: aws
  runtime: go1.x

functions:
  ok:
    handler: bin/ok
    events:
      - http:
          path: ok
          method: get
  other:
    handler: bin/other
    events:
      - http:
          path: /test/other
          method: get

Expected behavior/code

Multiple functions to be accessible.

Environment

• serverless version: v1.58.0
• serverless-offline version: 6.0.0-alpha.55
• node.js version: 12.14.0
• docker version: 19.03.2
• OS: Ubuntu 19.10

Possible Solution

My hunch is that docker needs to be told to bind to a unique port for each function.

[dnalborczyk]

hey @gauntface thanks for filing this issue!

yes, you are right, docker needs a port for every lambda. the docker implementation is also fairly new, so there might be bugs. that being said, we have a port finder in place. we even have some tests covering this scenario. I'll have a look. labeling as bug for now.

/cc @frozenbonito if you have an idea.

[dnalborczyk]

just tried to repro on Ubuntu 19.10, although with node.js v10.15.2. as well as node.js as runtime (as opposed to go-lang). no luck.

@gauntface the port 54321 seems a bit high and kinda doesn't look like it's been "randomly" sequentially chosen. portfinder currently starts at port 9000 and scans up (as far as I know).

also, this message Lambda API listening on port 9002... is from the container. the way it's currently set up, is that the exposed port should be the same as the internal port. meaning both should be 9002 and not 54321.

could you verify that 54321 this is the container port - or something else?

[frozenbonito]

I found the cause. It is golang image specific problem.

https://github.com/lambci/docker-lambda/blob/972e3e1fb117bd8c7ef2c43143e45db09e1824fb/go1.x/run/aws-lambda-mock.go#L82

[frozenbonito]

It only occurs on linux because serverless-offline runs the container with host networking mode to access the host service (e.g. local MySQL).

[dnalborczyk]

great, thanks again @frozenbonito !!

I suppose the port should be (or could be) set through the env variables as well (e.g. DOCKER_LAMBDA_API_PORT)?

runs the container with host networking mode to access the host service

I'm not really familiar with all the ins-and-outs of docker. is that something we should change? does it have any advantages? or is that the only way?

[frozenbonito]

@dnalborczyk

I suppose the port should be (or could be) set through the env variables as well (e.g. DOCKER_LAMBDA_API_PORT)?

Yes, I think it would solve this problem. But if the lambci/lambda image come to use a new port, this problem would occur again (not only golang image).

runs the container with host networking mode to access the host service

I'm not really familiar with all the ins-and-outs of docker. is that something we should change? does it have any advantages? or is that the only way?

Strictly speaking, I don't want use the host networking mode.

On windows and mac, docker provides the special DNS name host.docker.internal, which resolves to the internal IP address used by the host. However, it is not supported on linux.

• https://docs.docker.com/docker-for-windows/networking/#i-want-to-connect-from-a-container-to-a-service-on-the-host
• https://docs.docker.com/docker-for-mac/networking/#i-want-to-connect-from-a-container-to-a-service-on-the-host
• https://github.com/docker/for-linux/issues/264

I think it seems to be solved by using --add-host option of docker (ref: https://docs.docker.com/engine/reference/commandline/create/).
Let me try it.