Serverless-python-requirements: Unable to get good bind path format / docker: Error response from daemon

[MitchFrankel]

I've been beating on this for 3 days and been through all sorts of forum issues and posts and cannot resolve. I'm trying to package numpy in a function, individually building requirements (I have multiple functions with multiple requirements that I'd like to keep separate).

Environment:
Windows 10 Home
Docker Toolbox for Windows:
```Client:
Version: 18.03.0-ce
API version: 1.37
Go version: go1.9.4
Git commit: 0520e24302
Built: Fri Mar 23 08:31:36 2018
OS/Arch: windows/amd64
Experimental: false
Orchestrator: swarm

Server: Docker Engine - Community
Engine:
Version: 18.09.0
API version: 1.39 (minimum version 1.12)
Go version: go1.10.4
Git commit: 4d60db4
Built: Wed Nov 7 00:52:55 2018
OS/Arch: linux/amd64
Experimental: false

serverless version 6.4.1
serverless-python-requirements version 6.4.1

serverless.yml

service: test
plugins:

• serverless-python-requirements
  custom:
  pythonRequirements:
  zip: true
  dockerFile: Dockerfile
  dockerizePip: non-linux
  provider:
  name: aws
  runtime: python3.6
  stage: dev
  environment: ${file(./env.yml):${opt:stage, self:provider.stage}.env}
  region: ${file(./env.yml):${opt:stage, self:provider.stage}.aws.region}
  profile: ${file(./env.yml):${opt:stage, self:provider.stage}.aws.profile}
  package:
  individually: true

AWS Lambda execution environment is based on Amazon Linux 1

FROM amazonlinux:1

Install Python 3.6

RUN yum -y install python36 python36-pip

Install your dependencies

RUN curl -s https://bootstrap.pypa.io/get-pip.py | python3
RUN yum -y install python3-devel mysql-devel gcc

Set the same WORKDIR as default image

RUN mkdir /var/task
WORKDIR /var/task

I have my project files in C:\Serverless\test. I run `npm init`, followed by `npm i --save serverless-python-requirements`, accepting all defaults. I get the following on `sls deploy -v`. even though I've added C:\ to Shared Folders on the running default VM in VirtualBox, and selected auto-mount and permanent.

![image](https://user-images.githubusercontent.com/24832609/50174470-811ddf00-02b7-11e9-9b46-208a0b395802.png)

Serverless: Building custom docker image from Dockerfile...
Serverless: Docker Image: sls-py-reqs-custom

Error --------------------------------------------------

Unable to find good bind path format

 For debugging logs, run again after setting the "SLS_DEBUG=*" environment variable.

Stack Trace --------------------------------------------

Error: Unable to find good bind path format
at getBindPath (C:Serverlesstestnode_modulesserverless-python-requirementslibdocker.js:142:9)
at installRequirements (C:Serverlesstestnode_modulesserverless-python-requirementslibpip.js:152:7)
at installRequirementsIfNeeded (C:Serverlesstestnode_modulesserverless-python-requirementslibpip.js:451:3)

If I move my project to C:\Users\, I get this instead:

Serverless: Docker Image: sls-py-reqs-custom
Serverless: Trying bindPath /c/Users/Serverless/test/.serverless/requirements (run,--rm,-v,/c/Users/Serverless/test/.serverless/req
uirements:/test,alpine,ls,/test/requirements.txt)
Serverless: /test/requirements.txt

Error --------------------------------------------------

docker: Error response from daemon: create "/c/Users/Serverless/test/.serverless/requirements": ""/c/Users/Serverless/test/.serv
erless/requirements"" includes invalid characters for a local volume name, only "[a-zA-Z0-9][a-zA-Z0-9_.-]" are allowed. If you in
tended to pass a host directory, use absolute path.
See 'docker run --help'.

 For debugging logs, run again after setting the "SLS_DEBUG=*" environment variable.

Stack Trace --------------------------------------------

Error: docker: Error response from daemon: create "/c/Users/Serverless/test/.serverless/requirements": ""/c/Users/Serverless/test/
.serverless/requirements"" includes invalid characters for a local volume name, only "[a-zA-Z0-9][a-zA-Z0-9_.-]" are allowed. If y
ou intended to pass a host directory, use absolute path.
See 'docker run --help'.

at dockerCommand (C:\Users\Serverless\test\node_modules\serverless-python-requirements\lib\docker.js:20:11)
at getDockerUid (C:\Users\Serverless\test\node_modules\serverless-python-requirements\lib\docker.js:162:14)

```

I'm a bit at a loss as to what to do next and advice would be greatly appreciated. TIA.

[jackson-jackson]

I'm having the exact same problem on Windows 10 Pro.

[nhudacin]

I'm not sure this will friends but I had the same issue myself, Windows 10 1803. Simply un-sharing my C drive and re-sharing it got me past the Unable to find good bind path format error.

I can't even take credit for the solution, I pulled it from this thread: #210

I have a new error that I'm sure is waiting just beyond this one for you. docker: Error response from daemon: invalid mode: /test.. I will investigate and file an additional issue if I cannot resolve it.

[MitchFrankel]

The unsharing/resharing has not worked for me. I've been at this for almost 3 weeks with no resolution at this point.

[MitchFrankel]

I was unable to make the plugin work but I found a better solution anyhow - Lambda Layers. This is a bonus because it reduces the size of the lambda and allows code/file reuse. There is a pre-built lambda layer for numpy and scipy that you can use, but I built my own to show myself how it all works. Here's how I made it work:

Create a layer package:

1. Open an EC2 instance or Ubuntu or Linux or whatever - This is needed so we can compile the runtime binaries correctly
2. Make a dependencies package zip - Must use the directory structure python/lib/python3.6/site-packages for python to find during runtime

mkdir -p tmpdir/python/lib/python3.6/site-packages 
pip install numpy==1.15.4 --no-deps -t tmpdir/python/lib/python3.6/site-packages 
cd tmpdir zip -r ../py_dependencies.zip . 
cd .. 
rm -r tmpdir

1. Push layer zip to AWS - requires latest awscli

sudo pip install awscli --upgrade --user
sudo aws lambda publish-layer-version \
--layer-name py_dependencies \
--description "Python 3.6 dependencies [numpy=0.15.4]" \
--license-info "MIT" \
--compatible-runtimes python3.6 \
--zip-file fileb://py_dependencies.zip \
--profile python_dev_serverless

1. To use in any function that requires numpy, just use the arn that is shown in the console or during the upload above

f1:
    handler: index.handler_f_use_numpy
    include:
      - functions/f_use_numpy.py
    layers:
      - arn:aws:lambda:us-west-2:XXXXX:layer:py_dependencies:1

1. As an added bonus, you can push common files like constants to a layer as well. Here's how I did it for testing use in windows and on the lambda:

import platform

# Set common path
COMMON_PATH = "../../layers/common/"
if platform.system() == "Linux": COMMON_PATH = "/opt/common/"

def handler_common(event, context):
    # Read from a constants.json file
    with open(COMMON_PATH + 'constants.json') as f:
        return text = json.load(f)

[dschep]

Yeah, if all you need is numpy & scipy, I definitely recommend using AWS's layer.

Creating your own on EC2 or with docker (but you'll need to figure out the bind path 😉) or a VM is definitely a good option, you can also use the serverless framework to publish your layer: https://serverless.com/blog/publish-aws-lambda-layers-serverless-framework. I'd recommend the last technique used, the one with cloudformation exports.

[MitchFrankel]

I only use the EC2 because I have a nice development AMI that is essentially a fully configured Ubuntu machine with remote desktop connection. Ubuntu for Windows would work ok just fine as well [EDIT: Ubuntu for windows does not work with layer publishing as of 1/4/19]. The reason I push the layers separately from serverless is because I want to use the layers across multiple services (e.g., a py36_core_dependencies layer with numpy, scipy, scikit-learn, etc.). It's easy to then add the add the layers via an env.yml or the serverless.yml. While I wish I had made this plugin work, I think layers is a better overall solution because it reduces lambda package size and allows code reuse across services.

[ajwilkinson]

Ok so, to get all this stuff to fly on Windows 10 (today's challenge, as I had to switch from a work Mac)...

• Installed Docker Toolbox (as I also use VMware, so couldn't use the Hyper-V based Desktop one)
• Go into VirtualBox, make sure that your C drive is automount shared as /C in the configuration there, and restart the virtual machine - go in with docker-machine ssh to check that /C does indeed reflect your C drive - this deals with not being able to find bind paths.
• Now for a bit of code hacking - in node_modules/serverless-python-requirements/lib/pip.js, inside dockerPathForWin, change the return value for windows to be

return `${path.replace(/\\/g, '/')}`;

(we're removing the extra double quotes - this deals with the complaints about incorrect characters in paths)

• And finally, inside your serverless.yml file , make sure to specify

custom:
  pythonRequirements:
    dockerizePip: non-linux
    pythonBin: python

(this deals with the package defaulting to 'python.exe' which clearly won't work inside the docker container).
And thus, I have been able to sls deploy from Windows 10.

[bongbang]

I solved the problem by changing the name of the shared folder from C_DRIVE to c. Try it.

[dschep]

I'm trying to debug this.. does anyone know how to run docker toolbox in windows in a VirtualBox VM? I don't have a windows machine :/