Bentoml: Add --with-docker argument for running BentoML inference job from CLI

[parano]

Is your feature request related to a problem? Please describe.

When launching a batch inference job from CLI, it is quite common for users to ask BentoML to set up all the conda/PyPI dependencies automatically for the job.

In the older BentoML version, there's a --with-conda argument that does exactly this. But we removed it in #898 due to its inconsistency with the docker environment and the service may be missing additional dependencies when only considering its conda and python dependencies.

Currently, a user will need to first build a docker container image and then launch the inference job with the docker image, here’s an example to run CLI predict with docker, using the quickstart guide example:

git clone [email protected]:bentoml/BentoML.git
pip install -r ./bentoml/guides/quick-start/requirements.txt
python ./bentoml/guides/quick-start/main.py

bentoml containerize IrisClassifier:latest -t iris-classifier

docker run -v $(PWD):/data iris-classifier:20200923110808_241FF4 bentoml run /bento predict --input-file=/data/test.csv

It works but it's quite cumbersome for a user to set up the docker environment and figure out mounting local files to the docker container etc.

Describe the solution you'd like

BentoML should provide a more convenient way for the user to do this via one CLI command, e.g.:

$ bentoml run IrisClassifier:latest predict --input-file=./test.csv --format=csv --with-docker

Which should automatically build a docker image and start a docker container to run an inference job with the specified input data

Describe alternatives you've considered

Suggestions for alternatives are welcomed

Additional context

N/A

[jackyzha0]

To me, it's not obvious that --with-docker will also build the image as well. Maybe adding a --build flag to make the Docker build step optional so it just does the docker run part? If the flag isn't included we should look for an existing Docker image and if it doesn't exist, throw a warning.

[parano]

@jackyzha0 I think it's ok to just always build the docker image and users don't need to provide this extra argument. Note that if the docker image has been built before even with a different tag, docker build should just use the cached layers and the build will be very fast.

[iancoffey]

As a bit of an aside- Ive also run into an issue where most of the environments I want to build BentoML services will not have Docker running, so I am working on this small helper to create K8s Jobs or Tekton Tasks automatically - https://github.com/iancoffey/bentoml-builder