Bentoml: SkLearnModelArtifact serving can lead to OSError 24: too many open files

[timlod]

Describe the bug
I am trying to bentoml serve Model:latest where Model includes in my case two large SkLearnModelArtifacts (around 2GB).
I've run into OSError: [Errno 24] Too many open files, and conjecture that the reason is that bentoML uses sklearn_model = joblib.load(model_file_path, mmap_mode='r') to load the artifacts (see https://github.com/joblib/joblib/issues/81).

To Reproduce

1. Pack a bento with (several?) large SkLearnModelArtifacts
2. Try to serve it with a "normal" ulimit -n (I had to set ulimit -n 10000 for this to actually run)
3. See error

Expected behavior
Any artifact is loaded without trouble (or there is an error when the model is saved, indicating that it is too large/otherwise bad to load).

Environment:

• Arch Linux
• Python 3.8.5
• BentoML-0.9.1

[parano]

Hi @timlod - it looks like there isn't much we can do on BentoML side for this issue, one workaround is to use BentoML's PickleArtifact to package your Scikit learn model: https://docs.bentoml.org/en/latest/api/artifacts.html?highlight=PickleArtifact#common-artifacts

[timlod]

I see, thanks for the heads-up!

[timlod]

Actually, it would be possible for BentoML to use joblib.load(model_file_path, mmap_mode=None) - then memory mapping is not used and the error won't occur. How vital is memmapping here?

[parano]

Actually, it would be possible for BentoML to use joblib.load(model_file_path, mmap_mode=None) - then memory mapping is not used and the error won't occur. How vital is memmapping here?

We can potentially make that an option and allow user to turn it off. E.g.

_SklearnArtifact("my_model", mmap_mode=None)_