Sagemaker-python-sdk: Is it possible to deploy GluonCV models with SageMaker without providing a user script?

[velociraptor111]

I am using GluonCV and hoping to be able to simply deploy a trained GluonCV model.

I am following the steps listed here: https://aws.amazon.com/blogs/machine-learning/bring-your-own-pre-trained-mxnet-or-tensorflow-models-into-amazon-sagemaker/

I have done everything up to this point:

sagemaker_model = MXNetModel(model_data = 's3://' + sagemaker_session.default_bucket() + '/model/model.tar.gz',
                             role = role,
                             entry_point = 'classifier.py')

I already have the tar.gz files in my s3 bucket.

HOWEVER, I do not have a training script to put as the entry point, since I am simply loading a gluon model like follows:

from gluoncv import model_zoo, data, utils
detector = model_zoo.get_model('yolo3_mobilenet1.0_coco', pretrained=True)

Then converting this to MXNet model and saving with the help of this: https://medium.com/@julsimon/quick-tip-converting-gluon-models-to-symbolic-format-9513e5dd1d73

Question is:
Is it possible to simply deploy without the training script provided?

[ChoiByungWook]

Hi @velociraptor111,

Unfortunately you will need to provide an entry_point pointing to a file. This script is needed to specify how to load and serve the model in SageMaker.

Since you are only deploying your model and not doing training, your entry_point doesn't need to have any training code and would only have to specify the functions needed for deploying as defined here: https://sagemaker.readthedocs.io/en/stable/using_mxnet.html#load-a-model

There are default implementations for each of the functions, which will be used if the function isn't defined in your "entry_point" file. Information on those defaults can be found here: https://sagemaker.readthedocs.io/en/stable/using_mxnet.html#using-input-fn-predict-fn-and-output-fn

If you wish to use defaults for all of the function definitions, you will need to provide an empty file and point the entry_point to that, however the default might not suffice for your use case depending on the type of model, serialization and deserialization it is expecting.

Apologies for the inconvenience.

[velociraptor111]

Hi thank you very much for replying,

I have looked into the links you sent me and implemented a deploy.py where this is the content

import mxnet as mx 
import os

def model_fn(model_dir):
    print(mx.__version__)
    sym, arg, aux = mx.model.load_checkpoint('%s/yolo_object_person_detector' % model_dir , 0)
    mod = mx.mod.Module(symbol=sym, label_names=None)
    mod.bind(for_training=False, data_shapes=[('data', (1,3,512,512))])
    mod.set_params(arg, aux)
    return mod

However, after running

sagemaker_session = sagemaker.Session()
sagemaker_model = MXNetModel(model_data = 's3://' + sagemaker_session.default_bucket() + '/model/yolo_object_person_detector.tar.gz',
                                  role = role,
                                  entry_point = 'deploy.py',py_version='py3')

I am receiving this error message
mxnet.base.MXNetError: Cannot find argument 'infer_range', Possible Arguments:

I think this is due to the OLD version of MXNet being in the environment when the code deploy.py is being executed. I printed the version of MXNet during runtime and it is version 1.2.1 whereas the checkpoints saved in S3 are being produced from version 1.5.

How do I solve this issue? Is it possible to increase the versioning of MXNet?

[velociraptor111]

Sorry, I just finally found the right doc which explains the parameter
The fix is very simple, just add a framework_version parameter

So the fix is like so:

sagemaker_session = sagemaker.Session()
sagemaker_model = MXNetModel(model_data = 's3://' + sagemaker_session.default_bucket() + '/model/yolo_object_person_detector.tar.gz',
                                  role = role, entry_point = 'entry_point.py',py_version='py3',framework_version='1.4.1')

And this works since MXNet version 1.5 is backward-compatible with version 1.4.1 (at least for my code)

Nonetheless, thanks for being awesome and quick reply!