Sagemaker-python-sdk: Limited Size of Hyperparameters argument for Estimator

[InzamamAnwar]

Please fill out the form below.

System Information

• Framework (e.g. TensorFlow) / Algorithm (e.g. KMeans):
  Custom ARIMA algorithm (statsmodels package)
• Python Version:
  3.6
• CPU or GPU:
  CPU
• Python SDK Version:
  boto3=1.9.206, sagemaker=1.36.2
• Are you using a custom image:
  Yes
  

  Describe the problem

We are training more than 2 thousand ARIMA algorithms for which we try to pass hyperparameters. We need to pass a dict of dict with model number and its parameters. But as described in docs max number of parameters is 100 with each key, value length not exceeding 256 char. Kindly suggest some other way around or increase the Length Constraint as a new feature.

P.S. A way around could be to save parameters when creating image but we are looking for a dynamic way to handle this

[otter-bunny]

Hi @InzamamAnwar, thank you for using SageMaker! We have notified the corresponding team to review this request and prioritize it in work planning. In the meantime, we also welcome you to create a Pull Request if you would like to make the implementations. :)

[laurenyu]

reopening since this is a feature request.

as a workaround, the corresponding service team has suggested passing the large hyperparameter set as an additional config file passed in as a separate input channel to the training job.

[InzamamAnwar]

@laurenyu Already employed this solution and its working.

Thanks

[ricoms]

Hello @laurenyu, sorry for the ignorance. What do you call a "separate input channel to the training job"?

I see a solution that my code will receive a URL as a parameter, and my code will need to download the config file from that URL. Is that it?

Is that "input channel" something easier?

[laurenyu]

@ricoms when you pass input to your training job, it will be available to the Docker container under /opt/ml/input. Each channel has its own directory, so if you save your hyperparameters in a file in S3, you can tell SageMaker to download that file as an input file.

Relevant AWS documentation: https://docs.aws.amazon.com/sagemaker/latest/dg/API_CreateTrainingJob.html#SageMaker-CreateTrainingJob-request-InputDataConfig

[icywang86rui]

@ricoms Does the response from @laurenyu make sense? Do you have more questions?