Recommenders: Testing Hyperdrive notebooks

[anargyri]

Description

There are now a couple of notebooks showing hyperparameter tuning using Hyperdrive.
https://github.com/Microsoft/Recommenders/blob/649f0a8918c9aab1ed24b0f306c8691119bf89c8/notebooks/04_model_select_and_optimize/hypertune_aml_wide_and_deep_quickstart.ipynb
https://github.com/Microsoft/Recommenders/blob/b8490748cbd9072e395fcd31e5a4db3287c56b57/notebooks/04_model_select_and_optimize/azureml_hyperdrive_tuning_svd.ipynb

It is not straightforward how to test these notebooks, since this would involve the AzureML infrastructure.

Expected behavior with the suggested feature

Discuss how feasible this is and what is required. How long would the tests take to execute; do we want to have smoke and integration tests or just unit tests?

[yueguoguo]

It might be good that we make ALL the Azure practices well documented and tested. Quite a few notebooks and utils have duplicated content and they should be better organized. In addition to this, it should be made clear to reader that which of the notebooks are executable in a local environment and which are on cloud.

[loomlike]

Copied my comment from other thread:

One way to check error (failure) is start a run and wait until the run finishes (run.wait_for_completion()) and check its status with get_status() to see if it is "Completed" or "Failed".

[miguelgfierro]

This can be addressed using this repo https://github.com/Microsoft/MLAKSDeployAML made by @fboylu and @grecoe

[jingyanwangms]

Same issue for PR 575 and PR 681. We're looking into it