Azure-cli: az ml not a valid command in bash and powershell on azure cli in portal

[jtbarker]

Tried executing:
az ml account experimentation create -n jps00001 -g MyResourceGroup
Error:

az: error: argument _command_package: invalid choice: ml

Environment summary

az --version in BASH on azure portal:

jim@Azure:~$ az --version
azure-cli (2.0.23)

acr (2.0.17)
acs (2.0.22)
advisor (0.1.0)
appservice (0.1.22)
backup (1.0.3)
batch (3.1.7)
batchai (0.1.3)
billing (0.1.6)
cdn (0.0.10)
cloud (2.0.10)
cognitiveservices (0.1.9)
command-modules-nspkg (2.0.1)
configure (2.0.12)
consumption (0.2.0)
container (0.1.15)
core (2.0.23)
cosmosdb (0.1.15)
dla (0.0.15)
dls (0.0.18)
eventgrid (0.1.5)
extension (0.0.6)
feedback (2.0.6)
find (0.2.7)
interactive (0.3.11)
iot (0.1.15)
keyvault (2.0.15)
lab (0.0.13)
monitor (0.0.13)
network (2.0.19)
nspkg (3.0.1)
profile (2.0.16)
rdbms (0.0.9)
redis (0.2.10)
reservations (0.1.0)
resource (2.0.19)
role (2.0.15)
servicefabric (0.0.7)
sql (2.0.17)
storage (2.0.21)
vm (2.0.20)

Python location '/opt/az/bin/python3'
Extensions directory '/home/jim/.azure/cliextensions'

Python (Linux) 3.6.1 (default, Dec 15 2017, 17:51:12)
[GCC 4.8.4]

Legal docs and information: aka.ms/AzureCliLegal

az --version PowerShell in Azure:

Azure:> az --version
azure-cli (2.0.22)

acr (2.0.16)
acs (2.0.21)
advisor (0.1.0)
appservice (0.1.21)
backup (1.0.3)
batch (3.1.7)
batchai (0.1.3)
billing (0.1.6)
cdn (0.0.10)
cloud (2.0.10)
cognitiveservices (0.1.9)
command-modules-nspkg (2.0.1)
configure (2.0.12)
consumption (0.2.0)
container (0.1.14)
core (2.0.22)
cosmosdb (0.1.15)
dla (0.0.15)
dls (0.0.18)
eventgrid (0.1.5)
extension (0.0.6)
feedback (2.0.6)
find (0.2.7)
interactive (0.3.11)
iot (0.1.14)
keyvault (2.0.14)
lab (0.0.13)
monitor (0.0.13)
network (2.0.18)
nspkg (3.0.1)
profile (2.0.15)
rdbms (0.0.9)
redis (0.2.10)
reservations (0.1.0)
resource (2.0.19)
role (2.0.15)
servicefabric (0.0.6)
sql (2.0.16)
storage (2.0.20)
vm (2.0.19)

Python location 'C:Program Files (x86)Microsoft SDKsAzureCLI2python.exe'
Extensions directory 'C:UsersContainerAdministratorCloudDrive.pscloudshell.azurecliextensions'

Python (Windows) 3.6.1 (v3.6.1:69c0db5, Mar 21 2017, 17:54:52) [MSC v.1900 32 bit (Intel)]

Legal docs and information: aka.ms/AzureCliLegal

Shell Type Azure portal bash and PowerShell on Azure, both same result

[tjprescott]

ML is not a module in the CLI. The ML team offers a customized version of the CLI that includes these commands.

[tjprescott]

@derekbekoe link to the repo?

[derekbekoe]

cc: @trangevi

[tjprescott]

For questions on Azure ML CLI, please go to:
https://social.msdn.microsoft.com/Forums/azure/en-US/home?forum=MachineLearning

[BeifeiZhou]

The command of 'az ml' can not work in linux after installing by pip. The error is: 'az: error: argument _command_package: invalid choice: ml'

[tjprescott]

ML is not a module in the CLI. The ML team offers a customized version of the CLI that includes these commands.

[BeifeiZhou]

@tjprescott Could you share the customised version of the CLI for linux? It is not possible to use az ml in linux at this moment.

[tjprescott]

@BeifeiZhou there is no way to download and install the ML package for Azure CLI 2.0 on any OS. Instead you have to download Azure ML's customized CLI distribution. Please check here:
https://social.msdn.microsoft.com/Forums/azure/en-US/home?forum=MachineLearning

[vjrantal]

I had this same problem and in my case the issue was that pip installed azure-cli-ml to a location that wasn't looked at when executing the system default python.

You could either fix the python / pip installation so that both point to the same location or check which python pip uses and then explicitly load the module with that python binary. For example:

/usr/local/opt/python/bin/python2.7 -m azure.cli ml account -h

You can see how az loads the module with something like cat $(which az).

I also noticed that just doing pip install azure-cli-ml might result into version compatibility issues. Getting tested versions can be done with:

pip install -r https://aka.ms/az-ml-o16n-cli-requirements-file

[Mangaleshwar]

@vjrantal ,

I'm facing a similar issue where az ml does not work for normal user in Linux VM. But it works without any issues when run as a superuser.

I'm using MS ML on RHEL 7.2. I tried the steps you have provided, but that does not work as the VM already has az ml. Only issue here is that I'm unable to run az ml as normal user. Any possible workaround will be of great help. I'm also providing output from my VM below.

[azureuser@MLRHEL ~]$ az ml --help
az: error: argument _command_package: invalid choice: ml
usage: az [-h] [--verbose] [--debug] [--output {tsv,table,json,jsonc}]
[--query JMESPATH]

      {aks,backup,redis,network,cosmosdb,batch,iot,dla,group,webapp,acr,dls,storage,mysql,vm,reservations,account,keyvault,sql,vmss,eventgrid,managedapp,ad,advisor,postgres,container,policy,lab,batchai,functionapp,identity,role,cognitiveservices,monitor,sf,resource,cdn,tag,feedback,snapshot,disk,extension,acs,provider,cloud,lock,image,find,billing,appservice,login,consumption,feature,logout,configure,interactive}
      ...

[azureuser@MLRHEL ~]$ sudo az ml --help

Group
az ml: Machine Learning Server.
Perform administration tasks to configure nodes, set up the system, and run diagnostic and
capacity tests for Machine Learning Server.

Subgroups:
admin: Configure Machine Learning Server.

[azureuser@MLRHEL ~]$

Regards,
Mangaleshwar

[vjrantal]

@Mangaleshwar How was the az ml cli installed to the system?

What I've sometimes found useful is to create a new virtualenv and run this command there:

pip install -r https://aka.ms/az-ml-o16n-cli-requirements-file

Then, you are not affecting the versions installed in your system, but have a sandboxed environment of compatible versions for azure-cli-ml.

[Mangaleshwar]

@vjrantal ,

Thank you for the reply!!

I did not install az ml manually. This comes pre-installed in the "MS Machine Learning Server 9.3.0 on RHEL 7.2" image which is available in Azure marketplace. So I deployed VM from this image, and ran into the issue which I mentioned regarding az ml not working for normal user.

I did try pip install -r https://aka.ms/az-ml-o16n-cli-requirements-file , but it fails as the ml module is already available in the VM.

One of my colleague informed that it was working as expected previously, but something seems to have changed in the image. I have also raised a query on this in the Machine Learning MS forum, but haven't heard back anything yet.

[vjrantal]

@Mangaleshwar What about if you pass additionally --force-reinstall to pip install?

[Mangaleshwar]

@vjrantal I tried but getting below error.

Found existing installation: pyOpenSSL 0.13.1
Cannot uninstall 'pyOpenSSL'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.

I also tried with --ignore-installed, but still could not succeed.

[payalgaikwad42]

i have tried to install azure ml module using this command: pip install -r https://aka.ms/az-ml-o16n-cli-requirements-file. but i am still getting errors:
Error loading command module 'ml'
Error loading command module 'sf'
Error loading command module 'vm'
az: error: argument _command_package: invalid choice: ml