Cognitive-services-speech-sdk: Run the Speech-to-text example in Databricks - Install the python-client in Databricks

Created on 18 May 2020  路  3Comments  路  Source: Azure-Samples/cognitive-services-speech-sdk

Describe the bug
Install the python-client in Databricks

To Reproduce
Steps to reproduce the behavior:
I want to set my speech-to-text into production. Therefore, we are using the Azure Cloud. Within the Cloud, we have Databricks running. There I try to install the 'API Client library' (in the example done via pip install).

  1. Go to https://editor.swagger.io.
  2. Click File, then click Import URL.
  3. Enter the Swagger URL including the region for your Speech service subscription: https://.cris.ai/docs/v2.0/swagger.
  4. Click Generate Client and select Python.
  5. Save the client library.
  6. Extract the downloaded python-client-generated.zip somewhere in your file system.
  7. Install the extracted python-client module in your Python environment using pip: pip install path/to/package/python-client.
  8. The installed package has the name swagger_client. You can check that the installation worked using the command python -c "import swagger_client".

However, are there any best practices how you can install the API client library (the 'swagger client') within a Databricks notebook?

Steps followed:

Thanks!

Most helpful comment

In more detail, if needed:

Start a command prompt and go to the extracted python-client directory. Create an egg package with a command:

python setup.py bdist_egg

Note, I would use Python x.y version that matches the one installed on your Databricks cluster (check with import sys, sys.version in a Notebook cell).

The created egg package will be in a dist subdirectory under python_client, e.g. python_client/dist/swagger_client-1.0.0-py3.7.egg when generated with Python 3.7.

In your Azure Databricks portal, go to Workspace, Create, Library

  • Library Source: Upload
  • Library Type: Python Egg
  • drag and drop the egg package you previously created from your local disk
  • Create

If your cluster is already running, select it from the list at the end of library creation and Install the library, otherwise start the cluster, then go to Workspace and select the library, select the cluster, Install.

Alternatively, if you need the library only on a single cluster, go to Clusters, select the cluster, Libraries, Install New, and upload as above.

If you have a Notebook attached to the cluster, Detach and Re-attach it. Then import swagger_client etc. should work. I tested this on clusters with Databricks runtime versions 5.5 LTS and 7.0 Beta.

All 3 comments

If you using a python notebook, try the following things:

  1. Download the python client
  2. Build python egg
  3. import python egg as a library

In more detail, if needed:

Start a command prompt and go to the extracted python-client directory. Create an egg package with a command:

python setup.py bdist_egg

Note, I would use Python x.y version that matches the one installed on your Databricks cluster (check with import sys, sys.version in a Notebook cell).

The created egg package will be in a dist subdirectory under python_client, e.g. python_client/dist/swagger_client-1.0.0-py3.7.egg when generated with Python 3.7.

In your Azure Databricks portal, go to Workspace, Create, Library

  • Library Source: Upload
  • Library Type: Python Egg
  • drag and drop the egg package you previously created from your local disk
  • Create

If your cluster is already running, select it from the list at the end of library creation and Install the library, otherwise start the cluster, then go to Workspace and select the library, select the cluster, Install.

Alternatively, if you need the library only on a single cluster, go to Clusters, select the cluster, Libraries, Install New, and upload as above.

If you have a Notebook attached to the cluster, Detach and Re-attach it. Then import swagger_client etc. should work. I tested this on clusters with Databricks runtime versions 5.5 LTS and 7.0 Beta.

Closing as answered, please re-open if more info needed.

Was this page helpful?
0 / 5 - 0 ratings