Azure-docs: Deploying/Consuming AzureML webservices with 2+ arguments

[nansravn]

What is the right way to consume the AzureML Deployed Webservice when deploying a model with 2+ arguments?

---

I have deployed a model using 2 arguments on the function run.

def run(batch_string, model_id):
    #some code here 

I've tried like that:

import json
test_samples = json.dumps({"batch_string": "This is a string", "model_id":42})
test_sample = bytes(test_samples,encoding = 'utf8')
aks_service.run(test_sample)

But I've received the following error code:

---------------------------------------------------------------------------
WebserviceException                       Traceback (most recent call last)
<ipython-input-107-8f1ec51edd66> in <module>()
----> 1 aks_service.run(test_sample)

~/anaconda2/envs/aml36/lib/python3.6/site-packages/azureml/core/webservice/aks.py in run(self, input_data)
    330                                       'Response Code: {}\n'
    331                                       'Headers: {}\n'
--> 332                                       'Content: {}'.format(resp.status_code, resp.headers, resp.content))
    333 
    334     def update(self, image=None, autoscale_enabled=None, autoscale_min_replicas=None, autoscale_max_replicas=None,

WebserviceException: Received bad response from service:
Response Code: 502
Headers: {'content-length': '56', 'cache-control': 'max-age=0, private, must-revalidate', 'x-ms-request-id': '03c2ceae-1581-46f2-8745-46e3810bdd9f', 'connection': 'keep-alive', 'content-type': 'text/html; charset=utf-8', 'date': 'Tue, 09 Oct 2018 14:24:43 GMT', 'server': 'nginx/1.10.3 (Ubuntu)', 'x-ms-run-function-failed': 'True'}
Content: b"run() missing 1 required positional argument: 'model_id'"

I've also tried to consume this model via cURL, but received an equivalent error:
curl -X POST -H "Content-Type:application/json" -H "Authorization:Bearer <my-auth-key>" --data "{\"batch_string\": \"This is another string\", \"model_id\": 21}" <my-scoring-url>

Return:
run() missing 1 required positional argument: 'model_id'

---

Document Details

⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

• ID: cc87eb58-5bdf-15d3-1a52-ca9d0706726f
• Version Independent ID: 460fb6ba-ca28-2505-b4d9-1dec3972d968
• Content: Deploy models to Kubernetes from Azure Machine Learning service
• Content Source: articles/machine-learning/service/how-to-deploy-to-aks.md
• Service: machine-learning
• GitHub Login: @raymondlaghaeian
• Microsoft Alias: raymondl

[YutongTie-MSFT]

@nansravn Thanks for the feedback! We are currently investigating and will update you shortly.

[rlagh2]

@nansravn The input into the run(input_data) function is a json paylod which can have multiple nodes which mean you can include multiple arguments inside it. For example:

Input_data = { “batch_string": "this is a string", "model_id": 123 }

You would then parse the json inside the run function and access each node's value.

[nansravn]

Ok @raymondlaghaeian, I'll adapt my code to have only 1 argument which is a JSON with multiple keys inside it.

So, based on your answer, nowadays the only type of data accepted as an argument are JSONs (application/json)? Is it possible to send binary data to the webservice (application/octet-stream)? (for example if I want to make an Object Detection Webservice, sending binary images without any base64 or ASCII encoding)

[j-martens]

@YutongTie-MSFT - please assign to @Blackmist thank you.

[Blackmist]

@nansravn Support for binary data is coming soon-ish. I'm waiting on an example that demonstrates handling binary data, which I'll use to update this doc. The example is held up currently as it relies on some back-end bits that aren't ready yet. I'll respond here once the example is ready and documentation updated.

[Blackmist]

@nansravn Updating the doc today. Binary data requires an update to your score.py file to accept raw HTTP data. I don't have an example model/project for this, but do have an example score.py that accepts binary data, reverses the byte order, and then returns the reversed bytes.

from azureml.contrib.services.aml_request  import AMLRequest, rawhttp
from azureml.contrib.services.aml_response import AMLResponse

def init():
    print("This is init()")

@rawhttp
def run(request):
    print("This is run()")
    print("Request: [{0}]".format(request))
    if request.method == 'GET':
        respBody = str.encode(request.full_path)
        return AMLResponse(respBody, 200)
    elif request.method == 'POST':
        reqBody = request.get_data(False)
        respBody = bytearray(reqBody)
        respBody.reverse()
        respBody = bytes(respBody)
        return AMLResponse(respBody, 200)
    else:
        return AMLResponse("bad request", 500)

[nansravn]

@Blackmist, this already helps me a lot!

I'll test it and prepare an example code to publish on my github, so we can have a deployment example reference.

[Blackmist]

@nansravn Glad it helps! One disclaimer, which will be in the docs, is that his uses things from the azureml.contrib namespace. Things in here are going to change fairly often as improvements are made to the product. So any code you write against these objects might need to be updated in the next SDK release. Once the code/feature is "finished" it will move out of contrib and into another part of the SDK namespace.

[nansravn]

@Blackmist I'm supporting deployment in which we are testing this preview feature using azureml-core[databricks] version 1.0.2 (inside a Databricks Workspace).
Basically I'm trying to receive an image as bytes for a deployment of a Computer Vision ML model.

from azureml.contrib.services.aml_request  import rawhttp
from azureml.contrib.services.aml_response import AMLResponse

from PIL import Image
from io import BytesIO
import json

def init():
   pass

@rawhttp
def run(request):
   if request.method == 'POST':
      reqBody = request.get_data(False)
      myImage = Image.open(BytesIO(reqBody))
      return json.dumps(myImage.size) #Example, returning the size of the image
   else:
        return AMLResponse("bad request", 500)

If I want to load a PIL.Image from request.get_data(), should I use io.BytesIO() as in this documentation?

Theretically, am I doing something wrong in the example above? I'm basically trying to load an image from bytes and then return the image size as a json.

[nansravn]

@Blackmist The code above worked well, thanks a lot for your contribution! I've made a production deployment of a TensorFlow Object Detection model using the same logic of the above code.

In the future I'll update my code, when this feature leaves the azureml.contrib namespace.