Pulsar: Unable to create/register functions via REST API

Created on 23 Apr 2020  路  3Comments  路  Source: apache/pulsar

Describe the bug
Seems there was a change in the https://pulsar.apache.org/admin/v3/functions/{tenant}/{namespace}/{functionName} function, where the method signature changed, specifically the functionConfig parameter, which switched from a String (in 2.4.0) to an object (2.5.1) in FunctionsApiV3Resource.java as shown below in 2.5.1:

```
@POST
@Path("/{tenant}/{namespace}/{functionName}")
@Consumes(MediaType.MULTIPART_FORM_DATA)
public void registerFunction(final @PathParam("tenant") String tenant,
final @PathParam("namespace") String namespace,
final @PathParam("functionName") String functionName,
final @FormDataParam("data") InputStream uploadedInputStream,
final @FormDataParam("data") FormDataContentDisposition fileDetail,
final @FormDataParam("url") String functionPkgUrl,
final @FormDataParam("functionConfig") FunctionConfig functionConfig) {


POSTs that worked before now return `400, Bad Request, b'{"reason":"Function config is not provided"}'`
The result of this is it's seemingly impossible to register a function via the REST API (from python).  Perhaps I'm missing something?

**To Reproduce**
Steps to reproduce the behavior:
2 different approaches, neither seem to work?  Both return 

api_url = f"{API_URL_BASE}/v3/functions/{tenant}/{namespace}/{name}"
files = {
    'url': 'file://pulsar/lib/org.apache.pulsar-pulsar-functions-api-examples-2.5.1.jar',
    'functionConfig': json.dumps({ 'tenant': 'example'})
 response = requests.post(api_url, files=files)

error: 400, Bad Request, b'{"reason":"Function config is not provided"}'

mp_encoder = MultipartEncoder(
fields={
'url': 'file://pulsar/lib/org.apache.pulsar-pulsar-functions-api-examples-2.5.1.jar',
'functionConfig': json.dumps({ 'tenant': 'example'}),
}
)
headers = {'Content-Type': mp_encoder.content_type}
response = requests.post(api_url, data=mp_encoder, headers=headers)
```

error: 400, Bad Request, b'{"reason":"Function config is not provided"}'

Expected behavior
Expect an HTTP 204 and to see the function register.

Desktop (please complete the following information):

  • OS: osx posting to linux running pulsar within the pulsar-all docker container

Additional context
Add any other context about the problem here.

triagweek-17 typquestion

Most helpful comment

@qmnonic

The v2 endpoint still takes a serialized json string of FunctionConfig: /admin/v3/functions/{tenant}/{namespace}/{functionName}

There was a change in 2.5.0/2.4.1 - changing v3 endpoint was changed to be a more structured way for uploading a FunctionConfig.

What you basically need to add in the post payload is to make sure the functionConfig is a JSON payload in the form data.

I think what you can do in python:

mp_encoder = MultipartEncoder(
        fields={
            'url': 'file://pulsar/lib/org.apache.pulsar-pulsar-functions-api-examples-2.5.1.jar',
            'functionConfig': (None, json.dumps({ 'tenant': 'example'}), 'application/json')
        }
    )
    headers = {'Content-Type': mp_encoder.content_type}
     response = requests.post(api_url, data=mp_encoder, headers=headers)

All 3 comments

@qmnonic I am not sure if this is helpful, but I am able to upload functions via the REST API in versions 2.4.2 to 2.5.1 using this JavaScript code:

createFunction (cluster, functionPath, functionConfig, fileBlob, fileName) {
    let formData = new FormData()
    // Convert the functionConfig object to JSON then store it in a blob
    var blob = new Blob([JSON.stringify(functionConfig)], { type: 'application/json' })
    formData.append('functionConfig', blob)
    formData.append('data', fileBlob, fileName)

    const response = ApiBase().post(`${cluster}/functions/${functionPath}`, formData, {
      headers: {
        'Content-Type': 'multipart/form-data'
      }
    })
    return response
  },

Sorry, but I have no idea how to translate this to Python.

@qmnonic

The v2 endpoint still takes a serialized json string of FunctionConfig: /admin/v3/functions/{tenant}/{namespace}/{functionName}

There was a change in 2.5.0/2.4.1 - changing v3 endpoint was changed to be a more structured way for uploading a FunctionConfig.

What you basically need to add in the post payload is to make sure the functionConfig is a JSON payload in the form data.

I think what you can do in python:

mp_encoder = MultipartEncoder(
        fields={
            'url': 'file://pulsar/lib/org.apache.pulsar-pulsar-functions-api-examples-2.5.1.jar',
            'functionConfig': (None, json.dumps({ 'tenant': 'example'}), 'application/json')
        }
    )
    headers = {'Content-Type': mp_encoder.content_type}
     response = requests.post(api_url, data=mp_encoder, headers=headers)

@sijie solution works. thanks!

Thanks @cdbartholomew for the example !

Was this page helpful?
0 / 5 - 0 ratings