Azure-sdk-for-python: AttributeError: 'str' object has no attribute 'get'

[nlsmay1]

I am creating a pipeline to load data from Blob into AzureSQL table as follows:

I am using the latest version of Azure Data Factory and Python version is 3.7.0

Create a copying activity

act_name =  'copyBlobtoBlob'
blob_source = BlobSource()
table_sink  = AzureSqlTableDataset(ls_tgt_name,dsout_name)
dsin_ref = DatasetReference(ds_name)
dsOut_ref = DatasetReference(dsout_name)
copy_activity = CopyActivity(act_name,inputs=[dsin_ref], outputs=[dsOut_ref], source=blob_source, sink=table_sink)

# Create a pipeline with the copy activity
p_name =  'copyPipeline'
params_for_pipeline = {}
p_obj = PipelineResource(activities=copy_activity, parameters=params_for_pipeline)
p = adf_client.pipelines.create_or_update(rg_name, df_name, p_name, p_obj)
print_item(p)

When I run my script, it comes back with the following error:
Traceback (most recent call last):
File "/anaconda3/lib/python3.7/site-packages/msrest/serialization.py", line 1288, in _deserialize
found_value = key_extractor(attr, attr_desc, data)
File "/anaconda3/lib/python3.7/site-packages/msrest/serialization.py", line 1078, in rest_key_case_insensitive_extractor
return attribute_key_case_insensitive_extractor(key, None, working_data)
File "/anaconda3/lib/python3.7/site-packages/msrest/serialization.py", line 1101, in attribute_key_case_insensitive_extractor
return data.get(found_key)
AttributeError: 'str' object has no attribute 'get'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/anaconda3/lib/python3.7/site-packages/msrest/serialization.py", line 571, in body
data = deserializer._deserialize(data_type, data)
File "/anaconda3/lib/python3.7/site-packages/msrest/serialization.py", line 1258, in _deserialize
self._deserialize(local_type, value)
File "/anaconda3/lib/python3.7/site-packages/msrest/serialization.py", line 1258, in _deserialize
self._deserialize(local_type, value)
File "/anaconda3/lib/python3.7/site-packages/msrest/serialization.py", line 1258, in _deserialize
self._deserialize(local_type, value)
File "/anaconda3/lib/python3.7/site-packages/msrest/serialization.py", line 1298, in _deserialize
raise_with_traceback(DeserializationError, msg, err)
File "/anaconda3/lib/python3.7/site-packages/msrest/exceptions.py", line 51, in raise_with_traceback
raise error.with_traceback(exc_traceback)
File "/anaconda3/lib/python3.7/site-packages/msrest/serialization.py", line 1288, in _deserialize
found_value = key_extractor(attr, attr_desc, data)
File "/anaconda3/lib/python3.7/site-packages/msrest/serialization.py", line 1078, in rest_key_case_insensitive_extractor
return attribute_key_case_insensitive_extractor(key, None, working_data)
File "/anaconda3/lib/python3.7/site-packages/msrest/serialization.py", line 1101, in attribute_key_case_insensitive_extractor
return data.get(found_key)
msrest.exceptions.DeserializationError: Unable to deserialize to object: type, AttributeError: 'str' object has no attribute 'get'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "create_data_factory.py", line 166, in
srctgt(i)
File "create_data_factory.py", line 157, in srctgt
p = adf_client.pipelines.create_or_update(rg_name, df_name, p_name, p_obj)
File "/anaconda3/lib/python3.7/site-packages/azure/mgmt/datafactory/operations/pipelines_operations.py", line 163, in create_or_update
body_content = self._serialize.body(pipeline, 'PipelineResource')
File "/anaconda3/lib/python3.7/site-packages/msrest/serialization.py", line 574, in body
SerializationError, "Unable to build a model: "+str(err), err)
File "/anaconda3/lib/python3.7/site-packages/msrest/exceptions.py", line 51, in raise_with_traceback
raise error.with_traceback(exc_traceback)
File "/anaconda3/lib/python3.7/site-packages/msrest/serialization.py", line 571, in body
data = deserializer._deserialize(data_type, data)
File "/anaconda3/lib/python3.7/site-packages/msrest/serialization.py", line 1258, in _deserialize
self._deserialize(local_type, value)
File "/anaconda3/lib/python3.7/site-packages/msrest/serialization.py", line 1258, in _deserialize
self._deserialize(local_type, value)
File "/anaconda3/lib/python3.7/site-packages/msrest/serialization.py", line 1258, in _deserialize
self._deserialize(local_type, value)
File "/anaconda3/lib/python3.7/site-packages/msrest/serialization.py", line 1298, in _deserialize
raise_with_traceback(DeserializationError, msg, err)
File "/anaconda3/lib/python3.7/site-packages/msrest/exceptions.py", line 51, in raise_with_traceback
raise error.with_traceback(exc_traceback)
File "/anaconda3/lib/python3.7/site-packages/msrest/serialization.py", line 1288, in _deserialize
found_value = key_extractor(attr, attr_desc, data)
File "/anaconda3/lib/python3.7/site-packages/msrest/serialization.py", line 1078, in rest_key_case_insensitive_extractor
return attribute_key_case_insensitive_extractor(key, None, working_data)
File "/anaconda3/lib/python3.7/site-packages/msrest/serialization.py", line 1101, in attribute_key_case_insensitive_extractor
return data.get(found_key)
msrest.exceptions.SerializationError: Unable to build a model: Unable to deserialize to object: type, AttributeError: 'str' object has no attribute 'get', DeserializationError: Unable to deserialize to object: type, AttributeError: 'str' object has no attribute 'get'

[jarandaf]

I get the same error when trying to create an AzureDatabricksLinkedService.

[mightyblue01]

Any update on this?

[hvermis]

Usually these errors happen when the model expects some type, for example array, and gets a string instead. From the description pasted here it is hard to tell which one is causing the issues. Look into the python code for the type you are trying to use and check the value types they expect.

[hvermis]

It should be this one:
Instead of
p_obj = PipelineResource(activities=copy_activity, parameters=params_for_pipeline)
Use
p_obj = PipelineResource(activities=[copy_activity], parameters=params_for_pipeline)

[hvermis]

@jarandaf, @mightyblue01 Your issues could be different, check your types, or paste your code.

[mightyblue01]

Thanks; in my case, it's the ExecutePipelineActivity -

act_name = "something"
pipeline_ref = "some-existing-pipeline-name"
additional_prop = {}
input_parameters = None

ExecutePipelineActivity(act_name, pipeline_ref, additional_properties=additional_prop, description="test", depends_on=None, parameters=input_parameters, wait_on_completion=True)

---

Error, irrespective of python versions - 2.7 and 3.6 :

File "/home/root-user/.local/lib/python2.7/site-packages/msrest/serialization.py", line 574, in body
SerializationError, "Unable to build a model: "+str(err), err)
File "/home/root-user/.local/lib/python2.7/site-packages/msrest/exceptions.py", line 54, in raise_with_traceback
raise error
msrest.exceptions.SerializationError: Unable to build a model: Unable to deserialize to object: type, AttributeError: 'str' object has no attribute 'get', DeserializationError: Unable to deserialize to object: type, AttributeError: 'str' object has no attribute 'get'

[hvermis]

Did you put the activity into [] in PipelinResource?

[mightyblue01]

Did you put the activity into [] in PipelinResource?

Yes -

p_name = 'MasterPipeline'
params_for_pipeline = {}
p_obj = PipelineResource(activities=[execute_pipeline_activity], parameters=params_for_pipeline)
p = adf_client.pipelines.create_or_update(rg_name, df_name, p_name, p_obj)
print_item(p)

[hvermis]

Here's the source for it:
https://github.com/Azure/azure-sdk-for-python/blob/master/azure-mgmt-datafactory/azure/mgmt/datafactory/models/execute_pipeline_activity.py

[hvermis]

BTW, only 3 fields are required, so you don't have to provide empty values;
_validation = {
'name': {'required': True},
'type': {'required': True},
'pipeline': {'required': True},
}

[jarandaf]

@hvermis is right. In my case I was passing a secret as a raw string instead of a SecureString.

[mightyblue01]

BTW, only 3 fields are required, so you don't have to provide empty values;
_validation = {
'name': {'required': True},
'type': {'required': True},
'pipeline': {'required': True},
}

Got it working !! Here are the details in case somebody goes this route and gets stuck-

1) I somehow missed to typecast the pipeline reference(blunder !!). Figured it out on a closer look.
2) Though only 3 fields are required as per the comments in source code, I still had to pass optional parameters. Haven't yet tested which one of the optional parameters is mandatory in this case. I just passed all the optional params and it worked.

act_name = "something"
pipeline_ref = PipelineReference("some-existing-pipeline-name")
additional_prop = {}
input_parameters = None

ExecutePipelineActivity(act_name, pipeline_ref, additional_properties=additional_prop, description="test", depends_on=None, parameters=input_parameters, wait_on_completion=True)

[hvermis]

It's not actually typecasting, it is another class that takes pipeline name as parameter in constructor.
Closing this thread now.

[vipinanandcpp]

@hvermis is right. In my case I was passing a secret as a raw string instead of a SecureString.

Thanks I was running into the same issue while configuring AzureDatabrickslinkedserice.