Azure-docs: SecureString typed extended properties cannot be read back

[know-where]

Code example in https://docs.microsoft.com/en-gb/azure/data-factory/transform-data-using-dotnet-custom-activity#passing-objects-and-properties shows that extended properties could be secured using SecureString type. SecureString typed extended properties cannot actually be read by the custom activity in any of the documented ways. This is a bit misleading.

---

Document details

⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

• ID: a73a1ff2-94e8-a570-ff7b-7a57acca06e6
• Version Independent ID: 9ec244a7-d939-3918-fc87-34ab1eec5293
• Content: Use custom activities in an Azure Data Factory pipeline
• Content Source: articles/data-factory/transform-data-using-dotnet-custom-activity.md
• Service: data-factory
• GitHub Login: @douglaslMS
• Microsoft Alias: douglasl

[mimckitt]

Thanks for the feedback! We are currently investigating and will update you shortly.

[douglaslMS]

Forwarded question to PM.

[douglaslMS]

@Anded, here is the information I received from the Data Factory team:

Sensitive information tagged with "SecureString" is masked out in Monitoring tab. In actual pipeline execution, the "SecureString" property is serialized as JSON within the activity.json file as plain text. For example:

"extendedProperties": {
    "connectionString": {
        "type": "SecureString",
        "value": "aSampleSecureString"
    }
}

There is nothing "secure" about this serialization. The intent is to hint to Data Factory to mask the value in the Monitoring tab.

To access this property, the customer's EXE is expected to read the activity.json file that's placed in the same folder as their .exe, deserialize the JSON, then access the JSON property (extendedProperties => [propertyName] => value).

[douglaslMS]

@anded I'll see where to add this info in the docs.

[douglaslMS]

Fixed in PR MicrosoftDocs/azure-docs-pr#54583

[douglaslMS]

please-close

[know-where]

The misleading part here is that SecureString typed property's value can be read in plain text in the custom activity. Value of SecureString typed property in activity.json is also obfuscated and therefore cannot be used in custom activity.

[douglaslMS]

@Anded The Data Factory developers insist that this is not accurate - the value of a SecureString is NOT obfuscated in activity.json. For example, if you create a custom activity that includes this property:

  "extendedProperties": {
    "secret": {
        "type": "SecureString",
        "value": "ActuallyInPlainText"
    }
  }

... and you run a test program something like this ...

static void Main(string[] args)
        {
            string activityJson;
            using (TextReader reader = File.OpenText("activity.json"))
            {
                activityJson = reader.ReadToEnd();
            }
            dynamic activity = JsonConvert.DeserializeObject(activityJson);
            Console.WriteLine(activity.typeProperties.extendedProperties.secret.value);
        }

... the value of the SecureString will be output as ActuallyInPlainText.

[douglaslMS]

in-progress

[know-where]

I am really grateful for the continued support with this issue. I have tried replicating your example in my ADF and it worked. I have also tried redeploying version of my pipeline which was having issues reading SecureString values and it also works.
This leaves me a bit confused but it is possible that I have overlooked something while making the pipeline. I had a support request opened about this as well, so I wonder if anything could've been changed in ADF backend since.
Either way, thanks, I appreciate your responses a lot, it seems everything works just fine now and documentation is up to date too!

[know-where]

I think I just figured what was actually going on. When any changes to the pipeline, even ones not related to activity with SecureString extended properties, are being published from the ADF portal, local copy of the pipeline JSON definition is used. Since the SecureString extended properties are obfuscated, in the local copy they are presented as a string of stars. So, when the changes are published, actual values of the SecureString extended properties get replaced with that string of stars.
It was confusing to figure out because I did not know if the string of stars I was seeing when reading activity.json was the actual value or if it was the obfuscated value.
Once again, thanks a lot for the help figuring this out.

[douglaslMS]

@Anded I'm pleased to hear that your pipeline is working as expected, and that you've figured out what was going on. Thanks for giving us the opportunity to clarify in the docs how SecureString works.

[douglaslMS]

please-close

[daviburg]

Could SecureString be serialized with symmetrical encryption, e.g. AES, so they are not plain text yet custom de-serializer can rehydrate the property to a .NET SecureString in a safe manner?