Azure-activedirectory-identitymodel-extensions-for-dotnet: JwtSecurityToken.EncodedPayload is having problems with my Deserialized JSON to plain object

[ghstahl]

This is a "It used to work" issue. I have attached the code that shows the issue.
I want to be able to Deserialize a JSON into an object that I can add correctly to a JwtPayload and get properly encoded by the JwtSecurityToken.

I can add a object to the payload.

var custom = new Custom{}
payload.Add("custom", custom);

I can add a dynamic to the payload.

dynamic ccDynamic = new
            {
                a="b",
                b=2,
                id_token_signing_alg_values_supported = new List<string>{
                        "RS256"
                },
                d = new { 
                  name  = "bob"
                }
            };
payload.Add("ccDynamic", ccDynamic);

I can add an expando to the payload.

var converter = new ExpandoObjectConverter();
dynamic expando = JsonConvert.DeserializeObject<ExpandoObject>(json, converter);
payload.Add("expando", expando);

And these get encoded just fine in the final JWT.

I used to be able to deserialize a json to a dynamic or object and add that.

   var json = JsonConvert.SerializeObject(custom);
    var jObj = JsonConvert.DeserializeObject(json);
    dynamic jDynamic = JsonConvert.DeserializeObject<dynamic>(json);
    payload.Add("object", jObj);
    payload.Add("jDynamic", jDynamic);

However those Deserialized object now turn into the following.
JWT

{
  "nbf": 1592932718,
  "exp": 1592934518,
  "iat": 1592932718,
  "iss": "https://issuer.com/token",
  "aud": "my-audience",
  "object": {
    "Name": [],
    "Numbers": [
      [],
      [],
      []
    ],
    "Strings": [
      [],
      [],
      []
    ],
    "SomeObjects": [
      {
        "Name": []
      },
      {
        "Name": []
      }
    ],
    "SomeObject": {
      "Name": []
    }
  },
  "jDynamic": {
    "Name": [],
    "Numbers": [
      [],
      [],
      []
    ],
    "Strings": [
      [],
      [],
      []
    ],
    "SomeObjects": [
      {
        "Name": []
      },
      {
        "Name": []
      }
    ],
    "SomeObject": {
      "Name": []
    }
  },
  "ccDynamic": {
    "a": "b",
    "b": 2,
    "id_token_signing_alg_values_supported": [
      "RS256"
    ],
    "d": {
      "name": "bob"
    }
  },
  "custom": {
    "Name": "Bugs Bunny",
    "Numbers": [
      1,
      2,
      3
    ],
    "Strings": [
      "a",
      "bb",
      "ccc"
    ],
    "SomeObjects": [
      {
        "Name": "Daisy Duck"
      },
      {
        "Name": "Porky Pig"
      }
    ],
    "SomeObject": {
      "Name": "Daffy Duck"
    }
  },
  "expando": {
    "Name": "Bugs Bunny",
    "Numbers": [
      1,
      2,
      3
    ],
    "Strings": [
      "a",
      "bb",
      "ccc"
    ],
    "SomeObjects": [
      {
        "Name": "Daisy Duck"
      },
      {
        "Name": "Porky Pig"
      }
    ],
    "SomeObject": {
      "Name": "Daffy Duck"
    }
  }
}

CustomPayloadJwt.zip

[ghstahl]

btw: I first saw this in the following code.

var jsonValue = Newtonsoft.Json.Linq.JRaw.Parse(json);
payload.Add("jsonValue ", jsonValue );

The jsonValue here looks exactly like

 var jObj = JsonConvert.DeserializeObject(json);

the following in field code looks like it might be effected.
IdentityServer4 TokenExtension

Its doing a JRaw and puts that into the payload.

[brentschmaltz]

@ghstahl sorry for the hassle.
by "used to work" i assume you mean all the empty objects didn't exist.

[ghstahl]

the

"jDynamic": { currently bad}

looked like.

"expando": { all good here....}

Yes the payload should correctly parse that jDynamic and jObj object I pass in. I don't know if that happens in JwtPayload or JwtSecurityToken.

[brentschmaltz]

@ghstahl do you know a version that this was working in?

[ghstahl]

Works with this version

<PackageReference Include="System.IdentityModel.Tokens.Jwt" Version="5.6.0" />

If you take my attached code and change it to this its good to go.
I checked all the 6.x.x versions and they all exhibit the problem.
6.6.0,6.5.1,6.5.0

[brentschmaltz]

@ghstahl that helps thanks.
I think I have an idea now.

[brentschmaltz]

@ghstahl we have an internal version of Newtonsoft, i haven't found the exact issue, but I believe the two are out of sync.

To test this out, I added the following code and this seems to have things in sync and everything seems right.
You could use this workaround while we figure out what to do.

JsonExtensions.Serializer = JsonConvert.SerializeObject;
JsonExtensions.Deserializer = JsonConvert.DeserializeObject;

eyJhbGciOiJFUzI1NiIsImtpZCI6IjEyMzQiLCJ0eXAiOiJKV1QifQ.eyJuYmYiOjE1OTI5NTY5ODMsImV4cCI6MTU5Mjk1ODc4MywiaWF0IjoxNTkyOTU2OTgzLCJpc3MiOiJodHRwczovL2lzc3Vlci5jb20vdG9rZW4iLCJhdWQiOiJteS1hdWRpZW5jZSIsIm9iamVjdCI6eyJOYW1lIjoiQnVncyBCdW5ueSIsIk51bWJlcnMiOlsxLDIsM10sIlN0cmluZ3MiOlsiYSIsImJiIiwiY2NjIl0sIlNvbWVPYmplY3RzIjpbeyJOYW1lIjoiRGFpc3kgRHVjayJ9LHsiTmFtZSI6IlBvcmt5IFBpZyJ9XSwiU29tZU9iamVjdCI6eyJOYW1lIjoiRGFmZnkgRHVjayJ9fSwiakR5bmFtaWMiOnsiTmFtZSI6IkJ1Z3MgQnVubnkiLCJOdW1iZXJzIjpbMSwyLDNdLCJTdHJpbmdzIjpbImEiLCJiYiIsImNjYyJdLCJTb21lT2JqZWN0cyI6W3siTmFtZSI6IkRhaXN5IER1Y2sifSx7Ik5hbWUiOiJQb3JreSBQaWcifV0sIlNvbWVPYmplY3QiOnsiTmFtZSI6IkRhZmZ5IER1Y2sifX0sImNjRHluYW1pYyI6eyJhIjoiYiIsImIiOjIsImlkX3Rva2VuX3NpZ25pbmdfYWxnX3ZhbHVlc19zdXBwb3J0ZWQiOlsiUlMyNTYiXSwiZCI6eyJuYW1lIjoiYm9iIn19LCJjdXN0b20iOnsiTmFtZSI6IkJ1Z3MgQnVubnkiLCJOdW1iZXJzIjpbMSwyLDNdLCJTdHJpbmdzIjpbImEiLCJiYiIsImNjYyJdLCJTb21lT2JqZWN0cyI6W3siTmFtZSI6IkRhaXN5IER1Y2sifSx7Ik5hbWUiOiJQb3JreSBQaWcifV0sIlNvbWVPYmplY3QiOnsiTmFtZSI6IkRhZmZ5IER1Y2sifX0sImV4cGFuZG8iOnsiTmFtZSI6IkJ1Z3MgQnVubnkiLCJOdW1iZXJzIjpbMSwyLDNdLCJTdHJpbmdzIjpbImEiLCJiYiIsImNjYyJdLCJTb21lT2JqZWN0cyI6W3siTmFtZSI6IkRhaXN5IER1Y2sifSx7Ik5hbWUiOiJQb3JreSBQaWcifV0sIlNvbWVPYmplY3QiOnsiTmFtZSI6IkRhZmZ5IER1Y2sifX19.BLAH

{
"nbf": 1592956983,
"exp": 1592958783,
"iat": 1592956983,
"iss": "https://issuer.com/token",
"aud": "my-audience",
"object": {
"Name": "Bugs Bunny",
"Numbers": [
1,
2,
3
],
"Strings": [
"a",
"bb",
"ccc"
],
"SomeObjects": [
{
"Name": "Daisy Duck"
},
{
"Name": "Porky Pig"
}
],
"SomeObject": {
"Name": "Daffy Duck"
}
},
"jDynamic": {
"Name": "Bugs Bunny",
"Numbers": [
1,
2,
3
],
"Strings": [
"a",
"bb",
"ccc"
],
"SomeObjects": [
{
"Name": "Daisy Duck"
},
{
"Name": "Porky Pig"
}
],
"SomeObject": {
"Name": "Daffy Duck"
}
},
"ccDynamic": {
"a": "b",
"b": 2,
"id_token_signing_alg_values_supported": [
"RS256"
],
"d": {
"name": "bob"
}
},
"custom": {
"Name": "Bugs Bunny",
"Numbers": [
1,
2,
3
],
"Strings": [
"a",
"bb",
"ccc"
],
"SomeObjects": [
{
"Name": "Daisy Duck"
},
{
"Name": "Porky Pig"
}
],
"SomeObject": {
"Name": "Daffy Duck"
}
},
"expando": {
"Name": "Bugs Bunny",
"Numbers": [
1,
2,
3
],
"Strings": [
"a",
"bb",
"ccc"
],
"SomeObjects": [
{
"Name": "Daisy Duck"
},
{
"Name": "Porky Pig"
}
],
"SomeObject": {
"Name": "Daffy Duck"
}
}
}

[brentschmaltz]

@ghstahl this seems to be a broader issue.
Closing this and linking to #1475