Google-cloud-dotnet: Dialogflow Client Library: FormatException: "Name does not match template: incorrect number of segments"

[paulschuetz]

We are encountering a problem with the "Google.Cloud.Dialogflow.V2.ContextName.Parse" method. The full stacktrace can be found at the end of this issue. It seems like the method is completely fine with contexts such as:
projects/{project-id}/agent/sessions/{sessionId}/contexts/{context-name}
but fails at contexts in the following format:
projects/{project-id}/agent/environments/{environment-id}/users/{user-id}/sessions/{sessionId}/contexts/{context-name}

We currently can not reproduce when a request session is in the former or in the latter format. Both behaviors are currently observed with two different google users testing an action-on-google with dialogflow which is currently in Alpha phase.

Environment details

• OS: Windows 10
• .NET version: Net Standard (2.0.3)
• Package name and version: Google.Cloud.Dialogflow.V2 (1.0.0-beta02)

Steps to reproduce

1. Build an action-on-google with dialogflow and put it into alpha
2. Try get a specific context by name such as in the folling code:
   webhookRequest.QueryResult.OutputContexts.SingleOrDefault(context => context.ContextName.ContextId == "some-context-i-need-to-read")
3. If your session contains the /environment path it will fire the exception of the title.

Stacktrace:

System.FormatException:
   at Google.Api.Gax.PathTemplate.ParseName (Google.Api.Gax, Version=2.5.0.0, Culture=neutral, PublicKeyToken=3ec5ea7f18953e47)
   at Google.Cloud.Dialogflow.V2.ContextName.Parse (Google.Cloud.Dialogflow.V2, Version=1.0.0.0, Culture=neutral, PublicKeyToken=185c282632e132a0)
   at Google.Cloud.Dialogflow.V2.Context.get_ContextName (Google.Cloud.Dialogflow.V2, Version=1.0.0.0, Culture=neutral, PublicKeyToken=185c282632e132a0)
 ...

[jskeet]

Please could you provide some code to reproduce this, and ideally the full stack trace?

[paulschuetz]

Yea sorry I fatfingered and accidentally submitted before writing the down the issue, sorry for that. Hope you can help me get around this issue.

[jskeet]

Could you log the names (as strings) before that statement? (I can't immediately check which property you want instead of ContextName, but I suspect it's just Name)

[jskeet]

If you could also show how you're parsing the web hook request, that would help - as per the API docs, you mustn't just use the Json.NET default parsing. Not sure whether that would cause this issue, but it might be relevant.

[paulschuetz]

This is how I parse the request:

private static readonly JsonParser jsonParser = new JsonParser(JsonParser.Settings.Default.WithIgnoreUnknownFields(true));

WebhookRequest request;
using (var reader = new StreamReader(Request.Body))
{
      request = jsonParser.Parse<WebhookRequest>(reader);
}

[paulschuetz]

Here are some names of the context which are causing the issues;

`projects/fonterra-smart-control-dev/agent/environments/__aog-1/users/-/sessions/ABwppHHIEqXafXeiC-nEICs7_OCozlTFQ6EYwXkQQ5EaT22f723oryBk7IzbXJJjrRJLR0-8HGTySaYIsb-Lw9NGoysrWQs0eWo/contexts/google_assistant_input_type_touch

projects/fonterra-smart-control-dev/agent/environments/__aog-1/users/-/sessions/ABwppHHIEqXafXeiC-nEICs7_OCozlTFQ6EYwXkQQ5EaT22f723oryBk7IzbXJJjrRJLR0-8HGTySaYIsb-Lw9NGoysrWQs0eWo/contexts/leavehomeintent_dialog_params_homename

projects/fonterra-smart-control-dev/agent/environments/__aog-1/users/-/sessions/ABwppHHIEqXafXeiC-nEICs7_OCozlTFQ6EYwXkQQ5EaT22f723oryBk7IzbXJJjrRJLR0-8HGTySaYIsb-Lw9NGoysrWQs0eWo/contexts/actions_capability_screen_output`

[jskeet]

Thanks. On the move right now, but will look when I'm back at a laptop

[paulschuetz]

Here is my Application Insights snapshot

As you can see _parameterSegments has a count of 3 and _segments a count of 7. Maybe it has someting to do with this.

[jskeet]

Hmm. The format is documented as:

projects/<Project ID>/agent/sessions/<Session ID>/contexts/<Context ID>

The value of projects/fonterra-smart-control-dev/agent/environments/__aog-1/users/-/sessions/ABwppHHIEqXafXeiC-nEICs7_OCozlTFQ6EYwXkQQ5EaT22f723oryBk7IzbXJJjrRJLR0-8HGTySaYIsb-Lw9NGoysrWQs0eWo/contexts/actions_capability_screen_output simply doesn't fit that, so this looks like a server-side issue.

I'll file an internal bug and let you know what I hear.

[jskeet]

I have an update for you: it looks like this is a potentially expected format in v2beta1. Do you have beta functionality turned on in your project? If so, and if you don't need that functionality, you might want to try turning it off.

I have another workaround for you on the way, but it'll take a bit longer.

[jskeet]

Okay, here's the workaround. Basically it's another resource name that does match the format you're seeing.

First, the usage:

var context = sampleRequest.QueryResult.OutputContexts
    .SingleOrDefault(c => EnvironmentContextName.Parse(c.Name).ContextId == "leavehomeintent_dialog_params_homename");

Then the code:

// Copyright 2019 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

// Generated code. DO NOT EDIT!

using gax = Google.Api.Gax;
using gaxres = Google.Api.Gax.ResourceNames;
using sys = System;
using linq = System.Linq;

namespace Google.Cloud.Dialogflow.V2
{
    /// <summary>
    /// Resource name for the 'environment_context' resource.
    /// </summary>
    public sealed partial class EnvironmentContextName : gax::IResourceName, sys::IEquatable<EnvironmentContextName>
    {
        private static readonly gax::PathTemplate s_template = new gax::PathTemplate("projects/{project}/agent/environments/{environment}/users/{user}/sessions/{session}/contexts/{context}");

        /// <summary>
        /// Parses the given environment_context resource name in string form into a new
        /// <see cref="EnvironmentContextName"/> instance.
        /// </summary>
        /// <param name="environmentContextName">The environment_context resource name in string form. Must not be <c>null</c>.</param>
        /// <returns>The parsed <see cref="EnvironmentContextName"/> if successful.</returns>
        public static EnvironmentContextName Parse(string environmentContextName)
        {
            gax::GaxPreconditions.CheckNotNull(environmentContextName, nameof(environmentContextName));
            gax::TemplatedResourceName resourceName = s_template.ParseName(environmentContextName);
            return new EnvironmentContextName(resourceName[0], resourceName[1], resourceName[2], resourceName[3], resourceName[4]);
        }

        /// <summary>
        /// Tries to parse the given environment_context resource name in string form into a new
        /// <see cref="EnvironmentContextName"/> instance.
        /// </summary>
        /// <remarks>
        /// This method still throws <see cref="sys::ArgumentNullException"/> if <paramref name="environmentContextName"/> is null,
        /// as this would usually indicate a programming error rather than a data error.
        /// </remarks>
        /// <param name="environmentContextName">The environment_context resource name in string form. Must not be <c>null</c>.</param>
        /// <param name="result">When this method returns, the parsed <see cref="EnvironmentContextName"/>,
        /// or <c>null</c> if parsing fails.</param>
        /// <returns><c>true</c> if the name was parsed successfully; <c>false</c> otherwise.</returns>
        public static bool TryParse(string environmentContextName, out EnvironmentContextName result)
        {
            gax::GaxPreconditions.CheckNotNull(environmentContextName, nameof(environmentContextName));
            gax::TemplatedResourceName resourceName;
            if (s_template.TryParseName(environmentContextName, out resourceName))
            {
                result = new EnvironmentContextName(resourceName[0], resourceName[1], resourceName[2], resourceName[3], resourceName[4]);
                return true;
            }
            else
            {
                result = null;
                return false;
            }
        }

        /// <summary>
        /// Constructs a new instance of the <see cref="EnvironmentContextName"/> resource name class
        /// from its component parts.
        /// </summary>
        /// <param name="projectId">The project ID. Must not be <c>null</c>.</param>
        /// <param name="environmentId">The environment ID. Must not be <c>null</c>.</param>
        /// <param name="userId">The user ID. Must not be <c>null</c>.</param>
        /// <param name="sessionId">The session ID. Must not be <c>null</c>.</param>
        /// <param name="contextId">The context ID. Must not be <c>null</c>.</param>
        public EnvironmentContextName(string projectId, string environmentId, string userId, string sessionId, string contextId)
        {
            ProjectId = gax::GaxPreconditions.CheckNotNull(projectId, nameof(projectId));
            EnvironmentId = gax::GaxPreconditions.CheckNotNull(environmentId, nameof(environmentId));
            UserId = gax::GaxPreconditions.CheckNotNull(userId, nameof(userId));
            SessionId = gax::GaxPreconditions.CheckNotNull(sessionId, nameof(sessionId));
            ContextId = gax::GaxPreconditions.CheckNotNull(contextId, nameof(contextId));
        }

        /// <summary>
        /// The project ID. Never <c>null</c>.
        /// </summary>
        public string ProjectId { get; }

        /// <summary>
        /// The environment ID. Never <c>null</c>.
        /// </summary>
        public string EnvironmentId { get; }

        /// <summary>
        /// The user ID. Never <c>null</c>.
        /// </summary>
        public string UserId { get; }

        /// <summary>
        /// The session ID. Never <c>null</c>.
        /// </summary>
        public string SessionId { get; }

        /// <summary>
        /// The context ID. Never <c>null</c>.
        /// </summary>
        public string ContextId { get; }

        /// <inheritdoc />
        public gax::ResourceNameKind Kind => gax::ResourceNameKind.Simple;

        /// <inheritdoc />
        public override string ToString() => s_template.Expand(ProjectId, EnvironmentId, UserId, SessionId, ContextId);

        /// <inheritdoc />
        public override int GetHashCode() => ToString().GetHashCode();

        /// <inheritdoc />
        public override bool Equals(object obj) => Equals(obj as EnvironmentContextName);

        /// <inheritdoc />
        public bool Equals(EnvironmentContextName other) => ToString() == other?.ToString();

        /// <inheritdoc />
        public static bool operator ==(EnvironmentContextName a, EnvironmentContextName b) => ReferenceEquals(a, b) || (a?.Equals(b) ?? false);

        /// <inheritdoc />
        public static bool operator !=(EnvironmentContextName a, EnvironmentContextName b) => !(a == b);
    }
}

It's definitely not good that you need to do this, but hopefully it'll unblock you.

[paulschuetz]

Thank you very much for taking time for my issue. I am pretty sure that I do not have beta features enabled for the project but I will check it on monday and give some feedback. If that is the case I will try out your solution and report if everything works as expected.
Have a great weekend!

[paulschuetz]

So we do not have any beta features enabled. I do not know why some user requests contain the and tags and others not.
Nevertheless your workaround works :+1:

[jskeet]

Great - I'm really glad the workaround works. It may be a while before we know what's going to happen long-term, but at least being aware of it I can help try to guide things in a compatible way :)

[jskeet]

Will review this again tomorrow, to see if there's been any change here.

[jskeet]

Okay, I've spoken with folks internally, and developers should only see this if they're using the automatic Assistant integration, so at least the scope is reasonably narrow. But we're looking into how the other format could be exposed in the V2 API to make it easier to handle without the extra code.

[jskeet]

This is effectively fixed by the work done in the next-major-version branch. When the new major version comes out, it will support multiple patterns in a single resource name. I don't think we need to do anything else in this repo, so I'll close this now.

[DJ4ddi]

Shouldn't this be fixed with 2.0.0? Because I'm seeing NullReferenceExceptions when accessing Context.ContextName.ContextId for requests coming from the Actions on Google simulator. Context name looks like this:
projects/((PROJECT))/agent/environments/__aog-14/users/-/sessions/((SESSION))/contexts/((CONTEXTNAME))

The workaround also no longer compiles for the new release.

[jskeet]

@DJ4ddi: When you say "I'm seeing NullReferenceExceptions when accessing Context.ContextName.ContextId" do you mean that just evaluating that property throws an exception, or that the property returns null?

Could you provide a reasonably complete example? It's hard to tell exactly what's going on just from the expression in your comment. I'm aware that there are definitely issues with the 2.0.0 resources names - which will unfortunately lead to a breaking change, and a bump to 3.0.0 quite soon. Hopefully that'll be the last breaking change in Dialogflow for a while. I'd like to understand exactly what you're seeing first though, to make sure that it will be fixed by the change.

[DJ4ddi]

Sorry, it really was a poor description on my part. The exception is from my code, not yours. It's caused by the ContextName being unparsed with ContextId, SessionId and ProjectId all null and IsKnownPattern false. The UnparsedResource name has the format mentioned above.

I don't think an example is needed, I'm basically just using the same approach as the original poster of this issue (repro step 2).

[jskeet]

Right - I understand it now. That's because currently the resource name patterns for contexts (as specified in the API) don't match that name.

This is one of the things that will be fixed by the upcoming change. We're just trying to work out which languages the change will impact and in what way, but I'm reasonably hopeful it will be done some time next week - at which point I'll immediately do a 3.0.0-beta01 release.
Apologies for the brokenness :( If you want a workaround in the meantime, I could include "what the new ContextName class will look like" and you could have a version of that in your code, and temporarily write code to parse the name using that when you need to.

[DJ4ddi]

I wrote a hack for my specific case, it's not great but it'll get me through the week:

internal static string GetContextId(this Context context)
{
  if (!context.ContextName.IsKnownPattern || string.IsNullOrEmpty(context.ContextName.ContextId))
  {
    return context.ContextName.UnparsedResource.ToString().Split('/').Last();
  }
  return context.ContextName.ContextId;
}

[jskeet]

Great - glad you've got something for now. I've created #4665 to handle this "new and slightly different but with the same cause" breakage. I suggest you subscribe to that issue, and I'll close it when I publish 3.0.0-beta01.