Google-cloud-dotnet: Google.Cloud.Translation.V2 - Translate Html

[OpenSpacesAndPlaces]

Environment details

• OS: Windows
• .NET version: 4.5.2
• Package name and version:
  Google.Cloud.Translation.V2

Steps to reproduce

Large Html snippet to translate

Code
GoogleCredential gc = GoogleCredential.FromJson(System.IO.File.ReadAllText(sDrivePath));
TranslationClient tcClient = TranslationClient.Create(gc, TranslationModel.ServiceDefault);
TranslationResult result = tcClient.TranslateHtml(sHtml, GetLangCodeCookie(), LanguageCodes.English,TranslationModel.ServiceDefault);

Error

[UriFormatException: Invalid URI: The Uri string is too long.]
   System.Uri.CreateThis(String uri, Boolean dontEscape, UriKind uriKind) +6912000
   System.Uri..ctor(String uriString) +21
   Google.Apis.Requests.RequestBuilder.UriJoin(Uri baseUri, String path) +192
   Google.Apis.Requests.RequestBuilder.BuildUri() +279
   Google.Apis.Requests.RequestBuilder.CreateRequest() +49
   Google.Apis.Requests.ClientServiceRequest`1.CreateRequest(Nullable`1 overrideGZipEnabled) +26
   Google.Apis.Requests.<ExecuteUnparsedAsync>d__33.MoveNext() +69
   System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() +26
   Google.Apis.Requests.ClientServiceRequest`1.Execute() +149
   Google.Cloud.Translation.V2.TranslationClientImpl.TranslateHtml(IEnumerable`1 htmlItems, String targetLanguage, String sourceLanguage, Nullable`1 model) +150

Related reading would suggest the library code does not switch from GET to POST for large requests:
https://stackoverflow.com/questions/9432422/google-translate-v2-cannot-hanlde-large-text-translations-from-c-sharp

[jskeet]

Hmm. I thought it did, but I'll investigate. Thanks for reporting. (Where by "it" I mean the common core Google APIs library.)

[OpenSpacesAndPlaces]

If there is an override or alternate methods I missed - that works too.
Though I wasn't seeing anything on a cursory run through.

[jskeet]

Just so I can reproduce this, how large is the file? (We have an integration test to check a 7K file - I'll try making that 10x larger just as a first pass to reproduce.)

[jskeet]

Ah, this is interesting. There is code to handle it:
https://github.com/googleapis/google-api-dotnet-client/blob/master/Src/Support/Google.Apis.Core/Http/MaxUrlLengthInterceptor.cs
... but that's an interceptor. It looks like that would only kick in later than the stack trace shows in your case. Hmm.

[jskeet]

(I've reproduced this in BuildUri, admittedly with a slightly different stack trace.)

It looks like we'd have to also handle the maximum in another place. I'll look into that tomorrow.

cc @chrisdunelm who knows more about the client library than I do.

[jskeet]

It's possible that I can work around it for this specific API using the Translate method under the hood instead of List. That may well be a simpler initial workaround, although it'll only help for Google.Cloud.Translate.V2 rather than the more general case.

[OpenSpacesAndPlaces]

Just so I can reproduce this, how large is the file?

64kb

[jskeet]

Okay, I've reproduced the problem and created a solution that works - but only up to 100K, which is the current limit of the Translation API. So we have the following "bands" of size:

• 0-46K (roughly): Works in current implementation
• 46K-100K: Works in potential new implementation, fails in current implementation
• Over 100K: Request rejected by the API

Before I start checking with the Translate team whether they're happy for me to change the implementation (there may be subtle differences between the API calls that I'm unaware of), could you confirm that this will actually be useful to you? I can imagine that if you've got text of ~64K, you may well have text of over 100K, in which case you'd need to write the code to split the requests anyway.

[jskeet]

Oh, as a side note: this will still fail for DetectLanguages, but I expect that isn't an issue for you?

[jskeet]

(Note: the 46K is the text size; it looks like the maximum URI size is ~64K - the difference is due to the number of characters in the text that would need URI escaping.)

[OpenSpacesAndPlaces]

could you confirm that this will actually be useful to you?
I purposely tested the page with the largest payload - this a real-world production usage.

Oh, as a side note: this will still fail for DetectLanguages, but I expect that isn't an issue for you?
For my current use case, this is a non-issue.

Thanks for all the help on this insofar!

[jskeet]

Right - it sounds like this is a plan if the Translation team are happy with it. Will check internally, and merge the PR if I get the go-ahead.

[jskeet]

I've pushed Google.Cloud.Translation.V2 version 1.2.0-beta01. Could you give that a try? If all is working, I'll promote the code to 1.2.0.

[OpenSpacesAndPlaces]

@jskeet Worked for me! Thanks a bunch for the fast response on this!

[jskeet]

Hooray! Thanks for the quick feedback - and for a well-reported issue. If you're comfortable using the beta for a while, I'd like to leave it a week or two as "bake time" just in case any users discover issues with the change, then promote.

If that in itself is a blocker for you, I'll ask internally how other folks feel about me promoting it to 1.2.0 early next week.

[OpenSpacesAndPlaces]

No problem.

Using the beta is fine with me.

Thanks again!