Orchardcore: Localize content items (LocalizationPart)

[Skrypt]

I'm opening this issue for tracking this feature. We need to implement it. Discussion is opened for ideas/recommendations.

[Skrypt]

The Localization Middleware has currently 3 default RequestCultureProviders that can be used to set the admin UI in different cultures. The order of priority of these providers are : Url, Cookie, Browser culture.
They are preset in that order but it can be changed per need. Though we haven't changed it since the order is quite logical and standard for common use. For the LocalizationPart the idea will be to create a CustomRequestCultureProvider that will display the current Content Item in the correct culture even if the admin UI is set to a specific culture. You could for example browse french content items even if your admin UI is set to english.
So, I think the idea will be to have a second url param for displaying content items in a specific culture in the admin UI. It will work almost like the RequestUrlCultureProvider but instead of using culture-ui or culture as a url param it could use content-culture and content-culture-ui for example.

[sebastienros]

Each localized content item has its own url. Two content items should not be rendered on the same url, even if they represent the same thing on different languages.

Or change my mind.

[psijkof]

How would the specified culture help in finding the right content (items)?

[Skrypt]

Right now the content item id is part of the url.

https://localhost:44300/Admin/Contents/ContentItems/471k13jfrc7p37etw15bcyk4c6/Edit?returnUrl=%2FAdmin%2FContents%2FContentItems

If we add a LocalizationPart to this content type then this content item will now also have the ability to save it's culture in the JSON data.

Though if we want to translate this content item then we need to reload that form within a different context. Either we do it by using a content-culture url param so that it will try to load an empty form or we just duplicate the content of the current form to the other new form by specifying which culture we wanted it to be with that content-culture url param. Remember that in Wordpress when you hit the translate button it will reload the page with the form controls prefilled with the culture you we're previously in. So that content-culture param is only to detect if the the content item has already a specified content item in that specified culture.

How would the specified culture help in finding the right content (items)?

The default culture should dictate which content items are the master ones or each translated content items should have a reference to their master content item ID. That way we would avoid issues with people that would change their default culture later on.

So the CustomRequestCultureProvider here would just try to fetch the right content item in the specified culture we would have required through the content-culture url param. It would require to have both content item id and content-culture url params. And it wouldn't change the culture of the current thread at all.

Maybe I'm wrong about the CustomRequestCultureProvider object since it's meant to change the thread culture but could be used for what I described above still "I think".

[sebastienros]

When you switch languages using a drop down for instance, the action should be to lookup for the other content item in the specified language, and redirect to it if found.

[Skrypt]

I'm not sure if we then require to use a CustomRequestCultureProvider for that matter. It's just a controller action.

[sebastienros]

Providers should have nothing to do with how we find localized items.

[Skrypt]

You are right. I think we had talked about using a CustomRequestCultureProvider and I got confused by not rethinking about it twice 😄 Then don't read my first post. 😉

[Skrypt]

Though I recommend strongly that we save the master content item id this time since we had a lot of issue in 01 when people we're switching their default culture, adding, removing supported cultures. Or let me know if you have a better idea for this.

[sebastienros]

c.f. https://github.com/OrchardCMS/Orchard/issues/7352

[Skrypt]

Content items culture should not be versionable.

[sebastienros]

Writing down how it should be implemented while it's fresh in my mind:

A LocalizationPart is added to content types that can be translated. It will provide a custom UI element in the editor to navigate between translations of the same content item using a common identifier called __Localization Set__ which is represented by a unique identifier share across all content items of the same set. Each LocalizationPart has a unique (Culture, LocalizationSet) tuple as a single translation
of the same culture can exist in a localization set. The localization set identifier can generated the first time a content item is translated.

A setting is added to all fields so they can be marked as "Culture Neutral", meaning their value cannot vary by culture. In practice when a content item is changed, the values of these fields are cloned on other versions (draft/published) of the same localization set. We synchronize them.

Content handlers should have a Localizing event such that each part could define how its content should be localized. Some parts might need their content to be made unique (autoroute part) or copied (title) or translated automatically, or picked from other translated content (content picker).

All these services, part and events should be in a Localization module, a Core and an Abstractions project. The ContentManagement module should not have to reference it. Any module which needs to handle localization in a specific way will reference the Abstractions project and implement custom handlers.

[jan-chvojka]

It sounds reasonable and it covers what I require from the Localization module. Would you consider to add at least LocalizationPart (with just these two properties Culture, LocalizationSet) to the Core now? Let's say that I could try to create a pull request, would it be accepted or you don't won't to add it until the concept is approved?

[jptissot]

I started working on Content Cloning last night, which is the first step in starting work on the Localization feature. Hopefully I get some time to start working on the part soon.

[sebastienros]

Since Clone is implemented in the PR, I will change my previous design to actually have the Localization service to call Clone and then invoke Localizing events such that parts can change what was cloned if necessary.

[jan-chvojka]

@Skrypt @sebastienros I guess I can help with Localization module since I have basicaly implemented it for my current project(s). It covers:

• LocalizationPart with Culture, LocalizationSet
• LocalizationIndex with ContentItemId, Culture, LocalizationSet (latest and published content items)
• LocalizationPartDriver covers Editor
• LocalizationPartHandler sets LocalizationSet on Creating / Updating
• LocalizationPartIndexHandler for search/filtering
• Liquid filter for culture_url_prefix (adds culture with slash "/en" or nothing for default culture and is intended for autoroute)
• RequestCultureProvider
• LocalizationService with Translate and get localized Content Items stuff
• AdminController with POST method for translation itself
• Simple Culture Picker for frontend

Or is there someone who is already working on it?

[jptissot]

I started looking into it yesterday but if you already have all this I'd say feel free to make the pr.

[sebastienros]

@jan-chvojka This sounds amazing!

• Are you using the Clone methods from the ContentManager we added in the localization branch?
• Do you have events for letting handlers react to a content item being localized?
• Do you already support culture neutral fields?

I assume the RequestCultureProvider is to change the current culture based on the content item that is requested?

I would suggest to create a "Draft" PR (a new feature on GitHub) so we can take a look, and if you could also give us a demo that would be great. From there we'll be able to help you on this.

[jan-chvojka]

• I'm using my own cloning method but it should not be a problem to switch it for one from localization branch.
• No, but I will try to add them.
• No.
• My RequestCultureProvider sets the culture based on culture prefix in url. It would be better to set it from current content item that is requested and this would make culture url prefix optional. I will look into it.

I'm going to cut out my module from our solution and prepare a "Draft" PR over the weekend. I've never done it before but I will try.

[sebastienros]

For the culture provider, add some code in the Route handler when the DisplayrouteMetadata is checked, because a content item can be rendered by different controllers. Then you could set a Feature in the HttpContext.Features collection, that other middleware and services will use (like your custom CultureProvider).

[psijkof]

I can't wait to test this. Having localization for content items will help me push OC as the solution for a CMS.

[xperiandri]

Me too

[Skrypt]

@jan-chvojka Have you shared a PR yet so that we can see your progress on this? It just feels like people are waiting and that progress stopped on this feature.

[sebastienros]

@jan-chvojka If you can't create a PR, would you mind sharing your raw module at least so we can build something on top of that?

[jptissot]

I started working on this again as I might need it in my projects. Will post an update when I have something to show.

[jptissot]

WIP Branch here: https://github.com/jptissot/OrchardCore/tree/localization/src/OrchardCore.Modules/OrchardCore.ContentLocalization

[agriffard]

I gave a quick try to the Localization branch and I had an error after a few changes:

• Enable Content Localization.
• Add 'fr-fr' and 'en' cultures.
• Set 'fr-fr' as the default one.
• Add Localization part on the Page content type.
• Create page 'A propos'.
• Create the corresponding page in 'en' culture with the Tilte 'About'.
• Edit the page and change the permalink.
  => Internal Server Error :

InvalidOperationException: Sequence contains more than one matching element
AspNetCore._Areas_OrchardCore_ContentLocalization_Views_LocalizationPart_Edit.ExecuteAsync() in LocalizationPart.Edit.cshtml
 var translation = translations.SingleOrDefault(t => t.Culture == culture.ToLowerInvariant());

[plompd]

I'm going to cut out my module from our solution and prepare a "Draft" PR over the weekend. I've never done it before but I will try.

@jan-chvojka did you had some time to look into this? Also very eager to start with a new Orchard Core project, but waiting on the Localization options.

[jptissot]

I'm going to continue to work on this this weekend or next week.

[plompd]

@jptissot OK, nice. You work on the same project (PR) as @jan-chvojka? He seemed to have a lot of work already done. Or are you the same person :)

[jptissot]

I can collaborate with anyone that wants to on this. I am focusing on making the LocalizationPart with all the handlers, events and index for now. Maybe If I can get it completed and merged into the localization branch we could split the rest of the work (culture picker, etc)

I have my own version right now https://github.com/jptissot/OrchardCore/tree/localization/src/OrchardCore.Modules/OrchardCore.ContentLocalization which does not compile at the moment. I will probably work on it this weekend.

[Skrypt]

@zimplerconsulting @jan-chvojka didn't gave us news or shared a branch/PR yet so we can't rely on him for having this feature working. Also, we noticed that some things we're missing or not in par with what we want in things he did. I think it's better that @jptissot work on this ; so that we have at least what we are looking for in this feature since @sebastienros defined with @jptissot in a meeting (that I attended too) about what it should do and how. I think that I will also take a look at it once we will have a first iteration working on his branch. I can help my fellow french Canadian here on this (when I will get time) since I did AutoroutePart localization feature in 01 and I also implemented the Localization global settings in OC.

[sebastienros]

Somehow I feel better knowing that two people are working concurrently on the same thing, as knowledge of the internals will be shared by more contributors, and we should get much better feedback on the implementation.