Semanticmediawiki: Multilingual Semantic MediaWiki

[mwjames]

[0] pointed towards Multilingual Semantic MediaWiki (a.k.a MSMW) as reference on this topic.

@nemobis @kghbln Not sure SMW-core actually needs to do something here.

[0] https://www.semantic-mediawiki.org/w/index.php?title=Talk:AIFB_Web_Portal&offset=20141026172749&lqt_mustshow=1491#Translations_1491

[nemobis]

Hah, I didn't expect you to notice that edit ;) I was just leaving and
checking some links around.

mwjames, 26/10/2014 23:15:

Not sure SMW-core actually needs to do something here.

I'm not sure either of what needs doing, I asked updates at
https://bugzilla.wikimedia.org/show_bug.cgi?id=47736 earlier today.

[mwjames]

SIL should solve most of the issues related to language related content in connection with SMW as it allows to divide content into separate pages and therefore make it easier for an author to concentrate on content with the language of his/her choice (of course, one can still use the Translate extension if necessary). Just add INTERLANGUAGELINK to a selected page together with the language it represents and some reference (could be the page title or something else) that would allow to link content of similar nature to each other.

@kghbln I tried to cover most use cases (page transclusion, template inclusion etc.) but maybe I missed something. INTERLANGUAGELIST is provided to give a user a possibility to generate a LanguageBox within a wikipage.

Supports language filtering in Special:Search based on the known interlanguage annotations.

[hypergrove]

Managing language-specific property values is problematic in a semantic wiki that has pages in more than one language; there is no good support in SMW today, a situation that should be remedied. MediaWiki provides Special:MyLanguage/somepagename to display (a subpage of) somepagename in a manner that matches the user's language (per personal preferences) eg Main Page/es would be the Spanish version of Main Page in a default English wiki -- this is fine for content-only wikis, not so great for semantic wikis. There are two strategies I think of:

1. Place English-relevant annotations on the English page and Spanish-relevant annotations on its Spanish page
2. Place all annotations on the English (ie default) page, flagging language-specific values within the annotations themselves

SMW Ontologies (SMWO) takes the latter approach due primarily to the belief it is quite alot easier to input, translate, query and maintain annotations expressed in multiple languages if they are on the same default-language page. Thus pieces of a 'multilingual annotation strategy' are as follows.

1 property-specific templates (or parser functions) that store language-tagged values in subobjects named per the property
2 #set and in-line annotation syntax modified to let users specify a language relevant to an annotation
property-specific templates (or parser functions) that output language-tagged values
3 #ask and #show modified to give users the ability to specify a base retrieval language, with others as backups
4 #ask and #show modified to optionally return Special:MyLanguage invocations; to count attribute values sensibly; etc
5 Special:Browse modified to display language-qualified values for a property
6 page-specific templates to format a page's display based on the page's display language
7 SemForms extension modified for multi-lingual fields is a possibility
8 Special:PageAssembly to display a page's "structure" among its transclusions, language subpages & templates
9 Translate extension modified to create {{semantic}} template work-units

[nemobis]

You can now read a preprint of "Multilingual Semantic MediaWiki for
Finno-Ugric dictionaries" at
http://www.computing.dcu.ie/~tpirinen/2015-fiwclul-proceedings.pdf p. 85

[hypergrove]

thanks for posting this link -- oh, page 75 actually.

"What caused most headaches for us were the limitations of the semantic structure. The basic tool we have is a subject-property-value triplet, where the subject is always implicitly a page in the wiki. Properties we can define freely, but the values cannot have inner structure; in other terms, SMW can only store 2D data, not 3D or N-dimensional data."

Good problem statement. The prototype Niklas developed does not seem to leverage subobjects which I'd say is a decent solution for N-dimensional property values.

[mwjames]

Posted in/Copied from https://github.com/SemanticMediaWiki/SemanticMediaWiki/issues/1002#issue-73363571

Has anyone tried setting up a SemanticMediaWiki with Translations on a single server install?

Since I could find an answer online, maybe one of you has some experience.

Translation bundle

• Set up Translation Bundle on SMW single server/single database install

don't really know if it is possible.

Semantic property

Another approach, using 2 semantic properties per page (names can be improved):

• "Has language"
• "Has neutral article" or "Has original article"

Define a neutral language, which most know, example: english
Pages in the neutral language: no need to have these properties.

Pages in non-neutral languages: should define the 2 properties, has language and has neutral/original article

What do you guys/gals think is the best approach?

Since SMWCon is tomorrow, maybe this is a topic, someone has had experience with :)

Best regards

Roland Uuesoo

[mwjames]

My notes from https://phabricator.wikimedia.org/T93906#1149966.

We will probably need a different approach which explicitly ties a language dependent label to its property key (a user-property created in a specific language) where the label is being fed [0] as alias to a property key (the property-key stays as unique identifier).

We have to introduce a monolingual texttype which would allow to annotate a property with different labels in different languages such as [[Has property label::en:Foo]] [[Has property label::ja:テスト]] etc. but this happens entirely within the property namespace (so a query can be created to lookup the correct label for a property using PropertyLabelLanguageFallbackLookup which is injected into the PropertyLabelFinder ).

Replacing a text (property key) within a query based on a language key (or {{int}}) will not work as the identifier (property key/label) of that value refers to the key. The text presentation of a property can only be replaced after a query result has been generated where [1] provides the correct textual context (the same goes for a property label text outside of a query context in Special:Browse etc.).

Once a property key (user-defined or pre-defined) is defined, internally it needs to refer to that entity while a language dependent label is only acting as an alias to that property key.

A redirect is different from an alias as one property refers to another property under the sameAs notation (with both properties referring to the same thing as predicate) where a language dependent label only refers to a textual representation of that property key.

[hypergrove]

I am giving a presentation at SMWCon this week that deals in part with this topic.
I have installed Translation with SMW no problems!

[kghbln]

I think this is the presention you @hypergrove are referring to. Well I am keen to learn from this. Yeah, installing MLEB and SMW on one instance is not a problem. It's about making good use of it in their interaction.

[nemobis]

• Set up Translation Bundle on SMW
  [...] I have installed Translation

Do these refer to the MediaWiki Language Extensions Bundle (MLEB) and
the Translate extension specifically, or something else with confusingly
similar names?

[kghbln]

Do these refer to the MediaWiki Language Extensions Bundle (MLEB) and
the Translate extension specifically, or something else with confusingly
similar names?

@nemobis I very much think so. At least I am not aware of a bundle similar to this. MLEB is namewise more than most people can handle I suppose.

[hypergrove]

Hi - I posted a PDF of the presentation (linked above) I delivered at SMWCon where I spoke of changes I prefer to support multilingual requirements. Would appreciate your feedback on any of the ideas which I presented there.

[nemobis]

I got mostly lost at some point of that PDF but some small pieces of it are pretty clear and should be tracked in their own subreports:

• respecting $wgCapitalLinks,
• allowing Special:MyLanguage links,
• a standard way to store a value in a certain language and to present query results in a certain language (ideally with fallback handling).
  Where the third point is vastly more difficult but can be split in multiple pieces.

The parts about negative filtering and subPropertyOf seem beyond a matter of multilingualism.

[hypergrove]

sure just 14 of the slides are dealing with localization.
a. I sent an email to the list about wgCapitalLinks, no responses yet though
b. special:MyLanguage link is pretty straightforward - do you want me to add those "link=" options?
c. modify {{#set: propname@langcode=value}} I think is the first/next step - is there agreement that a subobject should be used to store the language-tagged values?

[mwjames]

c. modify {{#set: propname@langcode=value}} I think is the first/next step - is there agreement that a subobject should be used to store the language-tagged values?

Using a hidden encoding (propname@langcode) in #set/#subobject is the wrong approach. It needs a dedicated monolingual datatype.

[hypergrove]

Dont understand your view - nothing is hidden at all, why do you say that?

[mwjames]

Dont understand your view - nothing is hidden at all, why do you say that?

Because something like propname@langcode has no meaning in terms how the data are stored nor how they are represented as triple.

You assign an internal encoding propname@langcode to the parser function to work trough an annotation that is not part of a parser functions responsibility.

[hypergrove]

Let's address one issue at a time. {{#set:pname@lang=value}} is my proposal. What is yours?

[mwjames]

A language code is assigned to a content entity (mostly a blob value) and for such it represents an auxiliary information to a value (internally it might be stored as either `en:blob' or as subobject but not as a subobject you created via #subobject (might be similar in a way the record type is stored).

As I said, this needs to be addressed via a datavalue (of either a blob or __sob datatype) not through a parser function.

[hypergrove]

Please address the impact of language specification on the existing {{#set:}} parser function -- your responses seem to be about storage of language-tagged values, I wish to focus on impacts of this functionality on existing parser functions.

[hypergrove]

If you are suggesting that a given property X is used for a French value, and only a French value, then I categorically say that idea is a non-starter for me. Such design would be a complete departure from RDF and from Turtle and other guiding lights.

[mwjames]

If you are suggesting that a given property X is used for a French value, and only a French value, then I categorically say that idea is a non-starter for me.

I'm pretty sure that is not what I said.

To clarify as that what a parser function is ought to be used and how DataValues/Datatypes play a vital role in modeling a conceptional representation of language dependent labels (not values) (which relates to my earlier https://github.com/SemanticMediaWiki/SemanticMediaWiki/issues/594#issuecomment-99139951), I wrote quick prototype [0] for SMW of how multilingual labels for properties are expected to work (also it shows what happens when a language is changed for generated queries).

[0] https://youtu.be/-3DOkkSa2sQ

[hypergrove]

If you are suggesting {{#set:pname=pvalue@langcode}} then let me share why I think my slightly different syntax ... {{#set:pname=@langcode:pvalue}} ... is necessary. Because we don't quote text values in the {{#set:}} parser function, an email address for instance causes a real problem. For example {{#set:[email protected]}}.

[hypergrove]

MWJames, Oh we are talking about different things I believe. You are talking about localizing the label of a property and I am talking about localizing the value of a property, right?

[hypergrove]

thank you for the youtube presentation - it was helpful. I understand now your view that the machinery for localizing the label of a property is the very exact same used to localize the value of a property.

You are proposing the english-labelled special property "Has property label" with each value qualified by its language, eg en:xxxxx and de:xxxxxx. I am proposing special properties such as "@en" and "@de". Your approach is more compact however the special property should be named like "smw:displaytitle" because (a) this text IS a displaytitle not so much a (field) label (b) this property equally applies to other pages not just property pages (c) all properties belong in a namespace.

May we now discuss the rest of the story:
a. how does the {{#set:}} parser function change?
b. how does the {{#ask:}} parser function change?

The subobject approach I proposed is fully transparent -- it is easy to create a query that shows the values in more than one language simultaneously.... I doubt that your approach can show, in a single query, values of a property across multiple languages. Also, I want to ensure your approach allows language-tagged page values, not just text values.

Thanks for the discussion./john

[hypergrove]

I see earlier mentioned this syntax: [[Has property label::en:Foo]]
Due to application of this syntax for page properties, "en:" causes confusion.
Hence I suggest [[Has property label::@en:Foo]]
Use of the 'at' symbol is consistent with standards I can cite.
Annotations like [[My page property::@en:ns:pgnm]] would now be unambiguous.

(I'm interested in your proposed changes to {{#ask:}} syntax}
thanks/john

[hypergrove]

The property label "Has property label" is a question too.

Alternative property names

1. smw:displaytitle (a contextually perfect fit)
2. rdfs:label (a semantically perfect fit)

[hypergrove]

Hi - any thoughts yet about changes to #ask, #set and wikitext annotation for multilingual annotations and queries? I'm really interested in how to show multiple languages' values for a given property in the same #ask table - thanks/john

[mwjames]

The topic "Multilingual Semantic MediaWiki" has been more or less finalized with the upcoming 2.5 release from a feature perspective where (as short-list):

• MonolingualTextValue is provided as dataType to store and annotate text in a specific language
• Has property description and Has preferred property label are deployed as "meta" property to localize the display of a property and create local-specific context information
• #LOCL as #ask output modifier to allow for some localized (@lang) formatted output
• |+lang= as filter option for a #ask query
• Semantic Interlanguage Links extension to manage/switch the page content language and connect content of a different language

The https://sandbox.semantic-mediawiki.org hosts combination of above listed features including:

• https://sandbox.semantic-mediawiki.org/wiki/Property:P106
• https://sandbox.semantic-mediawiki.org/wiki/Issue/2037
• https://sandbox.semantic-mediawiki.org/wiki/Issue/1545_%28LOCL%29
• https://sandbox.semantic-mediawiki.org/wiki/Berlin

[krabina]

I guess the wikidocu missing label can be removed here:
https://www.semantic-mediawiki.org/wiki/Localization_and_multilingual_content

[mwjames]

@krabina @kghbln
In reference to https://twitter.com/vrandezo/status/1149012495497138178

• Being "Multilingual" isn't a prerogative of Wikibase, certainly SMW only provides a subset of multilingual possibility but it certainly supports more functionality then by the time of its interception.
• "... source tracking ..." while SMW is mostly about untested claims, it supports a datamodel where references can be given to turn an untested into a factual claim (see references type)