Semanticmediawiki: SMW 3.2.1 breaks py-3rdparty-mediawiki deserialization

[WolfgangFahl]

Setup

www.semantic-mediawiki.org setup since 2020-11-21

Issue

Nightly jenkins tests of py-3rdparty-mediawiki break.

Steps to reproduce the observation (recommendation is to use the sandbox):
run https://github.com/WolfgangFahl/py-3rdparty-mediawiki/blob/master/tests/test_SMWApi.py before modifications and tests run.

After that tests fail. The JSON result of the versions is different and an explanation for the change seems to be missing.

[WolfgangFahl]

I tried to figure things out by checking https://www.semantic-mediawiki.org/wiki/Semantic_MediaWiki_3.2.1 but there seem to be inconsistencies. E.g. #4670 points to #4870 but both seem not to fit the title Makes ask and askargs API return a value for format=count which seems the most likely change related to this issue. Looking for the title i found #4857 which might be the relevant change that was incorporated.

[WolfgangFahl]

https://github.com/SemanticMediaWiki/SemanticMediaWiki/commit/c9b7a6ff3af00f1497a0e6cc3bd17de11b652a76 might be having a logic problem and returning a count value instead of result. Even if it returns a count value shouldn't it be wrapped in a "result" json node?

[WolfgangFahl]

dict: {
  'query': {
    'printrequests': [{
      'label': '',
      'key': '',
      'redi': '',
      'typeid': '_wpg',
      'mode': 2
    }, {
      'label': 'WikiDataId',
      'key': 'Has_Wikidata_item_ID',
      'redi': '',
      'typeid': '_eid',
      'mode': 1,
      'format': ''
    }, {
      'label': 'finish',
      'key': 'Has_planned_finish',
      'redi': '',
      'typeid': '_dat',
      'mode': 1,
      'format': ''
    }, {
      'label': 'start',
      'key': 'Has_planned_start',
      'redi': '',
      'typeid': '_dat',
      'mode': 1,
      'format': ''
    }, {
      'label': 'location',
      'key': 'Has_location',
      'redi': '',
      'typeid': '_txt',
      'mode': 1,
      'format': ''
    }],
    'results': 0,
    'serializer': 'SMW\\Serializers\\QueryResultSerializer',
    'version': 2,
    'meta': {
      'hash': '95c4bf6dc348409bada0f3ea2b8425cd',
      'count': 2,
      'offset': 0,
      'source': '',
      'time': '0.012280'
    }
  }
}

is returned for the query:

{{#ask:  [[Concept:Semantic MediaWiki Cons 2012]]
        |?Has_Wikidata_item_ID = WikiDataId
        |?Has planned finish = finish
        |?Has planned start =    start
        |?Has_location  =        location
        |  format=table  }}

which should look like:
https://www.semantic-mediawiki.org/wiki/User:WolfgangFahl/Workdocumentation_2020-11-28

[JeroenDeDauw]

@kghbln I did not look into this issue/report itself but please beware that patch releases are for fixes only. Not new features like this format=count stuff.

[WolfgangFahl]

Here is the result from http://smw.bitplan.com which has a copy of semantic-mediawiki.org for the relevant pages but runs Semantic Mediawiki 3.2.0 see http://smw.bitplan.com/index.php/Special:Version

dict: {
    'printrequests': [{
      'label': '',
      'key': '',
      'redi': '',
      'typeid': '_wpg',
      'mode': 2
    }, {
      'label': 'WikiDataId',
      'key': 'Has_Wikidata_item_ID',
      'redi': '',
      'typeid': '_eid',
      'mode': 1,
      'format': ''
    }, {
      'label': 'finish',
      'key': 'Has_planned_finish',
      'redi': '',
      'typeid': '_dat',
      'mode': 1,
      'format': ''
    }, {
      'label': 'start',
      'key': 'Has_planned_start',
      'redi': '',
      'typeid': '_dat',
      'mode': 1,
      'format': ''
    }, {
      'label': 'location',
      'key': 'Has_location',
      'redi': '',
      'typeid': '_txt',
      'mode': 1,
      'format': ''
    }],
    'results': {
      'SMWCon Fall 2012': {
        'printouts': {
          'WikiDataId': ['Q42407116'],
          'finish': [{
            'timestamp': '1351209600',
            'raw': '1/2012/10/26'
          }],
          'start': [{
            'timestamp': '1351036800',
            'raw': '1/2012/10/24'
          }],
          'location': ['Cologne, Germany']
        },
        'fulltext': 'SMWCon Fall 2012',
        'fullurl': 'http:///smw.bitplan.com/index.php/SMWCon_Fall_2012',
        'namespace': 0,
        'exists': '1',
        'displaytitle': ''
      },
      'SMWCon Spring 2012': {
        'printouts': {
          'WikiDataId': ['Q42407127'],
          'finish': [{
                'timestamp': '13354848...

[WolfgangFahl]

By the way - this issue spoiled part of the wikipush presentation during the conference last week. I was baffled when a simple query returned no result via the API - now i know why this happened ...

[JeroenDeDauw]

@osnard

[kghbln]

Thanks for reporting.

tried to figure things out by checking https://www.semantic-mediawiki.org/wiki/Semantic_MediaWiki_3.2.1 but there seem to be inconsistencies.

Yes, actually https://github.com/SemanticMediaWiki/SemanticMediaWiki/issues/4857 / https://github.com/SemanticMediaWiki/SemanticMediaWiki/pull/4858 was meant. This needs to be amended.

I did not look into this issue/report itself but please beware that patch releases are for fixes only. Not new features like this format=count stuff.

I figured that the api not returning proper values for count was a bug fix. My bad but I admittedly only spent about 1 to 2 seconds on this which also explains why I f..ked up the link in the release notes.

Quick solution:
Revert that specific commit in 3.2.x since I do not expect a fix for this feature or whatever this is meant to be in place soon and do a 3.2.2 release.

[WolfgangFahl]

Yes please - life will be much easier again :-)

[JeroenDeDauw]

@osnard Seems like I was to quick merging your change. Will be reverting soon it if it remains problematic. This cost us a bunch of time already.

[osnard]

Yes, please remove it. My apologies for causing trouble.

[osnard]

@JeroenDeDauw, maybe you could give a quick reply to my comment on https://github.com/SemanticMediaWiki/SemanticMediaWiki/pull/4858#issuecomment-706948328

I could then try to provide a better patch.

[kghbln]

Closing this in favor of #4857 The revert basically fixes this issue reported here.

[WolfgangFahl]

Excellent ... I can do git based backups of the wiki again using https://www.semantic-mediawiki.org/wiki/Wikipush_Toolkit

wikibackup -s smw -g -q "[[modification date::+]]" progress --limit  10000
.downloading 7332 pages from smw to /home/wf/wikibackup/smw
1/7332 (   0%): downloading SMWCon Spring 2013/"Data is data" and the future of data browsing ...✅
2/7332 (   0%): downloading Help:Schema/Error handling/NULL value found, but an object is required ...✅
3/7332 (   0%): downloading "Semantic-MediaWiki.org" got a new look/es ...✅
4/7332 (   0%): downloading "Semantic-MediaWiki.org" got a new look/fr ...✅
5/7332 (   0%): downloading "Semantic-MediaWiki.org" got a new look ...✅
...

works again