Semanticmediawiki: [[Property::Value 1||!Value 2]] causes database error

[Dinoguy1000]

Setup and configuration

• SMW version: 3.0.0
• MW version: 1.30.0
• PHP version: 7.1.22-1+ubuntu16.04.1+deb.sury.org+1 (apache2handler)
• DB system (MySQL, Blazegraph, etc.) and version: MariaDB 10.1.31-MariaDB-1~xenial
  (see also Special:Version)

Issue

Attempting to run a query which includes a property with multiple values, at least one of which is negated ([[Property::Value 1||!Value 2]], [[Property::!Value 1||Value 2]], [[Property::!Value 1||!Value 2]]) causes a database error with a stack trace (e.g., and all other revisions of that page before 2 April 2017). This has been happening since at least SMW 2.5.4 or 2.5.5, and possibly earlier (the earliest version we had installed was probably one of those two).

This also affects page importation, with errors similar to the following:

[ec34ecca9ffaedbff6f2580a] [no req]   Wikimedia\Rdbms\DBQueryError from line 1149 of /var/www/html/includes/libs/rdbms/database/Database.php: A database query error has occurred. Did you forget to run your application's database schema updater after upgrading?
Query: INSERT IGNORE INTO `t10` SELECT DISTINCT t11.smw_id FROM `smw_object_ids` AS t11  WHERE t11.smw_sortkey!='Tuner monster'
Function: SMW\SQLStore\QueryEngine\QuerySegmentListProcessor::doResolveBySegment
Error: 1114 The table 't10' is full (localhost)

Stack trace

Sample stack trace (from most recent broken revision:

[f92b2e9efa523e725226b601] /index.php?title=Card_Tips:Junk_Anchor&oldid=3803808 Wikimedia\Rdbms\DBQueryError from line 1149 of /var/www/html/includes/libs/rdbms/database/Database.php: A database query error has occurred. Did you forget to run your application's database schema updater after upgrading? 
Query: INSERT IGNORE INTO `t10` SELECT DISTINCT t11.smw_id FROM `smw_object_ids` AS t11 WHERE t11.smw_sortkey!='Tuner monster'
Function: SMW\SQLStore\QueryEngine\QuerySegmentListProcessor::resolve
Error: 1114 The table 't10' is full (159.65.191.204)
Backtrace:

#0 /var/www/html/includes/libs/rdbms/database/Database.php(979): Wikimedia\Rdbms\Database->reportQueryError(string, integer, string, string, boolean)
#1 /var/www/html/extensions/SemanticMediaWiki/src/MediaWiki/Connection/Database.php(442): Wikimedia\Rdbms\Database->query(string, string, boolean)
#2 /var/www/html/extensions/SemanticMediaWiki/src/SQLStore/QueryEngine/QuerySegmentListProcessor.php(242): SMW\MediaWiki\Connection\Database->query(string, string)
#3 /var/www/html/extensions/SemanticMediaWiki/src/SQLStore/QueryEngine/QuerySegmentListProcessor.php(121): SMW\SQLStore\QueryEngine\QuerySegmentListProcessor->resolve(SMW\SQLStore\QueryEngine\QuerySegment)
#4 /var/www/html/extensions/SemanticMediaWiki/src/SQLStore/QueryEngine/QuerySegmentListProcessor.php(121): SMW\SQLStore\QueryEngine\QuerySegmentListProcessor->resolve(SMW\SQLStore\QueryEngine\QuerySegment)
#5 /var/www/html/extensions/SemanticMediaWiki/src/SQLStore/QueryEngine/QuerySegmentListProcessor.php(201): SMW\SQLStore\QueryEngine\QuerySegmentListProcessor->resolve(SMW\SQLStore\QueryEngine\QuerySegment)
#6 /var/www/html/extensions/SemanticMediaWiki/src/SQLStore/QueryEngine/QuerySegmentListProcessor.php(121): SMW\SQLStore\QueryEngine\QuerySegmentListProcessor->resolve(SMW\SQLStore\QueryEngine\QuerySegment)
#7 /var/www/html/extensions/SemanticMediaWiki/src/SQLStore/QueryEngine/QuerySegmentListProcessor.php(112): SMW\SQLStore\QueryEngine\QuerySegmentListProcessor->resolve(SMW\SQLStore\QueryEngine\QuerySegment)
#8 /var/www/html/extensions/SemanticMediaWiki/src/SQLStore/QueryEngine/QueryEngine.php(193): SMW\SQLStore\QueryEngine\QuerySegmentListProcessor->process(integer)
#9 /var/www/html/extensions/SemanticMediaWiki/includes/storage/SQLStore/SMW_SQLStore3.php(311): SMW\SQLStore\QueryEngine\QueryEngine->getQueryResult(SMWQuery)
#10 /var/www/html/extensions/SemanticMediaWiki/includes/storage/SQLStore/SMW_SQLStore3.php(299): SMWSQLStore3->fetchQueryResult(SMWQuery)
#11 /var/www/html/extensions/SemanticMediaWiki/includes/query/SMW_QueryProcessor.php(378): SMWSQLStore3->getQueryResult(SMWQuery)
#12 /var/www/html/extensions/SemanticMediaWiki/src/ParserFunctions/AskParserFunction.php(344): SMWQueryProcessor::getResultFromQuery(SMWQuery, array, integer, integer)
#13 /var/www/html/extensions/SemanticMediaWiki/src/ParserFunctions/AskParserFunction.php(183): SMW\ParserFunctions\AskParserFunction->doFetchResultsFromFunctionParameters(array, array)
#14 /var/www/html/extensions/SemanticMediaWiki/src/ParserFunctionFactory.php(385): SMW\ParserFunctions\AskParserFunction->parse(array)
#15 /var/www/html/includes/parser/Parser.php(3383): SMW\ParserFunctionFactory->SMW\{closure}(Parser, string, string, string, string, string, string, string, string, string)
#16 /var/www/html/includes/parser/Parser.php(3106): Parser->callParserFunction(PPFrame_DOM, string, array)
#17 /var/www/html/includes/parser/Preprocessor_DOM.php(1229): Parser->braceSubstitution(array, PPFrame_DOM)
#18 /var/www/html/includes/parser/Parser.php(2921): PPFrame_DOM->expand(DOMElement, integer)
#19 /var/www/html/includes/parser/Parser.php(1277): Parser->replaceVariables(string)
#20 /var/www/html/includes/parser/Parser.php(451): Parser->internalParse(string)
#21 /var/www/html/includes/content/WikitextContent.php(329): Parser->parse(string, Title, ParserOptions, boolean, boolean, integer)
#22 /var/www/html/includes/content/AbstractContent.php(516): WikitextContent->fillParserOutput(Title, integer, ParserOptions, boolean, ParserOutput)
#23 /var/www/html/includes/poolcounter/PoolWorkArticleView.php(144): AbstractContent->getParserOutput(Title, integer, ParserOptions)
#24 /var/www/html/includes/poolcounter/PoolCounterWork.php(123): PoolWorkArticleView->doWork()
#25 /var/www/html/includes/page/Article.php(585): PoolCounterWork->execute()
#26 /var/www/html/includes/actions/ViewAction.php(68): Article->view()
#27 /var/www/html/includes/MediaWiki.php(499): ViewAction->show()
#28 /var/www/html/includes/MediaWiki.php(293): MediaWiki->performAction(Article, Title)
#29 /var/www/html/includes/MediaWiki.php(851): MediaWiki->performRequest()
#30 /var/www/html/includes/MediaWiki.php(523): MediaWiki->main()
#31 /var/www/html/index.php(43): MediaWiki->run()
#32 {main}

[Nikerabbit]

Error: 1114 The table 't10' is full

Please search this error from the internet. It very strongly points to lack of disk space or database server configuration issue rather than a bug in SMW.

[Dinoguy1000]

This was noticed in the middle of a massive import, which otherwise proceeed smoothly. While the page was being processed, the import failed on each revision - that is, only a single revision of the page would be imported, after which the import failed again with a stack trace similar to the first one in my OP. Once the problematic revisions were fully imported, the import proceeded through the rest of the dump without error. I'm more than happy to be proven wrong, but to me this points to some issue other than a full disk or misconfiguration being the problem.

Even ignoring that, though, there is still the fact that this still actively errors out when attempting to view one of the affected revisions, or when attempting to run a query with the syntax in question (a single property with two or more values, at least one of which is negated), as well as that all other queries we've run to date are perfectly fine.

[kghbln]

First of all thank your for reporting the issue. I believe the issue is that you use unsupported syntax though admittedly it is not explicitly stated in the documentation. At least I new do this for a reason I cannot longer remember but rather do something like this:

{{#ask:
 [[Has colour::!red]]
 [[Has colour::!blue]]
 |?Has colour
}}

[kghbln]

I will label this as a question for now and let the developers decide if it is either a bug or an enhancement.

[Nikerabbit]

As far as I know, t10 is a temporary table used by SMW query engine. It is an INSERT statement which nicely fits the theory that you are out of disk space or mysql is unable to create new tables.

[Dinoguy1000]

To be clear, we do not currently use this syntax anywhere on the wiki that I'm aware of; the current version of the page that prompted this report uses a completely different query to achieve the same result, which works correctly.

As an extra data point, the old query also breaks on Wikia, though just with a generic HTTP 500 error.

I still doubt that the t10 error is actually relevant here, but I don't know enough personally to make that assessment, I'm just working off of my observations of the issue; I'll be sharing this with our server guy and asking him to comment here, and hopefully he'll be able to provide more information (or someone can provide him with some way to get more information about this).

[kghbln]

As an extra data point, the old query also breaks on Wikia, though just with a generic HTTP 500 error.

I believe this supports my assessment that we are talking about unsupported syntax and this not being any regression of some kind since wikia is using 2.4.6. The docu undoubtedly needs to be improved.

[Dinoguy1000]

I have no problem if this turns out to be unsupported syntax. I would argue that it still shouldn't result in a database error, though, and point out that it must have worked at some point in the (possibly distant) past, in order for the query to have been saved in the first place and not reverted as breaking the page.

[Dinoguy1000]

Interestingly, previewing on the SMW sandbox wiki with syntax like {{ #ask: [[Test::A||!B]] }} results in an indiscriminate list (of values stored to all properties, maybe? it doesn't seem to be pages on the wiki, given the redlinks in the list that have apparently never been created) rather than a database error. I would argue that this particular result is wrong regardless of anything else, simply because Property:Test doesn't exist on that wiki, has no pages storing values to it, and querying it for A or B individually, negated or not, does not return any results.

[Nikerabbit]

Maybe the unsupported syntax causes the query engine to generate an incorrect and pathological query that bumps into some limits.

[penguinstyles]

With 41GB of available storage space, I can confirm that the t10 error is not being caused by a lack of space.

[kghbln]

Thanks for providing all the information. I personally lean towards this being a bug. Let's leave this to the developers to assess.

[Dinoguy1000]

Just to note, this still results in a database error on SMW 3.1.5: page, Special:Version.

[kghbln]

Thanks for the note. Indeed the wiki should probably fail gracefully here or we need support for this syntax. https://github.com/SemanticMediaWiki/SemanticMediaWiki/issues/3524#issuecomment-430134377 did not work for you?

[Dinoguy1000]

Thanks for the note. Indeed the wiki should probably fail gracefully here or we need support for this syntax.

I would be fine with it failing gracefully; my main concern is just that it not result in a database error. Anything beyond that would be a pleasant bonus, but would not be necessary for me to consider this issue resolved.

#3524 (comment) did not work for you?

I haven't tried that syntax; we currently don't have any queries where this style of double property value negation is required, so testing it hasn't really struck me as necessary. If you'd like me to just for the data point, though, I have no problem doing so. =)