Semanticmediawiki: extension.json and $wgExtensionDirectory relation

Created on 22 Apr 2018 · 9Comments · Source: SemanticMediaWiki/SemanticMediaWiki

Setup and configuration

SMW version: 3.0.0-alpha
MW version: 1.30.0
PHP version: 7.1.16
MySQL version: 5.6.39

Issue

At my environment, I have two ways to deal with MW extensions: one way to extensions that are installed via composer and another way to extensions installed via git (the "Wikimedia way"):

SMW and other extensions installed via composer are placed at the standard w/extensions/ directory.
General Wikimedia extensions (and skins) that I installed via git are placed outside the main MW directoy:

/home/user/mw-REL1_28/extensions/
/home/user/mw-REL1_28/skins/
/home/user/mw-REL1_29/extensions/
/home/user/mw-REL1_39/skins/
/home/user/mw-REL1_30/extensions/
/home/user/mw-REL1_30/skins/

So I use $wgExtensionDirectory (and $wgStyleDirectory) to tell MW that extensions and skins are not at the standard path.
This configuration serves more than one MW instance (let's say I have 2 MW instances pointing to REL1_29 extensions, and 1 pointing to REL1_30, etc. - like a wiki farm) and eases the updating and the downgrading processes of MW versions, because I only have to change one setting ($wgExtensionDirectory) to the right directory.

For what I know, until SMW 2.5.6 these two methods of instalation were independent. composer installed extensions is always placed at w/extensions/ directory. But, I think after #1732, MW is trying to find SMW extension.json where $wgExtensionDirectory is pointing to, causing the Fatal error below.

Stack trace

Fatal error: Uncaught Exception: .../mw-REL1_30/extensions/SemanticMediaWiki/extension.json does not exist! in /w/includes/registration/ExtensionRegistry.php:99

 #0 /w/includes/GlobalFunctions.php(120): ExtensionRegistry->queue('/home/jaideraf/...')
 #1 /w/extensions/SemanticMediaWiki/src/GlobalFunctions.php(220): wfLoadExtension('SemanticMediaWi...')
 #2 /w/LocalSettings.php(459): enableSemantics('wikincat.org')
 #3 /w/includes/WebStart.php(102): require_once('/home/jaideraf/...')
 #4 /w/index.php(40): require('/home/jaideraf/...')
 #5 {main} thrown in /w/includes/registration/ExtensionRegistry.php on line 99

Steps to reproduce

Set $wgExtensionDirectory in LocalSettings.php to something other than {$IP}/extensions (the default value).
Install SMW 3.0.0-alpha.

bug

Source

jaideraf

Most helpful comment

Based off of @cicalese 's explanation a few comments above, I submitted #3138, which explicitly tells wfLoadExtension() which extension.json file to use, instead of looking in $wgExtensionDirectory.

legoktm on 26 Apr 2018

👍2

All 9 comments

@jaideraf First of all thanks for effort and time to investigate the issue.

think after #1732, MW is trying to find SMW extension.json where $wgExtensionDirectory is pointing to, causing the Fatal error below.

How did it came to this?

After a private conversation in Feb 2018 where I've been listening to arguments such as " ... the new extension registration mechanism is an improvement over the previous mechanism in many ways ..." and "... extension registration mechanism is still evolving and improving ..." and while my reservation towards extension.json is of public record, I thought I'm a good sport and help those that believe "... Semantic MediaWiki is built on the core MediaWiki software and should participate in the supported extension management mechanism ..." and so I merged #1732 to be part of the 3.0 release.

This configuration serves more than one MW instance (let's say I have 2 MW instances pointing to REL1_29 extensions, and 1 pointing to REL1_30, etc. - like a wiki farm) and eases the updating and the downgrading

Another argument I've heard was that with extension.json things are going to be easier for wiki-farm and enterprise users and I'm wondering now if I have misinterpret that argument.

What we know

We know that things did work for the described scenario before #1732.

We know that after the #1732 merge, extension.json does work in a standard setting but as above report indicates it does somehow not behave when $wgExtensionDirectory is involved.

What next?

We could:

Revert #1732 and stop trying to use extension.json
Keep things as they are and don't care about $wgExtensionDirectory users
other alternatives?

mwjames on 22 Apr 2018

Is it possible to code something like this? If extension.json is not located in $wgExtensionDirectory/SemanticMediaWiki then look for extension.json in {$IP}/extensions/SemanticMediaWiki (maintaining functionality).

jaideraf on 22 Apr 2018

I still think that reverting #1732 is the least favorable option here, however I am indeed surprised as everybody else here that $wgExtensionDirectory is being ignored by the extension registration method.

kghbln on 22 Apr 2018

I am indeed surprised as everybody else here that $wgExtensionDirectory is being ignored by the extension registration method.

I do not believe that this is the case. MediaWiki is expecting to find extensions at the location pointed to by $wgExtensionDirectory. That is, after all, what that variable is intended for: to point to where the extensions are. The fact that the composer-installed extensions are being put elsewhere, with the extensions split between two different locations, is the issue. The extension registration code is looking for the Semantic MediaWiki extension.json in a subdirectory where $wgExtensionDirectory is pointing.

Is it possible to code something like this? If extension.json is not located in $wgExtensionDirectory/SemanticMediaWiki then look for extension.json in {$IP}/extensions/SemanticMediaWiki (maintaining functionality).

That would be a MediaWiki core change that would alter the meaning of the $wgExtensionDirectory variable. That could have other implications and would possibly add unnecessary complexity.

Instead, would it be possible for you to add a composer.json file to the parent directory of your extension directory (the parent of the path pointed to by $wgExtensionDirectory) and to install the composer-installed extensions from there? That way, they would be located with the rest of your extensions instead of in {$IP}/extensions.

Another argument I've heard was that with extension.json things are going to be easier for wiki-farm and enterprise users and I'm wondering now if I have misinterpret that argument.

Having extensions split between two separate directories is not a typical wiki farm setup and is not the situation I have sought a solution to in our previous discussions. Those discussion focused composer-installed extensions that were being automatically enabled rather than allowing them to be selectively enabled.

cicalese on 23 Apr 2018

Instead, would it be possible for you to add a composer.json file to the parent directory of your extension directory (the parent of the path pointed to by $wgExtensionDirectory) and to install the composer-installed extensions from there? That way, they would be located with the rest of your extensions instead of in {$IP}/extensions.

Yes. But composer says:

Class ComposerHookHandler is not autoloadable, can not call pre-update-cmd script
Loading composer repositories with package information
Updating dependencies (including require-dev)
Package operations: 0 installs, 1 update, 0 removals
Package mediawiki/at-ease is abandoned, you should avoid using it. Use wikimedia/at-ease instead.
Writing lock file
Generating optimized autoload files
Class ComposerVendorHtaccessCreator is not autoloadable, can not call post-update-cmd script

I am not a developer, so I am not sure what is going on (I am just posting the results here).

And with only enableSemantics( 'wikincat.org' ); (required by SWM installation) or

enableSemantics( 'wikincat.org');
wfLoadExtension( 'SemanticMediaWiki' );

wfLoadExtension( 'SemanticMediaWiki' ); 
enableSemantics( 'wikincat.org');

in LocalSettings.php, I get:

Fatal error: Uncaught Error: Call to undefined function enableSemantics() in /w/LocalSettings.php:459 
Stack trace: 
#0 /w/includes/WebStart.php(102): require_once() 
#1 /w/index.php(40): require('/home/jaideraf/...') 
#2 {main} thrown in /w/LocalSettings.php on line 459

jaideraf on 25 Apr 2018

I also tried:

require_once "$wgExtensionDirectory/SemanticMediaWiki/SemanticMediaWiki.php";
enableSemantics( 'wikincat.org' );

And I got:

Warning: Class 'SMW\MediaWiki\Deferred\CallableUpdate' not found in /mw-REL1_30/extensions/SemanticMediaWiki/src/Aliases.php on line 8
Warning: Class 'SMW\Parser\InTextAnnotationParser' not found in /mw-REL1_30/extensions/SemanticMediaWiki/src/Aliases.php on line 9
Warning: Class 'SMW\Encoder' not found in /mw-REL1_30/extensions/SemanticMediaWiki/src/Aliases.php on line 10
Warning: Class 'SMW\Query\ResultPrinter' not found in /mw-REL1_30/extensions/SemanticMediaWiki/src/Aliases.php on line 11
Warning: Class 'SMW\Query\ResultPrinter' not found in /mw-REL1_30/extensions/SemanticMediaWiki/src/Aliases.php on line 12
Warning: Class 'SMW\Query\ExportPrinter' not found in /mw-REL1_30/extensions/SemanticMediaWiki/src/Aliases.php on line 13
Warning: Class 'SMW\Query\ResultPrinters\ResultPrinter' not found in /mw-REL1_30/extensions/SemanticMediaWiki/src/Aliases.php on line 14
Warning: Class 'SMW\Query\ResultPrinters\ResultPrinter' not found in /mw-REL1_30/extensions/SemanticMediaWiki/src/Aliases.php on line 15
Warning: Class 'SMW\Query\ResultPrinters\FileExportPrinter' not found in /mw-REL1_30/extensions/SemanticMediaWiki/src/Aliases.php on line 16
Warning: Class 'SMW\Query\ResultPrinters\ListResultPrinter' not found in /mw-REL1_30/extensions/SemanticMediaWiki/src/Aliases.php on ine 17
Warning: Class 'SMW\Query\Parser' not found in /mw-REL1_30/extensions/SemanticMediaWiki/src/Aliases.php on line 18
Warning: Class 'SMW\SQLStore\ChangeOp\ChangeOp' not found in /mw-REL1_30/extensions/SemanticMediaWiki/src/Aliases.php on line 19
Warning: Class 'SMW\Connection\ConnectionProvider' not found in /mw-REL1_30/extensions/SemanticMediaWiki/src/Aliases.php on line 20
Warning: Class 'SMW\DataValues\TypesValue' not found in /mw-REL1_30/extensions/SemanticMediaWiki/src/Aliases.php on line 21
Warning: Class 'SMW\DataValues\PropertyValue' not found in /mw-REL1_30/extensions/SemanticMediaWiki/src/Aliases.php on line 22
Warning: Class 'SMW\DataValues\StringValue' not found in /mw-REL1_30/extensions/SemanticMediaWiki/src/Aliases.php on line 23
Warning: Class 'SMW\Store' not found in /mw-REL1_30/extensions/SemanticMediaWiki/src/Aliases.php on line 26
Warning: Class 'SMW\MediaWiki\Jobs\UpdateJob' not found in /mw-REL1_30/extensions/SemanticMediaWiki/src/Aliases.php on line 27
Warning: Class 'SMW\MediaWiki\Jobs\RefreshJob' not found in /mw-REL1_30/extensions/SemanticMediaWiki/src/Aliases.php on line 28
Warning: Class 'SMW\SemanticData' not found in /mw-REL1_30/extensions/SemanticMediaWiki/src/Aliases.php on line 29
Warning: Class 'SMW\DIWikiPage' not found in /mw-REL1_30/extensions/SemanticMediaWiki/src/Aliases.php on line 30
Warning: Class 'SMW\DIProperty' not found in /mw-REL1_30/extensions/SemanticMediaWiki/src/Aliases.php on line 31
Warning: Class 'SMW\Serializers\QueryResultSerializer' not found in /mw-REL1_30/extensions/SemanticMediaWiki/src/Aliases.php on line 32
Warning: Class 'SMW\DataValueFactory' not found in /mw-REL1_30/extensions/SemanticMediaWiki/src/Aliases.php on line 33
Warning: Class 'SMW\Exception\DataItemException' not found in /mw-REL1_30/extensions/SemanticMediaWiki/src/Aliases.php on line 34
Warning: Class 'SMW\SQLStore\PropertyTableDefinition' not found in /mw-REL1_30/extensions/SemanticMediaWiki/src/Aliases.php on line 35
Warning: Class 'SMW\DIConcept' not found in /mw-REL1_30/extensions/SemanticMediaWiki/src/Aliases.php on line 36
Warning: Class 'SMW\Query\ResultPrinters\TableResultPrinter' not found in /mw-REL1_30/extensions/SemanticMediaWiki/src/Aliases.php on line 37
Warning: Class 'SMW\Query\ResultPrinters\FileExportPrinter' not found in /mw-REL1_30/extensions/SemanticMediaWiki/src/Aliases.php on line 40
Warning: Class 'SMW\AggregatablePrinter' not found in /mw-REL1_30/extensions/SemanticMediaWiki/src/Aliases.php on line 41
Warning: Class 'SMW\CategoryResultPrinter' not found in /mw-REL1_30/extensions/SemanticMediaWiki/src/Aliases.php on line 42
Warning: Class 'SMW\DsvResultPrinter' not found in /mw-REL1_30/extensions/SemanticMediaWiki/src/Aliases.php on line 43
Warning: Class 'SMW\EmbeddedResultPrinter' not found in /mw-REL1_30/extensions/SemanticMediaWiki/src/Aliases.php on line 44
Warning: Class 'SMW\RdfResultPrinter' not found in /mw-REL1_30/extensions/SemanticMediaWiki/src/Aliases.php on line 45
Warning: Class 'SMW\ListResultPrinter' not found in /mw-REL1_30/extensions/SemanticMediaWiki/src/Aliases.php on line 46
Warning: Class 'SMW\RawResultPrinter' not found in /mw-REL1_30/extensions/SemanticMediaWiki/src/Aliases.php on line 47
Warning: Class 'SMW\SPARQLStore\SPARQLStore' not found in /mw-REL1_30/extensions/SemanticMediaWiki/src/Aliases.php on line 50
Warning: Class 'SMW\SPARQLStore\RepositoryConnectors\FourstoreRepositoryConnector' not found in /mw-REL1_30/extensions/SemanticMediaWiki/src/Aliases.php on line 51
Warning: Class 'SMW\SPARQLStore\RepositoryConnectors\VirtuosoRepositoryConnector' not found in /mw-REL1_30/extensions/SemanticMediaWiki/src/Aliases.php on line 52
Warning: Class 'SMW\SPARQLStore\RepositoryConnectors\GenericRepositoryConnector' not found in /mw-REL1_30/extensions/SemanticMediaWiki/src/Aliases.php on line 53
Warning: Class 'SMWSQLStore3' not found in /mw-REL1_30/extensions/SemanticMediaWiki/src/Aliases.php on line 56
Warning: Class 'SMW\Query\Language\Description' not found in /mw-REL1_30/extensions/SemanticMediaWiki/src/Aliases.php on line 57
Warning: Class 'SMW\Query\Language\ThingDescription' not found in /mw-REL1_30/extensions/SemanticMediaWiki/src/Aliases.php on line 58
Warning: Class 'SMW\Query\Language\ClassDescription' not found in /mw-REL1_30/extensions/SemanticMediaWiki/src/Aliases.php on line 59
Warning: Class 'SMW\Query\Language\ConceptDescription' not found in /mw-REL1_30/extensions/SemanticMediaWiki/src/Aliases.php on line 60
Warning: Class 'SMW\Query\Language\NamespaceDescription' not found in /mw-REL1_30/extensions/SemanticMediaWiki/src/Aliases.php on line 61
Warning: Class 'SMW\Query\Language\ValueDescription' not found in /mw-REL1_30/extensions/SemanticMediaWiki/src/Aliases.php on line 62
Warning: Class 'SMW\Query\Language\Conjunction' not found in /mw-REL1_30/extensions/SemanticMediaWiki/src/Aliases.php on line 63
Warning: Class 'SMW\Query\Language\Disjunction' not found in /mw-REL1_30/extensions/SemanticMediaWiki/src/Aliases.php on line 64
Warning: Class 'SMW\Query\Language\SomeProperty' not found in /mw-REL1_30/extensions/SemanticMediaWiki/src/Aliases.php on line 65
Warning: Class 'SMW\Query\PrintRequest' not found in /mw-REL1_30/extensions/SemanticMediaWiki/src/Aliases.php on line 66
Warning: Class 'SMW\MediaWiki\Search\Search' not found in /mw-REL1_30/extensions/SemanticMediaWiki/src/Aliases.php on line 67
Warning: Class 'SMW\SQLStore\Lookup\ListLookup' not found in /mw-REL1_30/extensions/SemanticMediaWiki/src/Aliases.php on line 71
Warning: Class 'SMW\SQLStore\Lookup\ListLookup' not found in /mw-REL1_30/extensions/SemanticMediaWiki/src/Aliases.php on line 72
Warning: Class 'SMW\Exporter\Element\ExpElement' not found in /mw-REL1_30/extensions/SemanticMediaWiki/src/Aliases.php on line 74
Warning: Class 'SMW\Exporter\Element\ExpResource' not found in /mw-REL1_30/extensions/SemanticMediaWiki/src/Aliases.php on line 75
Warning: Class 'SMW\Exporter\Element\ExpNsResource' not found in /mw-REL1_30/extensions/SemanticMediaWiki/src/Aliases.php on line 76
Warning: Class 'SMW\Exporter\Element\ExpLiteral' not found in /mw-REL1_30/extensions/SemanticMediaWiki/src/Aliases.php on line 77
Warning: Class 'SMW\DataValues\ImportValue' not found in /mw-REL1_30/extensions/SemanticMediaWiki/src/Aliases.php on line 78
Warning: Class 'SMW\SQLStore\QueryEngine\QueryEngine' not found in /mw-REL1_30/extensions/SemanticMediaWiki/src/Aliases.php on line 79
Warning: Class 'SMW\ParserParameterProcessor' not found in /mw-REL1_30/extensions/SemanticMediaWiki/src/Aliases.php on line 82
Warning: Class 'SMW\ParameterProcessorFactory' not found in /mw-REL1_30/extensions/SemanticMediaWiki/src/Aliases.php on line 83
Warning: Class 'SMW\RequestOptions' not found in /mw-REL1_30/extensions/SemanticMediaWiki/src/Aliases.php on line 86
Warning: Class 'SMW\StringCondition' not found in /mw-REL1_30/extensions/SemanticMediaWiki/src/Aliases.php on line 87
Warning: Class 'SMW\HashBuilder' not found in /mw-REL1_30/extensions/SemanticMediaWiki/src/Aliases.php on line 88
Warning: Class 'SMW\DataValues\BooleanValue' not found in /mw-REL1_30/extensions/SemanticMediaWiki/src/Aliases.php on line 89
Warning: Class 'SMW\QueryPrinterFactory' not found in /mw-REL1_30/extensions/SemanticMediaWiki/src/Aliases.php on line 92
Warning: Class 'SMW\ParserFunctions\SubobjectParserFunction' not found in /mw-REL1_30/extensions/SemanticMediaWiki/src/Aliases.php on line 93
Warning: Class 'SMW\ParserFunctions\RecurringEventsParserFunction' not found in /mw-REL1_30/extensions/SemanticMediaWiki/src/Aliases.php on line 94
Warning: Class 'SMW\SQLStore\PropertyTableDefinition' not found in /mw-REL1_30/extensions/SemanticMediaWiki/src/Aliases.php on line 95
Warning: Class 'SMW\DataModel\ContainerSemanticData' not found in /mw-REL1_30/extensions/SemanticMediaWiki/src/Aliases.php on line 96

Fatal error: Uncaught Error: Class 'SMW\NamespaceManager' not found in /mw-REL1_30/extensions/SemanticMediaWiki/src/GlobalFunctions.php:225 
Stack trace: 
#0 /w/LocalSettings.php(461): enableSemantics('wikincat.org') 
#1 /w/includes/WebStart.php(102): require_once('/home/jaideraf/...') 
#2 /w/index.php(40): require('/home/jaideraf/...') 
#3 {main} thrown in /mw-REL1_30/extensions/SemanticMediaWiki/src/GlobalFunctions.php on line 225

jaideraf on 25 Apr 2018

The class not found errors indicate that Semantic MediaWiki is not using $wgExtensionDirectory to locate the base directory in which to look for extensions. The class map that is being built by the composer autoloader expects the extensions to be in the "extensions" subdirectory of the MediaWiki install directory regardless of the value of $wgExtensionDirectory.

Also, according to #1732, you should only need the call to enableSemantics(), not wfLoadExtension() or require_once.

cicalese on 25 Apr 2018

legoktm on 26 Apr 2018

👍2

For clarification, wfLoadExtension queues all its registered extensions and only after LocalSettings.php has been finished executing, the extension processor would attempt to access and register those listed extensions including settings and/or constant definitions. It is of no importance whether callback in extension.json is used or not even though it provides the earliest possible entry point.

enableSemantics vs. wfLoadExtension

Now, what happens when you add:

wfLoadExtension('SemanticMediaWiki');
$GLOBALS['smwgBrowseFeatures'] = SMW_BROWSE_TLINK | SMW_BROWSE_SHOW_INCOMING | SMW_BROWSE_SHOW_GROUP ;

to your LocalSettings.php it would report with:

<b>Notice</b>:  Use of undefined constant SMW_BROWSE_TLINK - assumed 'SMW_BROWSE_TLINK' in <b>...\LocalSettings.php</b> on line <b>234</b><br />

The reason is that constants like SMW_BROWSE_TLINK referenced in LocalSettings.php would have not been defined since wfLoadExtension hasn't loaded the extension yet!

Now, enableSemantics calls wfLoadExtension('SemanticMediaWiki') but ensures that DefaultSettings and Defines are loaded right away at the point of invocation in LocalSettings.php so settings that rely on constants such as SNW_NS... (and any other for that matter) are available from within the scope of LocalSettings.php.