Semanticmediawiki: Add support for additional RDF stores like Apache Rya / Dgraph

[akuckartz]

The triple store Apache Rya recently became an Apache top-level project. See https://rya.apache.org/

It should be supported by SMW.

[kghbln]

Just recently Dgraph https://dgraph.io/ popped up, too. It seems that the number of alternatives for RDF stores are increasing.

Anybody tried to get these connected via the generic connector?

[kghbln]

I just transformed this into a meta issue to have a central spot to discuss this. Once the focus moves to one or both of them separate issues may be created for a developer to pick up.

[mwjames]

A couple of notes:

The triple store Apache Rya recently became an Apache top-level project. See https://rya.apache.org/

From a project perspective it would be nice to have a confirmation whether using Apache Rya is generally feasibly or not but it seems unlikely that the project can declare compatibility to an external service it does not test nor have developer resources who can actively engage in experimenting with Apache Rya.

Given the constraints on resources available, I think the project needs to focus on those that can be easily integrated in our existing CI infrastructure hereby allows to confirm its compatibility otherwise the project looses focus and is getting torn between all sorts of possible combinations of new arising RDF stores (e.g. Amazon Neptune, see also #3199).

It should be supported by SMW.

From a technical perspective, we can only test software and guarantee compatibility with external services that are part of the CI infrastructure. Now, if Apache Rya provides a REST service then using either [0] or [1] are the first two options to verify whether a connection and replication between the RDF store and SMW can be established or not.

Semantic MediaWiki provides specialized connectors where it deemed to not impose too much technical dept as in the case for Virtuoso, 4Store, and Fuseki which can be used as starting point in case the Generic connector doesn't cover required functionality by Apache Rya (e.g. special HTTP header etc.).

In any event, indicating and declaring compatibility with any new service (or repository connector) can only happen after it is confirmed that the entire test suite runs successfully against such service which is the same yardstick we apply for Blazegraph, Virtuoso, Sesame, and Fuseki.

Summary

I want to work on this, how should I proceed?

• Review existing documentation
• Build a baseline (as comparison matrix against a new service)
  
  
  
  • [ ] Install Fuseki either as download or via docker (see also [2] for details on its usage in the CI environment)
  
  
  • [ ] Add required settings (see [3]) that complies with your environment
  
  
  • [ ] Run the test suite (run composer test from the command shell and extension root) against Fuskei in order to see a similar output as in [4]
  
  
• Connect to Apache Rya
  
  
  
  • [ ] Try to establish a connection via a generic or custom connector to your Apache Rya service
  
  
  • [ ] Repeat the steps as outlined above
  
  
  • [ ] If the test suite was completed without errors then we can indicate compatibility and you may report the test results and version constraints
  
  
  • [ ] If the test suite fails then you need to investigate which tests failed and why
  
  

Notes

The following should provide some context in running RDF stores with SMW:

• https://www.youtube.com/watch?v=oi2bAWpl32w (Blazegraph)
• https://vimeo.com/126392433 (Sesame)
• https://vimeo.com/118614078 (Fuseki)

[0] https://www.semantic-mediawiki.org/wiki/Help:SPARQLStore/RepositoryConnector/Generic
[1] https://www.semantic-mediawiki.org/wiki/Help:SPARQLStore/RepositoryConnector/Custom
[2] https://github.com/SemanticMediaWiki/SemanticMediaWiki/tree/master/tests/travis
[3] https://www.semantic-mediawiki.org/wiki/Help:SPARQLStore/RepositoryConnector/Fuseki
[4] https://travis-ci.org/SemanticMediaWiki/SemanticMediaWiki/jobs/589250957