Ckan: Solr version should be explicitly defined as ckan requirement

[SergK]

CKAN Version if known (or site URL)

ckan (2.8.0a0)

Please describe the expected behavior

nosetest should pass with solr version 6.2 or at least it should be explicitly mentioned that tests use solr version solr-4.3.1 and the same version should be used for production use. We can also upgrade to solr-6.2, then tests should be updated as well.

Please describe the actual behaviour

Run nosetests test with solr version 6.2

nosetests --ckan --reset-db --with-pylons=test-core.ini --nologcapture --with-coverage --cover-package=ckan --cover-package=ckanext --with-xunit --xunit-file=$CIRCLE_TEST_REPORTS/nose/junit.xml --segments=$SEGMENTS ckan ckanext`

It is expected to pass the test, but it fails:

 /home/jenkins/workspace/bos-ckan.test/venv/bin/nosetests --ckan --reset-db --with-pylons=test-core.ini --nologcapture --with-coverage --cover-package=ckan --cover-package=ckanext --with-xunit --xunit-file=/home/jenkins/workspace/bos-ckan.test/nose/junit.xml --segments=01234 ckan ckanext
No handlers could be found for logger "ckan.lib.uploader"
...../home/jenkins/workspace/bos-ckan.test/venv/local/lib/python2.7/site-packages/sqlalchemy/orm/unitofwork.py:79: SAWarning: Usage of the 'related attribute set' operation is not currently supported within the execution stage of the flush process. Results may not be consistent.  Consider using alternative event listeners or connection-level operations instead.
  sess._flush_warning("related attribute set")
...................................................................................................................................................................................................................................................................................F........................S............................................................................................................................................................................................................................................................................................................................................................................................................................................................
======================================================================
FAIL: ckan.tests.legacy.lib.test_solr_search_index.TestSolrSearch.test_1_basic
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/jenkins/workspace/bos-ckan.test/venv/local/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/home/jenkins/workspace/bos-ckan.test/ckan/ckan/tests/legacy/lib/test_solr_search_index.py", line 54, in test_1_basic
    assert len(results) == 2
AssertionError

What steps can be taken to reproduce the issue?

• Configure solr version 6.2
• run nosetest

nosetests --ckan --reset-db --with-pylons=test-core.ini --nologcapture --with-coverage --cover-package=ckan --cover-package=ckanext --with-xunit --xunit-file=$CIRCLE_TEST_REPORTS/nose/junit.xml --segments=$SEGMENTS ckan ckanext`

[amercader]

@SergK You are right, we should document the supported Solr versions.

@TkTech you worked on this recently, any ideas on what we should recommend?

Are there any extra steps that need to be added to these instructions to support Solr 5 or 6?

http://docs.ckan.org/en/latest/maintaining/installing/install-from-package.html#install-and-configure-solr

As for this particular issue it looks like the behaviour of Solr 6 has changed (@wardi mentioned fq issues)

[TkTech]

We can list usable versions but they release so frequently and with such poor quality releases there's little point in supporting anything official. Currently CKAN can work with minor changes from v3 to v6 (see #3510)

[Cheomm88]

Hello,
I have just installed CKAN in Ubuntu and it seems it uses solR v.3.6. I'm working in another installation but this time in centOS and all tutorials/hows to talks about using solR 1.4.x

Which version should I install?

"Hows to"
https://github.com/ckan/ckan/wiki/How-to-install-CKAN-2.5.2-on-CentOS-6.8
https://github.com/ckan/ckan/wiki/How-to-install-CKAN-2.4.1-on-CentOS-7

[TkTech]

I'm in favor of only officially supporting recent versions of Solr with the next release. Solr has been pushing breaking update versions too quickly for us to support a wide range of versions perpetually.

Version 6.6.* is stable and trivial to install from the Solr site. Solr maintainers seem to be behind with debian and most other distros being stuck on 3.6.2. Our documentation and packages should never recommend using the system solr package.

Documentation really needs updating, we have some install guides recommending solr 1.4.1. We should stop covering "how to install solr" in all install guides and just point to the Solr guide. We're just creating duplicate, often incorrect/outdated documentation that adds extra maintenance.

@amercader thoughts on pegging to 6.6.2 for now? I will update the schema and lib/search, along with any documentation I can find.

[adborden]

Solr lists anything less than 7.7 as being end of life. We should pin to the 7.7.x series which is listed as long-term support.

https://lucene.apache.org/solr/downloads.html#about-versions-and-support