Readthedocs.org: Search index is broken

[humitos]

Expected Result

Do not get the 404 at latest step :)

Actual Result

1. go to http://docs.readthedocs.io/en/latest/search.html?q=break+internet&check_keywords=yes&area=default
2. just one result appears
3. click on the page
4. :bomb: get 404 (actual link http://docs.readthedocs.io/en/latest/faq.rst.html?highlight=break%20internet)

... and I broke the internet :grinning:

[humitos]

The problem is the extra .rst extension in the file in the URL. This one works: http://docs.readthedocs.io/en/latest/faq.html?highlight=break%20internet

[ericholscher]

Guessing this might be fallout from the 1.5 upgrade, but not sure.

[ericholscher]

This is from the Sphinx search (the RTD search results work fine: http://docs.readthedocs.io/en/latest/search.html?q=testing&check_keywords=yes&area=default#). Guessing we need to upgrade some of the search JS.

[agjohnson]

I can look into this. Mostly certain it's the search override JS

[agjohnson]

Also, for background here: our search overrides the default sphinx search. If our search returns no results, we back down to the sphinx search. What we're noticing here is that a missing search term from our index is a hit on the sphinx index, but something has changed with the sphinx implementation there.

So, our indexes aren't broken, but something is wrong with the sphinx failover.

[nhitchins]

Any resolution? Search links have been broken for 10 days

[nhitchins]

Any chance of rolling back to a working version? Having a broken search feature is not really viable for the majority of docs users.

[ericholscher]

Can you give an example of a broken link? This should only happen when Sphinx's default search is triggered, which is a minority of cases.

[ericholscher]

You can change the version of sphinx your doc builds use using a requirements.txt file, if you need to roll back to an old version of sphinx.

[dlech]

Can you give an example of a broken link?

http://docs.ev3dev.org/projects/lego-linux-drivers/en/ev3dev-jessie/search.html?q=i2c&check_keywords=yes&area=default

All of the links have the extra .rst in them that breaks the link.

[akien-mga]

Here's another example: http://docs.godotengine.org/en/stable/search.html?q=kinematicbody2d&check_keywords=yes&area=default
All results have the extra .rst.

[nicoddemus]

Another example from the pytest's docs:

• Query: https://docs.pytest.org/en/latest/search.html?q=pytest_runtest_call
• First result: https://docs.pytest.org/en/latest/writing_plugins.rst.html?highlight=pytest_runtest_call#_pytest.hookspec.pytest_runtest_call

(cc pytest-dev/pytest#2302)

[nhitchins]

Tried to use the requirements.txt file to specify an earlier sphinx version as suggested but the build is failing: https://readthedocs.org/projects/eagleio/builds/5258802/

[davendal]

Not a developer, however I just wanted to mention that to get search working properly again I had to go back to Sphinx 1.4.9 then it worked successfully. Otherwise it just hangs with 'Searching...'.
Might give a clue.

[nicoddemus]

Fixed pytest's doc also by pinning to Sphinx 1.4, see pytest-dev/pytest#2352. Worth noting that I also configured the path to the requirements.txt file in Settings.

[gravis]

We're now having this issue too:
http://docs.gemnasium.net/en/latest/search.html?q=upgrade&check_keywords=yes&area=default
=> broken links (404)

[nicoddemus]

@gravis see my comment above yours for a possible workaround.

[gravis]

@nicoddemus Thanks!
I confirm this workaround is working for us :)

[smrgit]

I just came across this bug when doing a multi-word search on our (docs)[http://isb-cancer-genomics-cloud.readthedocs.io/en/latest/index.html] -- if I search for a single word, eg "imaging", it works fine, but if I search for "imaging data" the search results include the extraneous "rst".

[ekpgh]

Related to this bug in Sphinx 1.5: https://github.com/sphinx-doc/sphinx/issues/3696

[slinkp]

Is there any workaround other than downgrading sphinx to 1.4.x? I have a (private) project that's affected by this but also relies on the graphviz directive align option which was added in 1.5 :(

[slinkp]

Has anyone else observed that (javascript) search results work fine in Sphinx 1.5 when built locally, but exhibit the bad path problem (foo.rst.html) only when built on readthedocs.org ?

[veenamj]

I have the same issue. I want to use Sphinx 1.5 but I want to get rid of this search broken link error (foo.rst.html).

Can someone let me know if there is any other workaround other than downgrading Sphinx version.

[nhitchins]

Still having same problem with docs failing to build via RTD when downgrading to Sphinx 1.4.x (builds fine locally). https://readthedocs.org/projects/eagleio/builds/5258802/

[agjohnson]

This will be resolved with a release of our extension, https://github.com/rtfd/readthedocs-sphinx-ext/pull/26 solves the issues here around Sphinx 1.5 using an incompatible version searchtools.js from Sphinx 1.4. I should have that released and deployed this week.

[nicoddemus]

@agjohnson that's great news, thanks for working on this! 👍

[agjohnson]

So our extension is now released and Read the Docs uses the new release by default. To test these changes, you'll likely need to wipe the version from your project's versions page to ensure you get the new static files. I'll test against our docs (and the original bug report project) later, but if anyone else gets a chance to try this, feel free to report back

[smrgit]

works nicely for me, thank you! :+1:

actually, correction -- it works when I search for a new two word string, but when I search for the phrase that I originally reported as causing me problems ("imaging data"), I still get the .rst.html links ... but if I search for "imaging archive", then I get clean links

(and I did try a "wipe" and "rebuild" after I first noticed the recurring error and even after that, I still get the error when searching for "imaging data") -- although perhaps that is because of my browser cache?

[stevepiercy]

Almost works for me.

Latest points to 1.8-branch, but does not appear to be working:
http://docs.pylonsproject.org/projects/pyramid/en/latest/search.html?q=traversal

And its latest build:
https://readthedocs.org/projects/pyramid/builds/5415000/

These are good:
http://docs.pylonsproject.org/projects/pyramid/en/master/search.html?q=traversal
http://docs.pylonsproject.org/projects/pyramid/en/1.9-branch/search.html?q=traversal
http://docs.pylonsproject.org/projects/pyramid/en/1.8-branch/search.html?q=traversal
http://docs.pylonsproject.org/projects/pyramid/en/1.7-branch/search.html?q=traversal

I had to wipe and rebuild twice in some cases.

[stevepiercy]

...and without doing anything except allowing time to pass, latest now works, too.
http://docs.pylonsproject.org/projects/pyramid/en/latest/search.html?q=traversal

Thank you!

[agjohnson]

Given we only cache build data for ~24h, projects should slowly start magically working. Poking various projects above, it looks like the search results are now correct. I'll close this for now, but feel free to reopen if this didn't help your project and you are now using Sphinx 1.5.

[dlech]

Still broken for me after a wipe/rebuild: http://docs.ev3dev.org/projects/lego-linux-drivers/en/ev3dev-jessie/search.html?q=i2c&check_keywords=yes&area=default