Cockroach: sql: Full Text Search

[Linicks]

All,
I'm sure most of you know about Bleve (https://github.com/blevesearch/bleve) a Go-lang based full-text indexer. I was wondering if you've considered integrating it with CockroachDB? I'ts seems like it may be a good fit, and is being used in other distributed databases.

• It has an Apache licence.
• Written in Go.
• Already provides allot of value add functionality.

Thanks!
-- Nick

[petermattis]

@Linicks Full-text search is something we'd like to support and Bleve is on my radar, though there are no concrete plans to integrate it.

[alexander-manley]

One approach for integrating Bleve with Cockroach, and thus provide CockroachDB with text search, would be to modify hugoidx (https://github.com/blevesearch/hugoidx) to allow it to BLEVE-index the contents of a Cockroach BLOB store (...https://github.com/cockroachdb/cockroach/issues/243) pre-populated with corpus text (web page scrapes, text-doc-dumps etc...).

In addition to hugoidx, the associated Go utility "bleve-hosted" could be wrapped into the embedded UI (https://github.com/cockroachdb/cockroach/tree/master/ui) in order to pull-out and/or highlight text search results pulled from the BLOB store and displayed as an additional panel under the left side "DATABASES" UI tab.

Bleve is based on file indexes, which by default are stored in BoltDB, so that part would need to be ported over to RocksDB for full integration. For the curious, a Bleve benchmark graph with RocksDB was posted to the Bleve Twitter stream a while back.

Reference:
http://www.blevesearch.com/news/Site-Search/
http://www.blevesearch.com/videos/

[petermattis]

@alexander-manley Thanks for the notes. We'll definitely take a closer look at Bleve when considering full-text indexing.

[randyyaj]

Any updates on this?

[petermattis]

@randyyaj Full-text indexing is something we'd like to do, but still a ways off and not currently scheduled.

[SantoshSah]

@petermattis , any update?

[nstewart]

Full text search is something we want to support, but it is not on the roadmap for cockroachdb 2.1 or 2.2. While we are adding some new functionality, for the next couple releases we are focusing on improving the performance and stability of our current offering before we add major new features.

[RoachietheSupportRoach]

Zendesk ticket #3521 has been linked to this issue.

[OldhamMade]

Does this zendesk ticket mean that full-text indexing is being actively worked on?

[jordanlewis]

No, full text search isn't on the near term roadmap for the time being.

>