Gogs: Repository/user search (Code and Issues)

[unknwon]

• [x] Repositories
• [x] Users
• [ ] Issue
• [ ] Code

[lifeofguenter]

Ideally via sphinx or similar.

[rojaro]

Codesearch - https://code.google.com/p/codesearch/ - is already implemented in Go and could serve as a base for further development and a quick prototype. Currently it uses its own index and is therefore not dependend on external services, but it could be extended with a simple plugin API to add ElasticSearch, Sphinx etc.

[lunny]

Sounds great! @rojaro @Unknwon

[unknwon]

@lifeofguenter @rojaro Thanks for your info. I need some time to study it.

[timshannon]

git grep may get you pretty far for code search at least, but Issue search will need something more.

[CarlosGtrz]

This project: https://github.com/etsy/Hound (based in codesearch) already has repository indexing and a web front end.

When I tested it, it failed to index source files encoded in Windows Ansi. If you add this feature to gogs, plese be sure to also support non-utf8 encodings.

[unknwon]

@CarlosGtrz thanks for the tip!

[andrewhowdencom]

+1 - This would be awesome.

[m42e]

+1

[dentmaged]

+1

[tomas]

+1!

@Unknwon any plans to get something like this included?

[chazzbg]

+1

[dumblob]

+1

[sgohl]

+1

[reinos]

+1

[rogeriolino]

+1 for codesearch https://github.com/etsy/hound

[ChangePlaces]

I miss not being able to search the issues the most for a keyword! A huge +1

[unknwon]

Just a note: https://github.com/etsy/hound

[tboerger]

Just a note: https://github.com/etsy/hound

That looks promising

[lunny]

It seems https://github.com/etsy/hound is good enough.

[windsource]

+1

[dgellow]

Please, stop with the +1 madness. You are generating noise for everyone subscribed to this thread.

[tboerger]

@dgellow maybe you should unsubscribe from the issue. That is the only way for the community to show how important a feature is until github introduces another way for it.

[BurakDev]

+1 @tboerger :blush: and +1 for this feature :+1:

[k1ltr0]

+1

[star-buck]

+2

[syhlion]

+1

[DarthNihilus1]

+1

[i00]

+:100:

[giafar]

+1

[hexadecy]

Issues before

[tomas]

Is anyone working on this? I don't have much experience with Go, but I'm willing to take a dive and see how far I get.

[unknwon]

@tomas it is a huge huge huge one... Don't recommend.. try some others labeled with help wanted.

[tomas]

Haha, is it? Then it should be fun. :)

[unknwon]

Implement something is easy, implement and follow the convention are hard... which means code review is strict.

[laurent-simon]

May be this feature request must be split in 4 different ones:

• Repositories search
• Users search
• Issues search
• Code search

Even if they share the "search" word, they are different issues. They don't need to share anything. If they can share something (UI or code), it will be great but not a requirement.

I was looking how to improve issues search and I think that putting all 4 together is too ambitious.

[richmahn]

What work can I do on this? I am working with our Gogs fork full time for my job, so in a sense paid to make this better. I know as said above this is a big chore, but I have made tons of search engines and displayed results in grids and lists. It is one of our issues, as you can see from the reference I made.

[unknwon]

@richmahn do you guys have a IM channel that I can join...?

[richmahn]

@Unknwon We use Slack for our unfoldingWord development communication. It can be joined at http://door43.org/en/slack. You can find me on there as @rich. Yet if there is another way you want to chat, just let me know.

[unknwon]

It is now possible to search repository and user by name on develop branch, both on explore and admin panel pages.

[ChangePlaces]

Wow, this a great start to taking gogs to the "next level"! Congratulations @Unknwon and @richmahn

[moltam]

This is great! Thank you guys!

[richmahn]

@ChangePlaces I did nothing, other than push him (I like to think there was some whip cracking, but he was already done before I even got a chance to ask if he had a chance to work on it today, lol), heh. And I tested it with postgres.

[cmtec]

Search in admin panel only displays public repos. Is this intended?

[ChangePlaces]

Oh, actually, it may be worth pointing out that SQLite has problems with fulltext support. If I remember correctly, it needs a separate table for full text indexed columns.

[unknwon]

@cmtec good point! Just pushed a fix to develop.

@ChangePlaces any pointers?

[ChangePlaces]

@Unknwon A really simple breakdown of it - http://blog.xojo.com/2014/03/14/full_text_search_with_sqlite/

There's also details about FTS5 - full-text-search module 5, SQLite's module for full-text searches. I'm not sure if it's compiled in by default, but I seem to remember it is.

https://www.sqlite.org/fts5.html

I remember having to completely restructure my SQLite tables to take account of full text searches, otherwise, you are duplicating the data into another table. This may / may not be acceptable for searching here.

[unknwon]

@ChangePlaces thanks! Does search not work in SQLite3 right now?

[ChangePlaces]

@Unknwon do you mean if searching works in gogs with sqlite? i'm not sure, I don't use SQLite with it, but just wanted to point it out!

[pnovotnak]

Thanks for fixing the "only public repos appear in search" issue @Unknwon !

[tomas]

I jumped yesterday into this and got a raw but working implementation of code search via git grep. It's a hacky solution but it works, although the search results aren't immediate (takes about 4 secs to search through my ~100 repos on a small VPS).

This is what it looks like:

If anyone's interested I can put my code available on a fork. But be warned that this not the ideal solution and also my very first dive into Go. :)

[denji]

git grep can down the performance of disk i/o (nice + ionice)

Code indexer

• https://github.com/etsy/hound (based google/codesearch)
• https://github.com/google/zoekt (Indexing git repositories / web interface)
• https://github.com/google/codesearch (Google Code Search – indexer prefetch regex search.
• http://awgn.github.io/cgrep/

Better git grep:

• http://beyondgrep.com/
• https://github.com/ggreer/the_silver_searcher
• https://github.com/BurntSushi/ripgrep
• https://github.com/monochromegane/the_platinum_searcher
• https://github.com/tkengo/highway
• https://github.com/svent/sift

[tomas]

@denji Yeah, using an indexer like Hound would be the ideal solution. Mine is just a temporary one until someone better at Go gives it a shot. Thankfully it works. :)

[ericbsdr]

@tomas could you explain how you implemented git grep search ? Could you make a fork please ?

[tomas]

@ericbsdr Sure, I have to clean up some stuff first though. I wrote a simple shell script that walks through the repo root path and runs git grep over each repo and returns those results. Then I parse those results and render them in the template. Definitely not rocket science. :)

[ericbsdr]

Thanks @tomas . I'm not a big developer but i'll try to understand how you got this working.

[andreasotto]

+1

We need search issues

[KangoV]

I'm coming from a Java background and have done a lot of work with Lucene, which is awesome fast. Is there a Go version of Lucene? I really need to get into Go as it sound pretty neat.

[timshannon]

That would be bleve which is awesome.

On Dec 13, 2016 6:47 PM, "Darren Bell" [e-mail] wrote:

I'm coming from a Java background and have done a lot of work with Lucene,
which is awesome fast. Is there a Go version of Lucene? I really need to
get into Go as it sound pretty neat.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/gogits/gogs/issues/13#issuecomment-266908876, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ABRNNRE7MgRKqFxL0fbXT_r0u3yN7Jz7ks5rHzy0gaJpZM4BoKca
.

[laoshaw]

please add search to the code, searching repo and username are great, but not enough...

[matfeb]

+1

[ChangePlaces]

What exactly are the stumbling blocks with this issue? Maybe the community can help out

[matfeb]

There is no built in function to look for a string pattern in Gogs. You can search inside repos or users but not inside the code as GitLab can do.

IMO it's a MUST HAVE feature for this kind of tool.

One way is to implement etsy/hound inside Gogs or CodeSearch from Google.

I'm trying to use etsy/hound outside Gogs with two docker containers but I'm dealing with SSH Keys issues at the moment and it's not very user friendly.

etsy/hound use Go, it could be easier than implementing CodeSearch IMO.

[timshannon]

FYI, here's the corresponding issue in gitea:

https://github.com/go-gitea/gitea/issues/1102

which I assume will have a better chance of getting done, than in gogs.

[atomi]

if you want to use hound/gogs just mount your private key when you run the hound container and link your gogs cotainer to it then you can use ssh-keyscan to add your gogs container to the known_hosts file in the hound container

docker exec houndcontainer sh -c 'ssh-keyscan -t rsa -p 2222 gogscontainer > /root/.ssh/known_hosts'

[mwaeckerlin]

Please don't forget a global search: search repository and user names. I got 60+ repos and handling them is a pain in the ass…

[unknwon]

@mwaeckerlin I don't understand, isn't explore page can already do this? https://try.gogs.io/explore/repos

[mwaeckerlin]

@My fault, I have to apologize, I didn't see that. Either I'm blind or it was again the proxy? Anyway, yes, that's it!

The other (non related) point is: Structuring the projects would be nice. What I did to overcome this is to use organizations, one for docker-images, admin-scripts, nodejs-libs and c++-libs. Some kind of relationsship between the projects would be nice, such as parent-/ child-/ sub-projects.

This is confusing: https://mrw.sh/explore/repos
So this is a bit better: https://mrw.sh/explore/organizations

Any other idea?

[unknwon]

@mwaeckerlin maybe this is what you're looking for https://github.com/gogits/gogs/issues/1253.

[mwaeckerlin]

Now I see my previous problem: The search bar is only visible in the repos-view. I'd suggest to add a global search bar in the top header line, that is accessible from everywhere. (Similar to the one in GitHub.)

[hellochenwang]

With convenient code/issue search functionality, Gogs could become an internal knowledge base/dictionary.

Developers will use Gogs a lot more frequent than just a bug tracking and source control system.

[ghost]

+1 this would be really awesome. I think that this is a "must have feature"

[ghost]

@tomas did you create a fork with your search string version?

[tomas]

@giudon I didn't because I had very little time ATM, and then when Gitea was born I thought someone would be able to implement this "the right way" (as suggested by @denji).

If there's still any interest I can push it somewhere, though. :)

Edit: Looks like Gitea did it using Bleve.

[ghost]

Yes @tomas , I can confirm that is already implemented in Gitea, but only on master branch (actually is in testing). ASAIK It will be released officially from few weeks. Thank you!

[tomas]

Sure, no problem @giudon.

I just pushed my dirty little trick in case anyone's interested.

[ghost]

@ChangePlaces My thoughts was the same few months ago...but now...after I tried few SCM tools (gogs, gitea, gitlab, gitbucket), I can confirm to myself that even if gogs it's the more "stable", gitea cover all my needs (it's fast, developed continuosly, have an active community...). I am sad about this, because gogs was my first choice, but I don't have time to wait, I need a good and "in develoment tool" right now

[BreakfastSerial]

Are there any news about this? Is this not planned, planned, in progress or maybe already in testing?

I feel like this is a critical feature and I don't want to grep over all cloned repositories.
We use Gogs as an locally hosted version controlled but also as a knowledge base.

This has been an issue for quite some years and I see no development.

[unknwon]

@BreakfastSerial Hi, it's still in plan stage. :sleepy:

[rokuu]

ripgrep, is a very fast search tool, which supports .gitignore rules and configuration files. This might help.

[IssueHuntBot]

@0maxxam0 has funded $10.00 to this issue.

---

• Submit pull request via IssueHunt to receive this reward.
• Want to contribute? Chip in to this issue via IssueHunt.
• Checkout the IssueHunt Issue Explorer to see more funded issues.
• Need help from developers? Add your repository on IssueHunt to raise funds.

[unknwon]

Mostly likely the code search feature will be supported as a native integration to Sourcegraph APIs, and uses Sourcegraph for searching the code.