Sherlock: Search by email

[conilas]

I really love this tool.

But imagine a world where we can search by email instead of the nick name.

[sdushantha]

Yes, that would be very cool, but how would we do that? I dont think it is possible for us to see if an email is registered.

[sdushantha]

I think I have found a way to do that.
For each site, we would have to try do the _forgot my password_ option and put in the email and see if the response is _This email is not associated with any account_ or something similar.

I also think that if we were going to do this, we would have to have each site have its own file/script like I mentiond in issue #13

[hoadlck]

I think I have found a way to do that.
For each site, we would have to try do the _forgot my password_ option and put in the email and see if the response is _This email is not associated with any account_ or something similar.

That would be very uncool. Way over the line. Some of those password reset forms would actually reset the user's password. Each time a scan happened for a given user, it might require them to log in again.

[sdushantha]

@hoadlck You are right, that is very uncool. I was not aware that this would affect the owner of the username.

[eddvrs]

Ethics aside, many websites protect their users by not indicating whether an email address is related to any of their users, so this still wouldn't be reliable.

[rajdeepbharati]

We will need this kind of a People API: https://pipl.com/corp/lp/find-social-profiles-by-email-address/.
It won't show all the results since all social sites don't allow us access using email, but it we can have 2 options: search using username, and search using email

This API isn't open source, so we should consider building something like this.

[JonanOribe]

Another way to improve the project and achieve more info about the users would be to scrap the websites where the user´s name appears. We can use tools like Scrapy, for example, to get data (email, city, interest) of users and at the end print a profile.

If you try against your github profile, you can pick up your location and email, without cross any line, because our profiles are public. Making the same in other websites, we can gather up different information about the user just scraping the public profiles.

[nareddyt]

Just wanted to bring up a related discussion in Issue #40. Particularly, this quote by @hoadlck is relevant:

There are approximately 5 bazillion Internet Forums out there, and if all of them were added to Sherlock then it is really going to add much churn. It would be like trying to release a Python module that had a snapshot of a search engine index embedded inside of it. It just would not scale.

Would adding options to find users by profile information or email transform Sherlock into a search engine. If so, is that something we want to do?

I have not invested enough time into this issue to have an opinion on this solution, I just wanted to bring up this similar discussion. It's up to @TheYahya, @hoadlck, and you guys to decide on the scope!

[TheYahya]

I'm agree with you @nareddyt. It's not going to be scale. Also from a point, we should decide which type of sites & social networks going to be in sherlock. or maybe categorize list of site or anything.

[Czechball]

Categorization would be excellent. I was thinking of categories by countries...

[hoadlck]

This is something that you get with databases. There are many ways to categorize content, and there is no way to have one definitive categorization. You could use developer, music, books, games, picture, eCommerce, blog, hosting, country...all are examples of ways that someone might want to sort.

As I said in #40, if the scope of Sherlock is going to be so wide, I do not see how the current approach will scale. There will be an endless flurry of pull requests adding one more site. And, the list of sites will be so long that it will be hard for people to absorb the information.

I could see it morphing into a web application. Or, if we were really careful in designing the schema, Sherlock could download the JSON (or database) as the first action, and determine the queries from there.

[TheYahya]

Web application idea seems good, In fact with a good interface it would be much easier for everyone to use sherlock, Would be easier to scale and categorize, caching, etc. I think in this case sherlock would be a core module of the web application.

[nareddyt]

We should make a different issue for "web server" related discussion, but I think that would be a good long term goal. It would require a bit of refactoring to make turn sherlock into an API (either web API or library API).

After thinking through the original discussion (in this issue), I am worried about scalability if we were to implement any of the solutions above.

[crock]

I think the focus of Sherlock should remain as a CLI application since that’s what it started as.

I’ve built and maintained a REST API for social media availability checking off nearly a year now. I’d love some contributions.
https://github.com/checker/api

[sdushantha]

@conilas Check out https://github.com/iojw/socialscan which claims it performs with 100% accuracy. It looks like a really nice tool and it might be something we could look into adding to Sherlock once we are done with the refractoring.