Sherlock: Display response times in verbose mode

[nareddyt]

Feature Request

It would be nice to display the response time for each request.

Why?

• It would help us determine which sites cause the program to slow down. If a site is very slow and non-critical, we could remove it to speed up Sherlock.
• The more metrics the merrier! Has very little overhead, so why not :)

User Experience

We would only display this information in verbose mode (-v flag). The information can be displayed in each output line as:

[+] [78 ms] Quora: https://www.quora.com/profile/nareddyt

Furthermore, these response times can be exported in the CSV.

Implementation Details

We would just need to store the time reach request was created in the net_info or results_site dictionaries. Then when the request is done, we would note the end time.

Note there is some math involved in calculating times. It's not a simple end-start because the result for each response is extracted in the order of requests (not as soon as a request finishes). This is something I still need to think about...

[Czechball]

Sounds good. Maybe then we could sort the sites by their response time and check the fastest ones first.

[TheYahya]

Sounds great @nareddyt 😊, Let me know if you going to give it a hand ☺️

[nareddyt]

Here is my initial prototype of this feature. I was able to make use of requests-futures' built-in hooks API to compute the response times per request in a parallel fashion. This introduces very little overhead.

For now, I will keep this feature simple and only display the response times in the console / csv. We can work on more advanced uses of these metrics (like the suggestion by @Czechball) once this initial feature is merged.

I'll send out a PR once I clean up my code and also thoroughly test it :)

[TheYahya]

Seems great @nareddyt, can't wait to use it ☺️