Sherlock: python3 -m unittest tests.all --verbose occurs errors without changes
Envs
$ python3 --version
Python 3.7.3
Error Log
$ python3 -m unittest tests.all --verbose
test_detect_false_via_message (tests.all.SherlockDetectTests)
Test Username Does Not Exist (Via Message). ... ok
test_detect_false_via_response_url (tests.all.SherlockDetectTests)
Test Username Does Not Exist (Via Response URL). ... ok
test_detect_false_via_status_code (tests.all.SherlockDetectTests)
Test Username Does Not Exist (Via Status Code). ... ok
test_detect_true_via_message (tests.all.SherlockDetectTests)
Test Username Does Exist (Via Message). ... ok
test_detect_true_via_response_url (tests.all.SherlockDetectTests)
Test Username Does Exist (Via Response URL). ... ok
test_detect_true_via_status_code (tests.all.SherlockDetectTests)
Test Username Does Exist (Via Status Code). ... ok
test_coverage_false_via_message (tests.all.SherlockSiteCoverageTests)
Test Username Does Not Exist Site Coverage (Via Error Message). ... test_coverage_false_via_response_url (tests.all.SherlockSiteCoverageTests)
Test Username Does Not Exist Site Coverage (Via Response URL). ... ok
test_coverage_false_via_status (tests.all.SherlockSiteCoverageTests)
Test Username Does Not Exist Site Coverage (Via HTTP Status). ... skipped "Skipping Username 'noonewouldeverusethis7' available on Site 'Otzovik': Site returned error status."
test_coverage_total (tests.all.SherlockSiteCoverageTests)
Test Site Coverage Is Total. ... ok
test_coverage_true_via_message (tests.all.SherlockSiteCoverageTests)
Test Username Does Exist Site Coverage (Via Error Message). ... test_coverage_true_via_response_url (tests.all.SherlockSiteCoverageTests)
Test Username Does Exist Site Coverage (Via Response URL). ... ok
test_coverage_true_via_status (tests.all.SherlockSiteCoverageTests)
Test Username Does Exist Site Coverage (Via HTTP Status). ... skipped "Skipping Username 'blue' claimed on Site 'Otzovik': Site returned error status."
======================================================================
FAIL: test_coverage_false_via_message (tests.all.SherlockSiteCoverageTests) [Checking Username 'noonewouldeverusethis7' available on Site 'Euw']
Test Username Does Not Exist Site Coverage (Via Error Message).
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/asuha/sherlock/sherlock/tests/base.py", line 138, in username_check
result['status'].status)
AssertionError: <QueryStatus.AVAILABLE: 'Available'> != <QueryStatus.CLAIMED: 'Claimed'>
======================================================================
FAIL: test_coverage_false_via_message (tests.all.SherlockSiteCoverageTests) [Checking Username 'noonewouldeverusethis7' available on Site 'EyeEm']
Test Username Does Not Exist Site Coverage (Via Error Message).
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/asuha/sherlock/sherlock/tests/base.py", line 138, in username_check
result['status'].status)
AssertionError: <QueryStatus.AVAILABLE: 'Available'> != <QueryStatus.CLAIMED: 'Claimed'>
======================================================================
FAIL: test_coverage_false_via_message (tests.all.SherlockSiteCoverageTests) [Checking Username 'noonewouldeverusethis7' available on Site 'Tinder']
Test Username Does Not Exist Site Coverage (Via Error Message).
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/asuha/sherlock/sherlock/tests/base.py", line 138, in username_check
result['status'].status)
AssertionError: <QueryStatus.AVAILABLE: 'Available'> != <QueryStatus.CLAIMED: 'Claimed'>
======================================================================
FAIL: test_coverage_false_via_message (tests.all.SherlockSiteCoverageTests) [Checking Username 'noonewouldeverusethis7' available on Site 'VirusTotal']
Test Username Does Not Exist Site Coverage (Via Error Message).
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/asuha/sherlock/sherlock/tests/base.py", line 138, in username_check
result['status'].status)
AssertionError: <QueryStatus.AVAILABLE: 'Available'> != <QueryStatus.CLAIMED: 'Claimed'>
======================================================================
FAIL: test_coverage_false_via_message (tests.all.SherlockSiteCoverageTests) [Checking Username 'noonewouldeverusethis7' available on Site 'Wattpad']
Test Username Does Not Exist Site Coverage (Via Error Message).
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/asuha/sherlock/sherlock/tests/base.py", line 138, in username_check
result['status'].status)
AssertionError: <QueryStatus.AVAILABLE: 'Available'> != <QueryStatus.CLAIMED: 'Claimed'>
======================================================================
FAIL: test_coverage_false_via_status (tests.all.SherlockSiteCoverageTests) [Checking Username 'noonewouldeverusethis7' available on Site 'Medium']
Test Username Does Not Exist Site Coverage (Via HTTP Status).
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/asuha/sherlock/sherlock/tests/base.py", line 138, in username_check
result['status'].status)
AssertionError: <QueryStatus.AVAILABLE: 'Available'> != <QueryStatus.CLAIMED: 'Claimed'>
======================================================================
FAIL: test_coverage_true_via_message (tests.all.SherlockSiteCoverageTests) [Checking Username 'blue' claimed on Site 'Repl.it']
Test Username Does Exist Site Coverage (Via Error Message).
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/asuha/sherlock/sherlock/tests/base.py", line 138, in username_check
result['status'].status)
AssertionError: <QueryStatus.CLAIMED: 'Claimed'> != <QueryStatus.AVAILABLE: 'Available'>
======================================================================
FAIL: test_coverage_true_via_status (tests.all.SherlockSiteCoverageTests) [Checking Username 'blue' claimed on Site 'Oracle Community']
Test Username Does Exist Site Coverage (Via HTTP Status).
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/asuha/sherlock/sherlock/tests/base.py", line 138, in username_check
result['status'].status)
AssertionError: <QueryStatus.CLAIMED: 'Claimed'> != <QueryStatus.AVAILABLE: 'Available'>
======================================================================
FAIL: test_coverage_true_via_status (tests.all.SherlockSiteCoverageTests) [Checking Username 'blue' claimed on Site 'Vero']
Test Username Does Exist Site Coverage (Via HTTP Status).
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/asuha/sherlock/sherlock/tests/base.py", line 138, in username_check
result['status'].status)
AssertionError: <QueryStatus.CLAIMED: 'Claimed'> != <QueryStatus.AVAILABLE: 'Available'>
======================================================================
FAIL: test_coverage_true_via_status (tests.all.SherlockSiteCoverageTests) [Checking Username 'red' claimed on Site 'Xbox Gamertag']
Test Username Does Exist Site Coverage (Via HTTP Status).
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/asuha/sherlock/sherlock/tests/base.py", line 138, in username_check
result['status'].status)
AssertionError: <QueryStatus.CLAIMED: 'Claimed'> != <QueryStatus.AVAILABLE: 'Available'>
======================================================================
FAIL: test_coverage_true_via_status (tests.all.SherlockSiteCoverageTests) [Checking Username 'cupparober' claimed on Site 'Filmogs']
Test Username Does Exist Site Coverage (Via HTTP Status).
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/asuha/sherlock/sherlock/tests/base.py", line 138, in username_check
result['status'].status)
AssertionError: <QueryStatus.CLAIMED: 'Claimed'> != <QueryStatus.AVAILABLE: 'Available'>
======================================================================
FAIL: test_coverage_true_via_status (tests.all.SherlockSiteCoverageTests) [Checking Username 'adam' claimed on Site 'fixya']
Test Username Does Exist Site Coverage (Via HTTP Status).
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/asuha/sherlock/sherlock/tests/base.py", line 138, in username_check
result['status'].status)
AssertionError: <QueryStatus.CLAIMED: 'Claimed'> != <QueryStatus.AVAILABLE: 'Available'>
======================================================================
FAIL: test_coverage_true_via_status (tests.all.SherlockSiteCoverageTests) [Checking Username 'green' claimed on Site 'svidbook']
Test Username Does Exist Site Coverage (Via HTTP Status).
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/asuha/sherlock/sherlock/tests/base.py", line 138, in username_check
result['status'].status)
AssertionError: <QueryStatus.CLAIMED: 'Claimed'> != <QueryStatus.AVAILABLE: 'Available'>
----------------------------------------------------------------------
Ran 13 tests in 387.735s
FAILED (failures=13, skipped=2)
$ python3 sherlock.py noonewouldeverusethis7
[*] Checking username noonewouldeverusethis7 on:
[+] BuzzFeed: https://buzzfeed.com/noonewouldeverusethis7
[+] Euw: https://euw.op.gg/summoner/userName=noonewouldeverusethis7
[+] EyeEm: https://www.eyeem.com/u/noonewouldeverusethis7
[+] Medium: https://medium.com/@noonewouldeverusethis7
[+] Tinder: https://www.gotinder.com/@noonewouldeverusethis7
[+] VK: https://vk.com/noonewouldeverusethis7
[+] VirusTotal: https://www.virustotal.com/ui/users/noonewouldeverusethis7/trusted_users
[+] Wattpad: https://www.wattpad.com/user/noonewouldeverusethis7
Is this something that hasn't been intentionally fixed?
Should I make PR to fix it?
bug
Asuha-a
All 9 comments
@Asuha-a Thank you for the detailed report. I am a little confused with your issue. Are you reporting false positives or is your issue about the error given out by the unittest? I know the unittest fails at times because of false positives. Sherlock has unfortunately been facing a lot of false positives and I havent had the time to fix them.
sdushantha
on 1 Nov 2020
@sdushantha
I reported the unit test failure. As you know this failure is because somebody's creates accounts using the name 'noonewouldeverusethis7'.
How about using a long random string instead of 'noonewouldeverusethis7' to test?
Then false positives will be resolved.
Asuha-a
on 2 Nov 2020
How about using a long random string instead of 'noonewouldeverusethis7' to test?
This would be a little difficult to do because some sites have a specific criteria for usernames. For example there might be a max/min length or certain characters might not be allowed. All the sites in the tests do not have the same criteria. Therefore we would have to either generate a random username for each site or manually set a username for each site which we would of course have to update if the username got registered.
sdushantha
on 3 Nov 2020
@sdushantha
Understood. Thank you for taking the time to explain that.
You can close this issue if you don't need it.
Asuha-a
on 4 Nov 2020
@Asuha-a It seems like you are correct. There is something wrong with the tests.
I unfortunately do not know much about unit testing, but I vaguely remember @hoadlck saying somewhere that the tests are currently unreliable? I could be wrong about that.
But this is definitely an issue we need to fix.
sdushantha
on 5 Nov 2020
@sdushantha
There are 2 types of unit test failure, that username_claimed and username_unclaimed.
They have a different cause. (I'm not sure)
Will we fix both in this issue?
Asuha-a
on 6 Nov 2020
@Asuha-a Yes we would have to fix both of them. I dont know much about unit tests, so it might take a while before i get it fixed because I need to find some time to read about it.
sdushantha
on 6 Nov 2020
👍1
These are all failures because of false positives and false negatives. There may be a few that are happening because an IP address get banned by a given site.
For example, consider the following error message:
FAIL: test_coverage_true_via_status (tests.all.SherlockSiteCoverageTests) [Checking Username 'green' claimed on Site 'svidbook']
Test Username Does Exist Site Coverage (Via HTTP Status).
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/asuha/sherlock/sherlock/tests/base.py", line 138, in username_check
result['status'].status)
AssertionError: <QueryStatus.CLAIMED: 'Claimed'> != <QueryStatus.AVAILABLE: 'Available'>
The site svidbook should show that username "green" has been claimed. But instead, Sherlock is saying that it is available.
It is also possible that the username which was included in the tests has changed had been registered by someone, or deleted. It takes some research to determine what is what.
hoadlck
on 6 Nov 2020
👍1
@hoadlck Thank you, that helped me clear up the confusion :)
sdushantha
on 6 Nov 2020
Was this page helpful?
0 / 5 - 0 ratings
Related issues
ptalmeida
路
3Comments
nareddyt
路
4Comments
senthilnayagam
路
4Comments
x0rzkov
路
5Comments
Berghopper
路
6Comments