Fasthub: Abuse detection mechanism triggered for searches

[ishitatsuyuki]

FastHub Version: 4.6.4
Android Version: 7.0 (SDK: 24)
Device Information:

• Manufacturer: HUAWEI
• Brand: HUAWEI

- Model: P10 Plus

After performing a combination of repository, path and code (files) search, it will trigger the abuse detection and the IP-token combination is banned from searching files for a while.

The amount of searches before getting banned (5 on average) is significantly less than what is documented as search rate limit (30 req/min).

You have triggered an abuse detection mechanism. Please wait a few minutes before you try again.

The possible causes are documented at here. The reasons I can think of are:

• FastHub may be repeatedly performing requests in background, like the status API.
• FastHub may be retrieving the pagination too rapidly.

[k0shk0sh]

Search API has its own limit and is not tied to the other apis.

[ishitatsuyuki]

Yes it is and as I mentioned it's 30/min compared to the normal 5000/min. We're not triggering a documented rate limit but their private abuse detection mechanism.

[k0shk0sh]

Well, I can't tell what's the problem here honestly, I never encountered such behavior.

Maybe you do alot of pagination in the search? This could be the cause as well. I can't help you tho as idk what is going.

[ishitatsuyuki]

Please follow the following reproduction steps:

1. Search repo for FastHub
2. Open this repo
3. Search the code with generic keyword (e.g. get, API, kotlin) for 10 times

I got banned this time very quickly.

[k0shk0sh]

10 times straight?? or there is some seconds in between?

[ishitatsuyuki]

You probably should do the search as fast as you can type, as it's likely still below the rate limit.

By the way, if you can't reproduce, I would file a case with GitHub support and post back with what they says.

[k0shk0sh]

I could produce it but after many requests and selecting All from the spinner. which is legit in my opinion. its up to you if you like to fire a case to GitHub :)

[ishitatsuyuki]

All GitHub suggested is to respect the Retry-After header:

In general, the best thing to do is to spread your requests over time and check the Retry-After header in case you do hit the secondary limits, so that you know when you can resume making requests.

We should show "retry after x seconds" when the header is present, and optionally fail all requests internally instead of requesting GitHub server (so the count doesn't get worse).

[yakov116]

@ishitatsuyuki rather don't try make the server fail and you won't have any issues at all

[ishitatsuyuki]

@yakov116 FastHub can perform multiple requests or implicitly request the server, such as fetching status.

[yakov116]

@ishitatsuyuki but as you understand this is a non standard thing. You are more than welcome to submit a pr

[ishitatsuyuki]

What? Non-standard? Please elaborate.

[yakov116]

@ishitatsuyuki this is issue number #1619 you are the first person I know that has run into this issue

[ishitatsuyuki]

I believe the root cause of this is because in the "search everything" interface we load the four types of search result concurrently. Can you reopen this issue?

[k0shk0sh]

I doubt that this is the cause :) for instance, issues and pr tabs in main screen are also search and in some other places as well where there is no API i use search api.

[ishitatsuyuki]

To elaborate, I have some more points of that being a cause:

• The abuse rate limit is triggered in the "search everything" panel the first time in most of my cases. The rest of triggered in repository code search.
• The 4x amplification makes sense when it comes to hitting 30req/min rate limit.
• GitHub recommends against concurrent requests.
  > Make requests for a single user or client ID serially. Do not make requests for a single user or client ID concurrently.

[yakov116]

@ishitatsuyuki its interesting since you are the only one reporting this. I am using FastHub for a year now and I have never encountered this issue!

[ishitatsuyuki]

@yakov116 Please stop repeating the fact that this is "interesting". It does not contribute to discussion. The only thing I can say is that, it affects me.

[jensdpersson]

This looks like the thing I just got. First interaction with github since at least 24 hours, I search for a specific project (not having logged in first) and get the "You have triggered an abuse detection mechanism. Please wait a few minutes before you try again." message. It happened before some weeks or so back in the same way. Go to github, search for a project, get denied.

[k0shk0sh]

This changed in v5, please be patient!

[hpg-mheck]

I can trigger this issue with a single, manual search for "fabric-composer" on the first try with no special crap. This is a vanilla, manual search from Chrome on Linux. This is a real pain in the ass.

[k0shk0sh]

Guys, this has been fixed in v5 as im using graphql for that, so please be patient.

[k0shk0sh]

[Kaju-Bubanja]

Why is this closed? This is still an issue. Trying to search something for the first time already triggered this for me.

[alogim]

I am experiencing the same issue, and I am doing one search every 2 or 3 seconds...

[bo3b]

Just did ONE search on a clean install of Win10 1903, using Chrome browser, and it triggered.

No idea why this would be closed. I had to login to be able to do anything.