Cidram: Recaptcha bug

[mikeruss1]

unusual sequence of events:
1 user blocked and offered recaptcha
2 fails recaptcha
3 reaccesses the same page and offered recaptcha
4 passes recaptcha and page successfully processed
5 requests another page via Google Matched content and gets a 503
6 repeats 5 several times

I think I have seen this before, not sure if you reckon its a bug, but it seems the original failed recaptcha is overriding the later success ?

thanks .. Mike

[mikeruss1]

unless its a variation of the bug you fixed a while ago where passing recaptcha didnt allow subsequent pages ?

[Maikuolan]

(Will investigate this shortly).

[Maikuolan]

Sorry for the delay. I haven't been able to replicate the problem on my end yet. Do you have any access logs available for the specific block events in question (e.g., for when the user was initially blocked and offered a reCAPTCHA instance)? Seeing why they were blocked might help shed some more light on what's going on here. Also, were any of the new block events that occurred after passing reCAPTCHA logged?

[Maikuolan]

unless its a variation of the bug you fixed a while ago where passing recaptcha didnt allow subsequent pages ?

I wouldn't think so.. but not sure. Probably, we'll need to analyse the logs to figure out more, I think.

Also (another thing I'd thought of, but hadn't asked in my previous reply): For the user in question, does their IP address change between when they first pass reCAPTCHA and when they get subsequently blocked again?

[mikeruss1]

the recaptcha log, the block log, the access logs
the IP and UA all look the same to me ?

IP Address: 110.54.201.111 - Date/Time: Sat, 09 Jun 2018 09:39:15 +0100 - reCAPTCHA State: Failed!
IP Address: 110.54.201.111 - Date/Time: Sat, 09 Jun 2018 09:44:16 +0100 - reCAPTCHA State: Passed!

ID: 1753
Date/Time: Sat, 09 Jun 2018 09:37:54 +0100
IP Address: 110.54.201.111
Referrer: https://r.search.yahoo.com/_ylt=AwrPiFXKkRtbQwQAMwOzRwx.;_ylu=X3oDMTBycWJpM21vBGNvbG8Dc2czBHBvcwMxBHZ0aWQDBHNlYwNzcg--/RV=2/RE=1528562250/RO=10/RU=https%3a%2f%2fworld-war.co.uk%2f/RK=2/RS=DQcuo2Yl0w8h82uNJPN5eMclkr8-
Signatures Count: 1
Signatures Reference: botbrowsers_cidram.inc:L42
Why Blocked: Robot probe or scraper (C/IE/FF/NS/O/S)!
User Agent: Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1626.1 Safari/537.36
Reconstructed URI: https://xxxxx.co.uk/
reCAPTCHA State: Enabled.

ID: 1754
Date/Time: Sat, 09 Jun 2018 09:39:15 +0100
IP Address: 110.54.201.111
Referrer: https://xxxx.co.uk/
Signatures Count: 1
Signatures Reference: botbrowsers_cidram.inc:L42
Why Blocked: Robot probe or scraper (C/IE/FF/NS/O/S)!
User Agent: Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1626.1 Safari/537.36
Reconstructed URI: https://xxxxx.co.uk/
reCAPTCHA State: Failed!

following is log for the initial block ...

110.54.201.111 - - [09/Jun/2018:09:37:53 +0100] "GET / HTTP/1.0" 403 4608 "https://r.search.yahoo.com/_ylt=AwrPiFXKkRtbQwQAMwOzRwx.;_ylu=X3oDMTBycWJpM21vBGNvbG8Dc2czBHBvcwMxBHZ0aWQDBHNlYwNzcg--/RV=2/RE=1528562250/RO=10/RU=https%3a%2f%2fworld-war.co.uk%2f/RK=2/RS=DQcuo2Yl0w8h82uNJPN5eMclkr8-" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1626.1 Safari/537.36"

following is the failed recaptcla ...

110.54.201.111 - - [09/Jun/2018:09:39:15 +0100] "POST / HTTP/1.0" 403 4415 "https://xxxx.co.uk/" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1626.1 Safari/537.36"

the passed recaptcha ...

110.54.201.111 - - [09/Jun/2018:09:44:16 +0100] "POST / HTTP/1.0" 200 26043 "https://xxxx.co.uk/" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1626.1 Safari/537.36"

first 503 ...

110.54.201.111 - - [09/Jun/2018:09:44:46 +0100] "GET /spin.php3?vote=9&id=2004&seq=0&rank=44 HTTP/1.0" 503 - "https://xxxx.co.uk/" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1626.1 Safari/537.36"

second 503

110.54.201.111 - - [09/Jun/2018:09:45:30 +0100] "GET /battleships.php3 HTTP/1.0" 503 - "https://googleads.g.doubleclick.net/pagead/ads?client=ca-pub-3172139329835489&output=html&h=600&slotname=3859669572&adk=4190934876&adf=426771864&w=1200&crui=image_stacked&cr_col=4&cr_row=2&fwrn=4&lmt=1528505039&loeid=10583696&rafmt=9&guci=2.2.0.0.2.2.0&format=1200x600&url=https%3A%2F%2Fworld-war.co.uk%2F&flash=11.8.800&fwr=0&wgl=1&adsid=NT&dt=1528533839287&bpp=62&bdt=13560&fdt=62&idt=62&shv=r20180606&cbv=r20180604&saldr=aa&abxe=1&prev_fmts=728x90%2C336x280&prev_slotnames=7518357502&correlator=7186043015169&frm=20&pv=1&ga_vid=1581310355.1528533832&ga_sid=1528533832&ga_hid=1757141428&ga_fc=1&iag=0&icsg=704290750126&dssz=53&mdo=0&mso=0&u_tz=480&u_his=3&u_java=1&u_h=768&u_w=1366&u_ah=728&u_aw=1366&u_cd=24&u_nplug=4&u_nmime=7&adx=76&ady=3321&biw=1351&bih=652&scr_x=0&scr_y=0&eid=10573695%2C10593696%2C21061122%2C368226401&oid=3&ref=https%3A%2F%2Fxxxx.co.uk%2F&rx=0&eae=0&fc=528&brdim=0%2C0%2C0%2C0%2C1366%2C0%2C1366%2C728%2C1366%2C667&vis=1&rsz=%7C%7CeEbr%7C&abl=CS&ppjl=u&pfx=0&fu=1168&bc=1&jar=2018-5-18-0&ifi=4&xpc=7zoSpIqRZN&p=https%3A//xxxxx.co.uk&dtd=140" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1626.1 Safari/537.36"

[mikeruss1]

another odd thing is that the :09:44:46 block is a 503. I am setup with the number of 403s allowed to be 4 before a permanent block. There are only 2 blocks so it should have got a 403. So it seems like the initial recaptcha failure is being remembered ?

[Maikuolan]

Definitely odd.

What's the current value for ban_override in your configuration?

[mikeruss1]

503

[mikeruss1]

one thought - the 09:44:46 +0100] "GET /spin.php3

this is an Ajax call - maybe this is confusing things ?

[Maikuolan]

Could be that. Any idea whether the call is being encoded properly before being sent by the browser? Normally, non-alphanumeric characters should be encoded before a request is sent, but we've sometimes seen cases where this doesn't occur properly with AJAX calls in some older browsers, devices, etc. In such cases, there may be additional signatures which get triggered a result of the AJAX request, specifically relating to requests containing improperly encoded characters. In theory, if any of these triggered signatures were any of the older "insta-ban" signatures included with some of the optional modules, they could result in the IP being "banned", thus explaining the 503, and explaining why the completed reCAPTCHA instance becomes insufficient for allowing the user in.

That said though, I see nothing in "spin.php3?vote=9&id=2004&seq=0&rank=44" specifically which would cause such a thing. Maybe in the request body itself..? :-/

I'll dig through all of the old insta-ban signatures to try to figure which signatures might be candidates for this type of problem occurring and let you know what I find; at that point, we should be able to either confirm or eliminate this theory by checking whether the relevant signatures are actually enabled for your particular installation, and if they are, disabling them temporarily, to see whether the problem persists.

[Maikuolan]

(Aside from that though, I can't think of any reasons theoretically why AJAX would make any difference. Anyhow, I'll reply again shortly).

[mikeruss1]

it wouldnt be a straight-forward encoding issue, I get 1000s of these a day. It must be something in combination with the preceeding recapthca's

[Dibbyo456]

@mikeruss1 do you still reproduce the same issue?
Because I also used Matched Content, don't know if users encountered the same issue.

[mikeruss1]

it is unusual to get a failed recaptcha - still waiting for a repeat

[Maikuolan]

@mikeruss1 I know it's been a while now since either of us really looked too closely at this issue. Can I check with you quickly whether you've seen any more instances of this problem occur recently?

[mikeruss1]

Hi Caleb

On vacation, likely 2 weeks before I can reply

Cheers Mike

[Maikuolan]

No problem at all, and no rush. Enjoy your vacation. :-)

[mikeruss1]

I havent noticed this sequence again, suggest you put it to sleep ?

[DanielRuf]

Ok, let's close it then.