Cidram: bypass rules not working anymore

[Dibbyo456]

While we were discussing about the issue #54 where legitimate BingBots were being blocked. I found myself in another issue. Before that let me explain the situation first.

Back in the December, 2017 I added some custom bypass rules for CIDRAM. But in January due to some misconception I sort of disabled CIDRAM (by enabling maintenance mode). But in February 25, 2018 I again enabled it.

Today when I was looking into my custom bypass rules, I found this.

/** Prevents execution from outside of CIDRAM. */
if (!defined('CIDRAM')) {
    die('[CIDRAM] This should not be accessed directly.');
}

/** Inherit bypass closure (see functions.php). */
$Bypass = $CIDRAM['Bypass'];

/** Bingbot bypass. */
$Bypass(
    $CIDRAM['BlockInfo']['SignatureCount'] &&
    strpos($CIDRAM['BlockInfo']['UALC'], 'bingbot') !== false,
    'Bingbot Bypass'
);

/** BingPreview bypass. */
$Bypass(
    $CIDRAM['BlockInfo']['SignatureCount'] &&
    strpos($CIDRAM['BlockInfo']['UALC'], 'bingpreview') !== false,
    'BingPreview Bypass'
);

So, as you can probably guess any user-agents having bingbot or bingpreview will pass.

Today I reported several IPs which were blocked even they are legit bingbots.
Now, Let's assume they are not legit bingbots. But even they are not, my custom bypass rules should allow them pass, right? So, why it did not?

I also tested 1 of the ip with IP Tester, with user-agent of bingpreview, it showed me pass, but why it was blocked in real time?

[Maikuolan]

What do the logs show as the reason for the block occurring (e.g., still Azure, or something else, like the search engine verification check)?

[Dibbyo456]

Azure

[Maikuolan]

Wondering at this point whether I should just bypass the entirety of the Azure signature section whenever BingBot or BingPreview is used in the UA, instead of bothering with search engine verification when Azure signatures are triggered..? Good chance it'll let some things through that way, but considering the difficulties we've seen lately with false positives relating to Azure and BingBot being mistakenly blocked, I'm wondering whether it might be worthwhile.

The other option is just dropping them from the main/default signature files entirely, seeing as they're essentially a stopgap measure for users that don't want to or haven't yet figured out how to download and install Macmathan's Azure signature file (his is updated more frequently, and I would generally recommend for users to install it, but of course, when they do, then the Azure signature section in the main/default signature files becomes redundant at that point anyway). 🤔

[Maikuolan]

Working on an update at the moment. Once it's been committed, the problem should be fixed internally and you won't need these bypasses anymore.

[Dibbyo456]

But why bypasses rules did not work?

[Maikuolan]

TBH, I'm not sure. They should work, in theory. Would need to take a closer look at the installation; Could be something external interfering with it, maybe. The incoming update should fix it though, in any case.

[Maikuolan]

Committed. :-)

Old bypasses shouldn't be needed anymore. Let me know if the problem persists.

[ghost]

Just a quick note regarding my Azure list.

If you install it, then remember to also install the "Cloud Services Search Engine Bypass list" which is an extra module to cope with issues like these. Just thought it was worth mentioning :)

Added to clarify: It will only work together with my list, and will do nothing for the built in checks.

[Dibbyo456]

If you install it, then remember to also install the "Cloud Services Search Engine Bypass list" which is an extra module to cope with issues like these. Just thought it was worth mentioning :)

Hi, I wants to install the module but I'm a little confused. If I install this module which is available on the cidram updates page, will it install all the block list contains on your repo? or It will only install the bypass portion?