Adaway: Allow list ignored in v5.0.3 VPN mode.

[ipdev99]

It appears that the personal lists are handled separately of the hosts sources.

To describe, using example.adaway.org as the URL for the example.

Root mode.
example.adaway.org in Personal Blocking list will block the site regardless of hosts sources used.

example.adaway.org in Personal Allow list, the site will be removed from the generated hosts file as expected.

VPN mode.
example.adaway.org in Personal Blocking list will be blocked as expected.
example.adaway.org in Personal Allow list will be __blocked__ if it's included in the hosts sources. _Unexpected._

example.adaway.org in Personal Blocking list and not in the hosts sources, then adding example.adaway.org to Personal Allow list will work as expected and the URL is accessible as expected.

As an oddity (or feature) when the same URL is added to both Personal Blocking and Allow lists. After a refresh/download sources, the URL entry is removed from the older list.

• Add example.adaway.org to Blocking list-> Then add to Allow list-> Then refresh/download. The example.adaway.org entry in Personal Blocking list is removed.
• Add example.adaway.org to Allow list-> Then add to Personal Blocking list-> Then refresh/download. The example.adaway.org entry in the Allow list is removed.

I am guessing this is still part of the prior issue.
_#1917_
_#1965_
Maybe the fix had unintended results or exposed a different problem(?)

Originally found with the same app I was testing. (CNN Breaking News)

I decided it would be easier if I controlled the variables for testing so, I switched to blocking a website instead.

Quick webSite to be blocked for testing
http://adawaytest.ipfiles.com

A hosts source that blocks adawaytest.ipfiles.com
https://hosts.ipfiles.com/ntBlack.txt

Dropping back to v5.0.2 the Allow list works as expected in VPN mode.
(Save for the removal upon download issue in that build.)

Did not retest Root mode v5.0.2
It was working (again save for the known issues) when I used it and v5.0.3 seems to be working in Root mode. When I get the time I will double check v5.0.2 Root mode.

Tested on:
Pixel 3aXL (VPN mode)
abc ROM (Android 10 May 2020 Security)
Magisk canary (debug)
AdAway 5.0.3 (Updated from 5.0.2 using the built-in updater.)
AdAway 5.0.3 (Uninstalled and reinstalled.)
AdAway 5.0.2 (The test version from xda thread.)

Nexus 7 (2013) (Root mode)
AEX [AospExtended unofficial] (Android 10 APR 2020 Security.)
Magisk canary (debug)
AdAway 5.0.3 (Updated from 5.0.2 using the built-in updater.)

Poco F1 (Root mode)
Lineage 17 nightly
Magisk canary (debug)
AdAway 5.0.3 (Clean install.)

I never rule out user error so, might just be me.

[PerfectSlayer]

Hi @ipdev99

Thanks for your very detailed bug report!

Your observation is right, the VPN mode does not apply allowed or redirected domains, whatever they came from (user list or source list). It's a limitation I though I could fix quickly after the first beta release but it appears my original database solution is not working and sadly can't be fixed.

I was trying to create a database view based on domain table that represents the final hosts file.
But it has too complex rules:

• Get all blocked domains
  
  
  
  • which source is enabled
  
  
  • which host does not match any allowed host pattern which source is enabled (that part is too expensive)
  
  
  • which does not have redirected host which source is enabled
  
  
• And append all redirected domains
  
  
  
  • which source is enabled
  
  
• Ideally sorted and with an index on domain to fasten search queries

It created a such query and works well on a small amount of data. But it doesn't scale at all (hang up forever with 10k rows).
Since then, I'm trying to improve query speed and write a lot of unit tests. But as the original implementation is flawed, I'll keep the tests and try a new one. Once fixed, I could fix this issue and #1997 too.

[PerfectSlayer]

I spent the morning on this rework. It passes all unit tests now so I am a little more confident 😅
But I still not support redirection yet in VPN mode. I need to find how to answer with a fake DNS response packet before.