Dnscrypt-proxy: (not an issue) QUESTION: Blocking-list on RAM?

[EstherMoellman]

Hi, please I have two simple questions:

1) I use DNSCrypt with a 4.5MB blocking-list (for ads and trackers). I wonder if it is possible to tweak DNSCrypt in order to load into RAM my blocking-list.
I already tweaked DNSCrypt' cache by increasing its size etc. But of course, to be benefit from cache, webpages need to be loaded first. And this is great, but this is not what I am asking.
I have plenty of RAM, so in order to gain browser performance (browsing speed), I would like to upload into RAM all my 4.5MB blocking-list. Is it possible to do that with DNSCrypt? Looking at DSNCrypt RAM' consume, independently on the size of the blocking-list, the consumed RAM is always the same (around 40MB). So, my wild guess is that DNSCrypt doesn't upload my blocking-list into RAM. And if by chance my ignorant guess is right, then DNSCrypt is using CPU instead of RAM for my blocking-list.
I tested DnsSever (https://github.com/TechnitiumSoftware/DnsServer/), which uploads blocking-lists into RAM (not just cache). And the browsing speed is amazing, webpages load almost instantaneously. Of course, for my 4.5MB blocking-list, DnsServer consumes around 800MB RAM the first 24 hours. After that, my 4.5MB blocking-list remains in RAM, but the RAM' consume stabilizes in 450MB.... which is perfect for me, and I don't mind at all.
I would like to do the same with DNSCrypt. Is it possible? Again, I don't care about the amount of RAM. I prefer to use RAM than CPU.

2) In my DNSCrypt, I already use wildcards at my blocking-list, and it woks like at charm. But I would like to ask if it will be possible to use REGEXPs? They are not good for trackers, malwares, mining, social media etc, but REGEXPs are great against ads. With few REGEXPs rules is possible to block 80% of the ads in a microscopic blocking-list file. REGEXPs for ads can tremendously reduce the size of the blocking-list file. It will be possible to use REGEXPs?

Thank you in advance

[jedisct1]

All the lists (not just the domain blacklist) are stored in RAM, and indexed using a suffix tree.
And memory is allocated in blocks, so it is expected that the consumption is constant unless a significant amount of rules is added.

450MB to store a 4.5MB list is completely nuts. they must be using very inefficient data structures.

Regexps are not supported, but you can block IP addresses and subnets. Ad servers are all on the same networks, so blocking these is way more efficient than blocking individual domains.

[EstherMoellman]

@jedisct1 , thanks a lot for your answer.

So, my "wild guess" was wrong. I wrongly thought DnsServer was faster than DNSCrypt, because DnsServer was using RAM for my blocking-list. I still need to understand why/how DnsServer is faster.

But I will keep using DNSCrypt. I'm not an expert, but according to my reading, its encryption seems to me better. And DNSCrypt plays nicer with wildcards. I just need to discover how to improve performance (browsing speed). I tried tweaking DNSCrypt' cache, but no improvement gains there. I already use the best resolver (Cloudflare) for me, with the lowest latency, so also nothing to change there. Is something else I can tweak in order to gain more performance?

DNSCrypt' Wiki says to be careful with wildcards, because it can hurt performance. I will take a new look on my wildcards, but don't believe I can gain performance there.
You mentioned IPs. But if it is such an efficient way to block ads or trackers, why blocking-lists are not made with IPs? Please, do you know IP or subnets lists that I can use?

EDIT, one more question: Does DNSCrypt works better with UTF-8-BOM? Or UTF-8?

This is my second year using DNSCrypt... perfect, never had a single problem... I love it!
Once again thank you, you always very responsive and so helpful... thanks a lot! :)

PS: Feel free to close the issue if you want.

[ibksturm]

hi @EstherMoellman

i think by own experiences that you couldn speed up dnscrypt (and also doh and dot) as fast as a plain :53 dns server is. that belongs to the nature of it. do encryption & de-encryption will allways need time... sending a postcard its faster ghan a letter (you don‘t have to put it in a papercase). so postcard could everybody read, same thing

hope i could explain you

greets

[jedisct1]

It's your resolver is Cloudflare, you're not using DNSCrypt, but DoH :)

Host names are all ASCII characters, so UTF-8 encoding is not relevant.

[EstherMoellman]

@jedisct1 , I can't stop thanking you for explaining me such interesting stuff...

I apologize to you, I expressed myself in the wrong way, I meant to say that I tested "cloudflare" as one of the ways trying to gain performance, and nothing. So, on my day-to-day, I use a resolver supporting DNSCrypt, always with the lowest latency possible (resolvers supporting DNSCrypt with ad-blocking, are too slow for me).

However and just for your information, in the example I mentioned comparing browsing speed with DnsServer VS DNSCrypt, the "encryption & de-encryption" should not be the cause of performance differences, because both are using "encryption & de-encryption".
Unfortunately I am an average-user, I don't have knowledge in this field, and I can't explain why DnsServer is faster (I thought it was due to RAM), or how I can improve DNSCrypt' performance. I am so ignorant here in this field, that by trying to improve DNSCrypt' performance, I asked you about UTF-8 (because I learnt that many software don't like UTF-8-BOM). This shows you I am looking for DNSCrypt' performance improvements, but without understanding what I'm doing... not good practice from my side (LOL).

By the way, any chance you recommend me blocking-lists using IPs or subnets?
Why your model-list (https://download.dnscrypt.info/blacklists/domains/mybase.txt) is not using IPs, subnets or wildcards?
I also saw at DNSCrypt' Wiki a guy offering blocking-lists for DNSCrypt, but he is not using IPs, nor subnets, nor wildcards (despite his GitHub' title repo announces "DNSCrypt blocking-list with wildcards and REGEXPs)... why nobody is using IPs, subnets, wildcards etc... I just ask you because I feel alone going in the wrong way.
Yeah, I am aware this has nothing to do with you/DNSCrypt. So, if you don't want to answer me, is OK, I understand. But if I am lucky and you answer me with the IP/subnet-lists... it will be great!... you will make my weekend even happier (LOL)

: )