Dnscrypt-proxy: Avoid using outdated cache files

[mibere]

Is your feature request related to a problem?

As we could read at Reddit there is the problem that there are users with very outdated cache files.

My server has a new IP for 27 days now, but there are still several thousand queries on the old IP per day (the old server is still running).

As server operator I want that the client (dnscrypt-proxy) respects my changes at some point.

Describe the solution you'd like

The client (dnscrypt-proxy) should in some way force an update of the files (or in other words: the client should force the user), or completely refrain from answering requests if the files are too old, e.g. older than 20 days.

Who will that feature be useful to?

It is beneficial for users and server operators.

/cc @ianbashford

[dodmi]

dnscrypt-proxy should never stop resolving, if it's able to.
Without being able to resolve names, also updating the config files wouldn't be possible anymore!
A configuration even with only static resolvers should be possible in future!
So think of a user (re)installing his machine with an installation image or restoring an older backup.
He'd end up with a "broken" internet connection, when the data files are older than x days.

[ianbashford]

Yeah I'd be trying to think of a solution too.
Outputting warnings for old cache files should be doable. We could create a wiki page to reference in the log messages too... I'm sure there are plenty of inexpert users we should be trying to support.

@dodmi I think you're right -- we shouldn't leave people stranded. Although if in your scenario the server was properly configured to start with then it should start by downloading the new cache files anyway. But badly configured servers are the real target here...
Imagine if someone had put this on their router - I've seen tutorials for ubiquity edge routers that do that...

I tried looking around for tutorials that seemed to set things up with possible errors. I need to follow the pi-hole tutorial through in more detail, as they suggest the first run as root before then running as a non-root user later. I thought that might be a possible cause... my attempt to put a post on the pihole reddit was thwarted by the admins though, even though I pointed out to them that their users might be using bad configurations...

[mibere]

Of course there should be entries in the log. But this changes and helps almost nothing. I guess nobody will look into the log, unless there are difficulties. Therefore I have no problems doing it the hard way.

[jedisct1]

How do people end up with outdated cache files? These are automatically updated while the proxy is running.

People may change the refresh delay to something ridiculously high, or copy everything into the [static] section, both being is equivalent to explicitly saying "I don't want updates".

We can still log warning messages (people may indeed not pay attention to them, though). And/or do sneaky things such as making the startup time slower and slower and files become older.

[mibere]

People may change the refresh delay to something ridiculously high

Let's assume

[sources.'public-resolvers']
...
cache_file = 'public-resolvers.md'
refresh_delay = 2160

Could we begin with a modification in the code?

if refresh_delay > 240 then refresh_delay = 240

[welwood08]

This symptom might not be caused by outdated cache files. Even with updated cache files, dnscrypt-proxy doesn't use the new files until it's restarted. I don't know about anyone else, but it can easily be months before that happens.

[mibere]

Even with updated cache files, dnscrypt-proxy doesn't use the new files until it's restarted.

Really? That would be a bug in my opinion.

[ianbashford]

@welwood08 I keep my dnscrypt-proxies running for months - don't they migrate to the new files on a daily schedule? I don't think a restart is required...

How do people end up with outdated cache files?

@jedisct1 One of my theories was bad instructions.

Suppose someone does a test run as root (so cache files are owned as root). After that's successful, they run as a non-root user. Its _still_ successful b/c it can read the files, but will never be able to update them.

e.g. if you look at the instructions here: https://github.com/pi-hole/pi-hole/wiki/DNSCrypt-2.0
Half way down, after testing : Warning I did not set dnscrypt-proxy to run as non-root user yet.

I've not walked through these yet, but it set some alarms off... I need to setup a pihole instance to confirm. The 'setcap' on a binary only allows it to bind to port 53, not update root owned files ( I think... ).
Of course, I might be way off the mark here - but the amount of traffic I'm getting to my old site made me think it's not an accidental config setting, but an installation issue.

[scubasunny]

That's true when i initially installed the synology native packages on my syno.
Process is started as root to open port 53 before switching to the user configured in toml file.
I only had to change user and group ownership of cache files ones!!! to grant update access to process them without restart of the dnscrypt proxy.
But still i have to perform a scheduled restart after updating the blacklist files ;-) as these are not read newly be a running process.

[jedisct1]

@vaporwave9 what is being discussed here is the cached resolvers lists. As for caching DNS responses, Unbound is not necessary, dnscrypt-proxy already does DNS caching.

So, back to the actual topic.

Warnings are logged when cache files cannot be written to. But people may indeed not pay attention to thee. Shall we make these fatal, or maybe only fatal when they are really old?

Doing so would cause the proxy to stop if the clock is not properly set. Right now, if the clock is not properly set, DNSCrypt&TLS negotiation will fail, but the proxy will keep retrying.

Maybe we should at least raise the warning level into an error?

These pi-hole instructions could really be updated. Unfortunately, this is not an editable Wiki any more.

[mibere]

The files can't be updated, therefore they will be old soon. So the software should immediately log it, from my point of view as fatal.
Also _refresh_delay_ should be limited to a low value.

[dodmi]

I'm definitely against handling this issue by stopping DNSCryptProxy from working.
You're talking about cutting people from the internet, because that's the result of not having name resolution.

[jedisct1]

If the resolver information is out of date, resolution may stop working anyway.

Here's a possible plan of action:

• Raise the level of the message on files that cannot be written to from warning to error (not fatal)
• Improve the documentation (in the toml file as most people don't bother reading the actual documentation) to mention that cache files have to be writable by the user the proxy is running as, and why it is important to keep the cache up to date
• When the server starts, if the cache files are very old, don't stop the proxy, but print an error. People are more likely to look at errors when the application starts, not while it has been perfectly running for a long time.

[scubasunny]

That sounds as a good plan in fact 👍

[ianbashford]

I like the above.

We _could_ retain them in a struct; for any long running process they'd still only re-download every _refresh_delay_ but be guaranteed not to go stale.
Is boot strapping ever such a regular problem that this could be an issue?

Perhaps the struct could only be used if the files are not writable...

[ianbashford]

I've had a go in this PR https://github.com/DNSCrypt/dnscrypt-proxy/pull/1564
It limits the max setting for caching as 1 week.
As fresh downloads are already in memory, the PR just registers them, even if the cache can't be written.
There's a few tidy up pieces so I've marked in WIP for now...

[ianbashford]

Should be tidied up and ready for comments.
Effectively it runs from in memory representations of the most recently downloaded server stamps. It will startup from the cached files, and it'll update those cache files if permissioned to do so - but otherwise it'll just run from memory.
The binary representations of the downloads were in memory anyway, so there shouldn't be an increase in memory use - it's just got a timer loop to reprovision them...
Note - it doesn't remove servers - but it will add new servers, or update stamps if they change.