Adguardhome: "Auto fix" binding on port 53, on Ubuntu, screws up resolv.conf

[fonix232]

Issue Details

• Version of AdGuard Home server:
  
  
  
  • 0.98.1
  
  
• How did you setup DNS configuration:
  
  
  
  • Does not matter
  
  
• Operating system and version:
  
  
  
  • Ubuntu 19.04
  
  

Expected Behavior

After setup, AdGuard Home should properly configure what gets added to /etc/resolv.conf to ensure the host system works

Actual Behavior

AdGuard Home screws up the /etc/resolv.conf entry, which becomes 127.0.0.53 instead of 127.0.0.1 or 127.0.0.1:53, thereby killing any and all DNS resolution on the host machine.

[szolin]

screws up the /etc/resolv.conf

I don't see how we could do it. We could only modify /etc/systemd/resolved.conf file by applying this command:
sed -r -i.orig 's/#?DNSStubListener=yes/DNSStubListener=no/g' /etc/systemd/resolved.conf
But these files are not the same.
We also save a backup file /etc/systemd/resolved.conf.orig - does it exist on your system?

What is the full contents of your resolv.conf ?

[fonix232]

It's quite easy to reproduce:

1. Create DigitalOcean Ubuntu 19.04 droplet
2. Follow official install instructions and install AGH as system service
3. Set up AGH to listen on all interfaces and bind to :53 on all interfaces as well
4. Upon launch, be greeted with the "couldn't bind to port 53, auto fix?" message in settings
5. Click "Auto fix"
6. Observe the change in /etc/resolv.conf
7. Modify /etc/resolv.conf and replace erroneous 127.0.0.53 with working 127.0.0.1
8. Upon restart, or system idling for some time, the corrected value gets replaced with 127.0.0.53

The file /etc/systemd/resolved.conf indeed contains said changes to DNSStubListener. However I'm talking about /etc/resolv.conf which is symlinked from /run/systemd/resolve/stub-resolv.conf

Without installing AGH, the default resolv.conf in my case contained 127.0.0.1, and after installation it got set to 127.0.0.53, and reset to it every time I restart or let the server idle.

[szolin]

AGH doesn't modify /etc/resolv.conf, we don't have the code that does it.
Try to run lsof on AGH, or run AGH under strace - you won't see that it opens /etc/resolv.conf at any time.

It seems like there is a service in Ubuntu that keeps modifying your file.

[fonix232]

@szolin that might be, but does not explain why it only happens after AGH is installed and started. In which case the fault still lies within AGH, as this is not an expected behaviour.

[cubakcabak]

chiming in, i can reproduce it

digging out the systemd-resolved documentation, 127.0.0.53 is their default resolver ip which their used as proxy dns for upstream whatever ip is set in /etc/resolve.conf, whether dynamically by dhcp, or manually.

setting DNSStubListener to "no" will disabling the dns proxying which in turn systemd-resolved daemon cant resolve any dns query, even if resolve.conf file consist of whatever ip dns server is.

my suggestions is dont touch the systemd-resolved configurations, but instead show the user which local (dns server) process using 127.0.0.1:53, and suggest them to stop/remove it.

[ameshkov]

@cubakcabak thank you, it's now clear what's going on!

Regarding your suggestion, dealing with DNSStubListener is a common issue, and we'd like to provide some default automatic solution for it.

I think we can simply replace 127.0.0.53 with 127.0.0.1 in /etc/resolv.conf, but we should also warn the user that this is what needs to be done to deal with systemd-resolved.

[szolin]

I think we can simply replace 127.0.0.53 with 127.0.0.1 in /etc/resolv.conf

I doubt it will be enough, because:

it got set to 127.0.0.53, and reset to it every time I restart or let the server idle.

as I understand, the service periodically resets it to .53

[Omoeba]

just disable systemd-resolved, install resolvconf and edit /etc/resolvconf/resolv.conf.d/head to include the nameservers you want

[cubakcabak]

@ameshkov @szolin
instead disabling the DNSStubListener in /etc/systemd/resolved.conf, put this in instead:

[Resolve]
DNS=127.0.0.1
Domains=~.

systemd-resolved will use it as global dns, overriding/ignoring /etc/resolv.conf nameserver without touching the file

per documentation in https://www.freedesktop.org/software/systemd/man/resolved.conf.html

Use the construct "\~." (which is composed of "\~" to indicate a routing domain and "." to indicate the DNS root domain that is the implied suffix of all DNS domains) to use the system DNS server defined with DNS= preferably for all domains.

it's much easier for automation to include it in AdGuardHome (un)installation routine, i guess

[ameshkov]

@cubakcabak the problem is that in this case, AGH won't be able to bind to 0.0.0.0:53, and this is what's causing troubles.

[fonix232]

Would enforcing the removal of systemd-resolved and replacement with resolvconf be an ideal approach? Maybe as a separate option after the "fix" to be able to bind on port 53?

[Omoeba]

@fonix232 without replacing systemd-resolved with resolvconf, things like package updates will be impossible because all dns queries will fail.

[zombu2]

so how is this gonna be fixed we can talk all day about it but in the end it's still breaking the system

[ameshkov]

Not sure yet what will be the final solution. I've assigned it to v0.101 so that we don't forget about this issue.

[Omoeba]

the resolvconf workaround is pretty useful for now

[zombu2]

it is but it should not become a permanent solution in the end

[Omoeba]

I like how resolvconf works so I use it permanently on any server, even if it is not strictly necessary.

[n1k0]

Just chiming in to confirm this is a major issue for users with no advanced system knowledge. On my end I wanted to toy around with an AdGuard Home local setup on my laptop which is running latest Ubuntu LTS. I just launched the executable and browsed the few configuration screen on the local webserver.

At some point, there was a warning about reachability of the DNS server ip, and I dumbly clicked "Fix". Since then, I'm experiencing the automatic updates to my /etc/resolv.conf as described in the bug description.

It is to be noted that I never ran ./AdGuard -s install on this machine, still I have my networking configuration messed up. I tried ./AdGuard -s uninstall which has no effect on this problem.

Right now I'm still perdiocally updating my resolv.conf file, eg. after each restart or hibernation. You can understand that it's not a sustainable solution.

You guys have to work out this in a way or another, and at the very least provide clear documentation on what to do when such things happen.

I suggest you include this documentation in the downloaded archive so it can be read offline, as the effect of the bug is to prevent any access to the Internet from the affected device.

Edit: Fixed my own situation using this procedure. Handle with care.

[vhfmag]

Any news on that? I know ping comments in issues are annoying, but this is quite a serious bug and it just happened with me, in a Ubuntu 18.04, 3 months after being first filled.

Maybe dropping the autofix altogether in affected OSes while waiting for a definitive fix would be a good enough workaround for the time being?

[zombu2]

yeah this needs to be fixed quickly lots of ppl install it then wonder what is happening and it throws a bad light on adguard

[Omoeba]

@n1k0 An actual undo of Adguard's "fix" is to edit /etc/systemd/resolved.conf and uncomment (remove hashtag) #DNSStubListener=yes or change DNSStubListener=no to DNSStubListener=yes

[ameshkov]

Any news on that?

Planned on v0.101 so it will take some more time.

[DrrAld]

So what’s the current, dev recommended, workaround until this is fixed? I’m asking because I’ve been recommending a few platforms such as this to friends who aren’t knowledgeable about code, but care about privacy. This is now the fifth code base that without knowledge of how systems work is broken on Ubuntu 18 LTS; one of the top five OS in the world. If we ever want to properly scale open-source, we need to be user friendly and though easier than PiHole to implement, this isn’t cutting it.

[ameshkov]

It depends on how exactly you use AG Home. The easiest way would be simply to choose a certain network interface while settings AdGuard Home up (instead of using 0.0.0.0 which is suggested by default).

[zombu2]

that will not change adguard setting it's own dns which will screw up every other interface

NEXT!!!!

[ameshkov]

that will not change adguard setting it's own dns which will screw up every other interface

Mm, why? How will it screw up anything if it simply listens to a specific address?

[szolin]

Ubuntu has this bug since 2016-09-16:
https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1624320

I propose we add some more things for the automatic fix to work:

in /etc/systemd/resolved.conf modify:

[Resolve]
#DNS=

->

[Resolve]
DNS=127.0.0.1

Then activate the correct resolv.conf file:

mv /etc/resolv.conf /etc/resolv.conf.backup
ln -s /run/systemd/resolve/resolv.conf /etc/resolv.conf

This method doesn't require stopping systemd-resolved or installing any additional packages.

[ameshkov]

@szolin I am okay with the proposed solution save for a few things:

1. Add an override instead of editing the system file: /etc/systemd/resolved.conf.d/adguard.conf
2. We should add a detailed explanation in the UI. Users should understand what exactly are the system changes AdGuard Home is going to make.

[szolin]

0ffe53405d8f167f5ccf3cfb044a4b4c1598e511