Crc: [BUG]macOS FATA /etc/hosts is not readable/writable by the current user

[jr00n]

General information

• OS: macOS
• Hypervisor: hyperkit
• Did you run crc setup before starting it: Yes

CRC version

version: 1.0.0-beta.5+f2aa58c

CRC status

CRC VM:          Stopped
OpenShift:       Stopped
Disk Usage:      0B of 0B (Inside the CRC VM)
Cache Usage:     10.48GB
Cache Directory: /Users/jr00n/.crc/cache

CRC config

output is empty

Host Operating System

~ » sw_vers
ProductName:    Mac OS X
ProductVersion: 10.14.6
BuildVersion:   18G95

Steps to reproduce

1. rm -rf ~/.crc
2. crc setup
3. crc start
4.

Expected

starting cluster

Actual

FATA /etc/hosts is not readable/writable by the current user
stops

Logs

https://gist.github.com/jr00n/9c3bc02e1fe9fdbde042dc7d5b317cfa

[praveenkumar]

@jr00n I did try same on my mac machine and didn't able to get the /etc/hosts file issue.

$ rm -fr ~/.crc
$ ./crc setup
INFO Checking if running as non-root              
INFO Caching oc binary                            
INFO Setting up virtualization with HyperKit      
INFO Will use root access: change ownership of /Users/prkumar/.crc/bin/hyperkit 
Password:
INFO Will use root access: set suid for /Users/prkumar/.crc/bin/hyperkit 
INFO Installing crc-machine-hyperkit              
INFO Will use root access: change ownership of /Users/prkumar/.crc/bin/crc-driver-hyperkit 
INFO Will use root access: set suid for /Users/prkumar/.crc/bin/crc-driver-hyperkit 
INFO Setting file permissions for /etc/resolver/testing 
INFO Setting file permissions for /etc/hosts      
INFO Unpacking bundle from the CRC binary         
Setup is complete, you can now run 'crc start' to start a CodeReady Containers instance

$ ./crc start
INFO Checking if running as non-root              
INFO Checking if oc binary is cached              
INFO Checking if HyperKit is installed            
INFO Checking if crc-driver-hyperkit is installed 
INFO Checking file permissions for /etc/resolver/testing 
INFO Checking file permissions for /etc/hosts     
? Image pull secret [? for help] 
....

Can you put the output of your /etc/hosts file permission, for me after the setup it is as following ?

$ ls -l /etc/hosts
-rw-r--r--  1 prkumar  wheel  428 Sep 20 12:18 /etc/hosts

[jsliacan]

@jr00n Same as @praveenkumar , can't reproduce :(

$ ./crc version
version: 1.0.0-beta.5+f2aa58c
OpenShift version: 4.1.14 (embedded in binary)

$ ./crc setup
INFO Checking if running as non-root              
INFO Caching oc binary                            
INFO Setting up virtualization with HyperKit      
INFO Will use root access: change ownership of /Users/redhat/.crc/bin/hyperkit 
Password:
INFO Will use root access: set suid for /Users/redhat/.crc/bin/hyperkit 
INFO Installing crc-machine-hyperkit              
INFO Will use root access: change ownership of /Users/redhat/.crc/bin/crc-driver-hyperkit 
INFO Will use root access: set suid for /Users/redhat/.crc/bin/crc-driver-hyperkit 
INFO Setting file permissions for /etc/resolver/testing 
INFO Setting file permissions for /etc/hosts      
INFO Unpacking bundle from the CRC binary         
Setup is complete, you can now run 'crc start' to start a CodeReady Containers instance

$ ./crc start
INFO Checking if running as non-root              
INFO Checking if oc binary is cached              
INFO Checking if HyperKit is installed            
INFO Checking if crc-driver-hyperkit is installed 
INFO Checking file permissions for /etc/resolver/testing 
INFO Checking file permissions for /etc/hosts     
? Image pull secret [? for help] 

[praveenkumar]

@jr00n looks like we found out the bug so if you are new user of crc you are hitting it because of recent changes in the file permission. As a work around can you do following and then use the crc ?

$ crc setup
$ chmod 0600 /etc/hosts
$ crc start

[jr00n]

@praveenkumar Thanks, that is working for me.

[praveenkumar]

fixed by #618 but it is affecting beta release so pining it.

[lkrzyzanek]

I had exactly same issue and workaround chmod 0600 /etc/hosts helped.

But then I got

ERRO Error occurred: Error running post start: open /etc/resolver/testing: permission denied

When I checked /etc/resolver/testing I got

ll /etc/resolver/testing
-r-x------  1 lkrzyzan  wheel     0B Sep 23 12:10 /etc/resolver/testing*

So I applied chmod 0600 /etc/resolver/testing as well and it works now.

[cfergeau]

Yes, this is the same issue, both will be fixed at the same time.

[boomkap]

I ran into the same permission issues with /etc/hosts and /etc/resolver/testing. After fixing the permissions, when I try to run crc start, I get the following error.

./crc start
INFO Checking if running as non-root
INFO Checking if oc binary is cached
INFO Checking if HyperKit is installed
INFO Checking if crc-driver-hyperkit is installed
INFO Checking file permissions for /etc/resolver/testing
INFO Checking file permissions for /etc/hosts
INFO Starting stopped VM ...
ERRO Error occurred: Failed to connect to the crc VM with SSH

[mcanoy]

@boomkap - I had the same issues and then got your error. I deleted the folder ~/.crc folder and went through setup again and was able to get the cluster to start

[boomkap]

@mcanoy - Thanks for the tip. That did the trick as well for me. I ran into the following error during rebuilding the cluster. Strange that the /etc/resolv.conf file exists and is readable by the user that I used to run crc start. It tried a few more times with the same error but finally completed the install.

ERRO Unable to read host resolv file (Temporary Error: Failed to read resolv.conf: open /etc/resolv.conf: no such file or directory********Temporary Error: Failed to read resolv.conf: open /etc/resolv.conf: no such file or directory**********************

[marchmallow]

Still hitting the same with latest version:

crc-macos-1.5.0-amd64 gallomas$ ./crc start --pull-secret-file ./pull-secret.txt 
INFO Checking if oc binary is cached              
INFO Checking if running as non-root              
INFO Checking if HyperKit is installed            
INFO Checking if crc-driver-hyperkit is installed 
INFO Checking file permissions for /etc/resolver/testing 
INFO Checking file permissions for /etc/hosts     
INFO Extracting bundle: crc_hyperkit_4.2.14.crcbundle ... 
INFO Creating CodeReady Containers VM for OpenShift 4.2.14-x86_64... 
INFO Verifying validity of the cluster certificates ... 
ERRO Error running post start: open /etc/hosts: permission denied

$ ls -l /etc/hosts
-rw-------  1 root  wheel  365  2 Feb 08:41 /etc/hosts

Changing permissions of /etc/hosts can be a workaround but it should be at least documented that one needs to do so.. also it should really fail at INFO Checking file permissions for /etc/hosts - I think this bug should be reopened unless tracked by some other issue?

[marchmallow]

Also.. to workaround this only way is eg the following for me (and then it hits another error)

$ sudo  chmod 666 /etc/hosts
Massimilianos-MacBook-Pro:crc-macos-1.5.0-amd64 gallomas$ ls -l /etc/hosts
-rw-rw-rw-  1 root  wheel  365  2 Feb 08:41 /etc/hosts
Massimilianos-MacBook-Pro:crc-macos-1.5.0-amd64 gallomas$ ./crc start --pull-secret-file ./pull-secret.txt 
INFO Checking if oc binary is cached              
INFO Checking if running as non-root              
INFO Checking if HyperKit is installed            
INFO Checking if crc-driver-hyperkit is installed 
INFO Checking file permissions for /etc/resolver/testing 
INFO Checking file permissions for /etc/hosts     
INFO A CodeReady Containers VM for OpenShift 4.2.14-x86_64 is already running 
Started the OpenShift cluster
WARN The cluster might report a degraded or error state. This is expected since several operators have been disabled to lower the resource usage. For more information, please consult the documentation 

After stop and start again, it hits the error reported by others in previous comments then it hits ERRO Failed to connect to the CRC VM with SSH

Overall it does not make for a smooth experience..

[borazem]

I have similar output when starting crc
WARN A new version (1.6.0) has been published on https://cloud.redhat.com/openshift/install/crc/installer-provisioned
INFO Checking if running as non-root
INFO Checking if oc binary is cached
INFO Checking if HyperKit is installed
INFO Checking if crc-driver-hyperkit is installed
INFO Checking file permissions for /etc/resolver/testing
*FATA /etc/hosts is not readable/writable by the current user *

Although I in my case the permissions for /etc/hosts and /etc/resolver/testing seems ok
-rw------- 1 root wheel 540 Feb 21 09:45 hosts
-rw-------@ 1 borazem wheel 46 Jan 1 13:00 testing

The proboem in my case is that the hosts file is actually ignored.
If I add additional record like: "127.0.0.1 bla"
I get the following response on ping: "ping: cannot resolve bla: Unknown host"
although the dnscacheutil command: "sudo dscacheutil -q host -a name bla" returns the correct result:
name: bla
ip_address: 127.0.0.1

Restart does not help, clearing the cache neither. It happened only once that I was able to do something (I though that it helped when I stopped Docker Desktop) that helped and everything worked for a while so I was able to run Docker Desktop, CRC and Minishift at the same time. Then I shut down CRC and Minishift and after a while I realized the probelm is back.

Tried to restart my MAC, stope Docker Deskotp, but no luck this time :-(

[cfergeau]

-rw------- 1 root wheel 540 Feb 21 09:45 hosts

Permissions are not correct here, this is not writable by a regular user, only by root. Are you running any VPN software?

[borazem]

OMG indeed :-) sudo ping works, so I changed it to 644. No Idea who change it :-)
Thanks @cfergeau

[bgehman]

Hitting the same problem, with the latest crc. That file should be remain readable by everyone, and writable only by root as is standard on all variants of Unix.

ls -altr /etc/hosts
-rw-------  1 root  wheel  1093 May 19 16:22 /etc/hosts

OS: macOS Catalina (10.15.4)

Latest CRC version from: https://mirror.openshift.com/pub/openshift-v4/clients/crc/1.10.0/
Dated: 2020-05-04

crc version
crc version: 1.10.0+9025021
OpenShift version: 4.4.3 (embedded in binary)

Not sure why this was closed 8 months ago, and the problem still exists...

[cfergeau]

Not sure why this was closed 8 months ago, and the problem still exists...

Because 8 months ago this was a different bug which was fixed by https://github.com/code-ready/crc/commit/1a6a88c43603c04dfd7fb0230873a9697932a855

[cfergeau]

Are you running any vpn/antivirus/... software which might change the permissions of that file?

[bgehman]

Hi @cfergeau crc is doing it. Repro steps:

Ownership and permissions of /etc/hosts (this is MacOS default):

$ ls -l /etc/hosts
-rw-r--r--  1 root  wheel  1093 May 27 09:23 /etc/hosts

Run crc setup... Note the: INFO Setting file permissions for /etc/hosts that is logged.

$ crc setup
INFO Checking if oc binary is cached              
INFO Caching oc binary                            
INFO Checking if podman remote binary is cached   
INFO Checking if CRC bundle is cached in '$HOME/.crc' 
INFO Unpacking bundle from the CRC binary         
INFO Checking if running as non-root              
INFO Checking if HyperKit is installed            
INFO Setting up virtualization with HyperKit      
INFO Will use root access: change ownership of /Users/bgehmanus.xxx.com/.crc/bin/hyperkit 
INFO Will use root access: set suid for /Users/bgehmanus.xxx.com/.crc/bin/hyperkit 
INFO Checking if crc-driver-hyperkit is installed 
INFO Installing crc-machine-hyperkit              
INFO Will use root access: change ownership of /Users/bgehmanus.xxx.com/.crc/bin/crc-driver-hyperkit 
INFO Will use root access: set suid for /Users/bgehmanus.xxx.com/.crc/bin/crc-driver-hyperkit 
INFO Checking file permissions for /etc/resolver/testing 
INFO Checking file permissions for /etc/hosts     
INFO Setting file permissions for /etc/hosts      
INFO Will use root access: change ownership of /etc/hosts 
Setup is complete, you can now run 'crc start' to start the OpenShift cluster

Ownership and permissions of /etc/hosts after crc setup:

$ ls -l /etc/hosts
-rw-------  1 bgehmanus.xxx.com  wheel  1093 May 27 09:23 /etc/hosts

The ownership & permissions of system files should not be mucked with (IMHO), and the installer should prompt for creds if/when it needs them to modify system files.

[cfergeau]

Ah yes, of course it's doing this. I misread your initial comment because the bug you answered to was a bug where crc failed to change these permissions. I agree with you that it's suboptimal, but we don't want to ask for sudo permissions at crc start time, we want to limit this at crc setup time. This is being reworked in https://github.com/code-ready/crc/pull/1136 through the use of a suid binary. And changing /etc/hosts would not be needed if oc was built to use native macos DNS resolution APIs :(