Kops: Cluster create fails with kops-version.txt not found

[mikejoh]

1. What kops version are you running? The command kops version, will display
this information.
v1.19.0-alpha.3

2. What Kubernetes version are you running? kubectl version will print the
version if a cluster is running or provide the Kubernetes version specified as
a kops flag.
N/A

3. What cloud provider are you using?
OpenStack

4. What commands did you run? What is the simplest way to reproduce this issue?
I'm running the following:

kops create cluster \
--v 10 \
--cloud openstack \
--name test-cluster.k8s.local \
--state file://states \
--zones nova \
--image "Ubuntu 20.04" \
--master-count 3 \
--node-count 3 \
--os-dns-servers 192.168.0.10 \
--network-cidr 10.233.0.0/16 \
--etcd-storage-type ssd \
--networking flannel-vxlan \
--node-size k8s-node-flavor \
--master-size k8s-master-flavor \
--kubernetes-version 1.18.8

5. What happened after the commands executed?

I0905 23:13:13.253155  332711 channel.go:153] VersionRecommendationSpec does not specify RecommendedVersion
I0905 23:13:13.253189  332711 channel.go:191] VersionRecommendationSpec does not specify RequiredVersion
I0905 23:13:13.253240  332711 channel.go:146] RecommendedVersion="1.18.8", Have="1.18.8".  No upgrade needed.
I0905 23:13:13.253283  332711 channel.go:184] RequiredVersion="1.18.0", Have="1.18.8".  No upgrade needed.

error reading last kops version used to update: open states/test-cluster.k8s.local/kops-version.txt: no such file or directory

6. What did you expect to happen?
Deployment of a cluster.

7. Please provide your cluster manifest. Execute
kops get --name my.example.com -o yaml to display your cluster manifest.
You may want to remove your cluster name and other sensitive information.

8. Please run the commands with most verbose logging by adding the -v 10 flag.
Paste the logs into this report, or in a gist and provide the gist link here.

9. Anything else do we need to know?
Not sure what's going on here and i've tried to investigate a bit in the apply_cluster.go file where this error is originating from. Could it be a regression bug from the following PR https://github.com/kubernetes/kops/pull/9362?

To me it looks like when the AllowKopsDowngrade option is false we try to read the kops-version.txt file but this file is written at a later stage in apply_cluster.go (as long as dry-run is set to false).

Note that i'm a new user of kops and i have limited experience running it.

[johngmyers]

It looks like the file: implementation of vfs.Path.Readfile() is violating the contract by returning an error other than os.ErrNotExist when the path does not exist.

[johngmyers]

By the way, I doubt file: is going to work well as a state store as the instances in the cluster won't be able to read from it.

[mikejoh]

By the way, I doubt file: is going to work well as a state store as the instances in the cluster won't be able to read from it.

Right, thanks for the info! I'll test using Swift (object store) from now on!