Spark-on-k8s-operator: Error starting spark operator Pod in Quick Start Guide (flag provided but not defined: -enable-batch-scheduler)

[ekraffmiller]

Hi,
I followed the directions in the Quick Start Guide, installing the operator with Helm Chart:

$ helm repo add incubator http://storage.googleapis.com/kubernetes-charts-incubator
$ helm install incubator/sparkoperator --namespace spark-operator

When I check the status of the operator, it is not available:

helm status honest-shark
LAST DEPLOYED: Sun Sep  8 15:01:24 2019
NAMESPACE: spark-operator
STATUS: DEPLOYED

RESOURCES:
==> v1/ClusterRole
NAME                           AGE
honest-shark-sparkoperator-cr  155m

==> v1/ClusterRoleBinding
NAME                            AGE
honest-shark-sparkoperator-crb  155m

==> v1/Deployment
NAME                        READY  UP-TO-DATE  AVAILABLE  AGE
honest-shark-sparkoperator  0/1    1           0          155m

==> v1/Pod(related)
NAME                                         READY  STATUS            RESTARTS  AGE
honest-shark-sparkoperator-7cdb7449f9-6fpmh  0/1    CrashLoopBackOff  34        155m

==> v1/ServiceAccount
NAME                        SECRETS  AGE
honest-shark-spark          1        155m
honest-shark-sparkoperator  1        155m

The log for the sparkoperator Pod shows this error:

 ++ id -u
+ myuid=0
++ id -g
+ mygid=0
+ set +e
++ getent passwd 0
0
0
root:x:0:0:root:/root:/bin/ash
+ uidentry=root:x:0:0:root:/root:/bin/ash
+ set -e
+ echo 0
+ echo 0
+ echo root:x:0:0:root:/root:/bin/ash
+ [[ -z root:x:0:0:root:/root:/bin/ash ]]
+ exec /sbin/tini -s -- /usr/bin/spark-operator -v=2 -namespace= -ingress-url-format= -install-crds=true -controller-threads=10 -resync-interval=30 -logtostderr -enable-batch-scheduler=false -enable-metrics=true -metrics-labels=app_type -metrics-port=10254 -metrics-endpoint=/metrics -metrics-prefix=
flag provided but not defined: -enable-batch-scheduler

[liyinan926]

The version of the operator used in the Helm chart has not been updated yet. The image version v2.4.0-v1beta1-latest has the changes for the flag. Please add the following to your helm installation cmd:

--set operatorVersion=v2.4.0-v1beta1-latest

[ekraffmiller]

That worked for me, thank you.