Syndesis: The syndesis-server pod fails to start on OCP4

[anik120]

This is a...

[ ] Feature request
[ ] Regression (a behavior that used to work and stopped working in a new release)
[*] Bug report  
[ ] Documentation issue or request

Description

I deployed the Syndesis Operator on an OpenShift cluster created by the OpenShift installer. The operator was deployed successfully after the following commands:

$ oc new-project syndesis
$ oc apply -f deploy/syndesis-crd.yml
$ oc apply -f deploy/syndesis-operator.yml
$ oc get templates -n syndesis

$ oc process syndesis-operator -p NAMESPACE=syndesis | oc create -f -

I then applied the Syndesis CR:

$ oc apply -f deploy/syndesis.yml

I then performed a search for available routes and found that a syndesis and todo route existed:

$ oc get routes
NAME HOST/PORT PATH SERVICES PORT TERMINATION WILDCARD
syndesis syndesis-syndesis2.apps.agreene2.devcluster.openshift.com syndesis-oauthproxy 8443 reencrypt/Redirect None
todo todo-syndesis-syndesis2.apps.agreene2.devcluster.openshift.com / todo 8080 edge/Allow None

Visiting the syndesis route showed that requests for integrations and reservations were failing:

    Failing Integration Requests: syndesis-syndesis.apps.clusteranik120.devcluster.openshift.com/api/v1/metrics/integrations
    Failing Reservation Requests: syndesis-syndesis.apps.clusteranik120.devcluster.openshift.com/api/v1/event/reservations

A quick review of the running pods showed that the syndesis-server pod kept restarting and could not reach a running state, the logs from a short lived pod are attached as syndesis-server.txt

[heiko-braun]

Looks like it boils down to

Caused by: java.io.FileNotFoundException: /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt (No such file or directory)

You said the cluster was created "created by the OpenShift installer.". What installer and OCP version?

[anik120]

@heiko-braun

What installer and OCP version?

I'm using the Openshift installer to install a OCP 4.0 cluster

[anik120]

@heiko-braun I doubled checked and it turns out I was running it on a OKD 4.0 cluster

[heiko-braun]

Thanks @anik120, your feedback is very useful. We didn't yet do any verification on OKD 4, but it looks like we better do :)

[heiko-braun]

@anik120 I am not sure the installation steps covered everything. Can you try it with install_ocp.sh taken from here: https://github.com/syndesisio/fuse-online-install/tree/1.5.x ?

[rhuss]

For me, it looks like that's its not an installation issue but an issue how syndesis-server handles the CAs mapped into the Pod. Have there been some changes between OCP 3 and 4 how the platform maps certificates into a Pod's filesystem?

@heiko-braun I suspect that there is more work with being OCP 4 compliant then just adjusting the installation process.

[anik120]

@heiko-braun @rhuss

Have there been some changes between OCP 3 and 4 how the platform maps certificates into a Pod's filesystem?

https://github.com/openshift/openshift-docs/issues/12487#issuecomment-431101731

Pods that currently consume the service-serving CA bundle from /var/run/secrets/kubernetes.io/serviceaccount/service-ca.crt should migrate to obtaining the CA bundle from a configMap annotated with "service.alpha.openshift.io/inject-cabundle=true"

Looks like some code changes are needed to make the operator OCP 4 compliant.

[heiko-braun]

Good catch @anik120 !

[rhuss]

If we want to stay backwards compatible we have to (a) detect either the OpenShift version dynamically or (b) make it a configuration option. Bot options require code changes and I indeed wonder what would be the target release for such a change. // @heiko-braun

[heiko-braun]

@rhuss @dhirajsb let's make it work on branch first and once we know what (other) steps are needed we can discuss when and how to include it.

[heiko-braun]

@dhirajsb If we can make it, an config switch to enable OCP4 support would be good to have in 7.3

[nicolaferraro]

Folks, I see the service CA is used here: https://github.com/syndesisio/syndesis/blob/62b62ae4c581e8b9dbda1a060d5f440c074e7a1a/app/server/openshift/src/main/java/io/syndesis/server/openshift/OpenShiftConfigurationProperties.java#L36

It's old code. Does anyone have any clue why it's needed? Usually a plain DefaultOpenshiftClient works inside the cluster...

I see no references to that file in the fabric8 client.

[heiko-braun]

why don't we start an "ocp4" branch a collect all fixes to make it work?

Once we an idea what it takes, and if we can make it backards compatible, we can decide how to move on.

A branch also has the benefit that we could ask QE for a smoke test.

[dhirajsb]

I created a branch ocp4 from master. Let's consolidate changes there. I am working on changes for the CA cert issue.

[paoloantinori]

hi @dhirajsb do you have any update on this one?

[dhirajsb]

@paoloantinori I have been busy with 7.3 issues until today. I will be resolving this later today.

[dhirajsb]

@paoloantinori With a little help Hiram was able to get the server pod and oauth-proxy for OCP4 working, so we have a Syndesis console on the Summit demo cluster now.
I will work on the template changes and push them to OCP4 branch tomorrow. There are still some open questions about how the certs are actually being used/ignored in OCP4, but at least it's working.

[paoloantinori]

Ah, great, thank you!

On Wed, 13 Mar 2019 at 07:41, Dhiraj Bokde [e-mail] wrote:

@paoloantinori https://github.com/paoloantinori With a little help
Hiram was able to get the server pod and oauth-proxy for OCP4 working, so
we have a Syndesis console on the Summit demo cluster now.
I will work on the template changes and push them to OCP4 branch tomorrow.
There are still some open questions about how the certs are actually being
used/ignored in OCP4, but at least it's working.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/syndesisio/syndesis/issues/4462#issuecomment-472299545,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABcz2p-m8rMq_7vUW90zhXNVxu6ErQOJks5vWJ2agaJpZM4amGby
.

[avano]

works ok now on openshift 4 preview from rhpds