After upgrading from 1.9.5 to 1.12.12 triggerViaEcho is enabled by default in deck, this results in echo producing NPEs on manual pipeline executions
Kubernetes
1.12.12
@spinnaker/ui-ux-team):The issue I鈥檓 having seems to be with invokePipelineConfigViaEcho https://github.com/spinnaker/gate/blob/release-1.12.x/gate-web/src/main/groovy/com/netflix/spinnaker/gate/controllers/PipelineController.groovy#L227
if I manually curl the old endpoint invokePipelineConfig https://github.com/spinnaker/gate/blob/release-1.12.x/gate-web/src/main/groovy/com/netflix/spinnaker/gate/controllers/PipelineController.groovy#L204 the pipeline will execute as expected
The request from invokePipelineConfigViaEcho is making it to echo, but I see it throwing a NullPointerException here: https://github.com/spinnaker/echo/blob/release-1.12.x/echo-pipelinetriggers/src/main/java/com/netflix/spinnaker/echo/pipelinetriggers/eventhandlers/ManualEventHandler.java#L75
2019-06-19 15:06:09.011 ERROR 1 --- [Scheduler-22366] c.n.s.echo.events.EventPropagator : failed processing event: {application=example, pipelineNameOrId=example: Redeploy-restart, trigger={type=manual, dryRun=false, [email protected], eventId=5b669905-fda1-4132-9c80-5d7cb810657e}, [email protected]}
java.lang.NullPointerException: null
at com.netflix.spinnaker.echo.pipelinetriggers.eventhandlers.ManualEventHandler.pipelineMatches(ManualEventHandler.java:75) ~[echo-pipelinetriggers-1.585.0-SNAPSHOT.jar:1.585.0-SNAPSHOT]
I located the feature flag for deck in 1.12 allowing me to set triggerViaEcho: false https://github.com/spinnaker/deck/blob/release-1.12.x/halconfig/settings.js#L171 this has resolved my issue for now, but this feature flag has been removed in later version
Upgrade from 1.9.5 to 1.12.12 and try to manually execute pipelines?
@jaypz : What's likely happening is that you have a pipeline with either a null name or application which should generally not occur, but which I have heard of in at least one other case. spinnaker/echo#510 should fix that issue as it will remove those invalid pipelines from consideration (and prevent them from breaking triggering of valid pipelines). That fix is in 1.14, but I'll cherry-pick it into 1.13 so that this works on the 1.13 branch. That should provide a reasonable upgrade path where you can keep the flag off in 1.12 then this will no longer occur in 1.13 when the flag is on.
I've also made a PR linked above to just do the null check in the 1.12 branch so this should work even with the flag on in 1.12. I didn't add this check to 1.13 or later as the @NonNull annotations on the Pipeline model are properly blocking deserialization of invalid pipelines in those branches (and logging warnings about what pipeline failed) so we never get to this NPE.
@ezimanyi I can't thank you enough here; this makes total sense to me why this was happening now, really appreciate the effort getting these PRs in!
Most helpful comment
I've also made a PR linked above to just do the null check in the 1.12 branch so this should work even with the flag on in 1.12. I didn't add this check to 1.13 or later as the
@NonNullannotations on thePipelinemodel are properly blocking deserialization of invalid pipelines in those branches (and logging warnings about what pipeline failed) so we never get to this NPE.