Configuration-as-code-plugin: Minimal JCasC yaml config ineffectual wrt. matrix-auth plugin

[x10an14]

Hi! Ref. conversation with @casz in gitter, here's my minimal setup:

jenkins:
  authorizationStrategy:
    globalMatrix:
      grantedPermissions:
        - name: 'anonymous'
          permissions: []
        - name: 'authenticated'
          permissions:
            - 'Overall/Read'
        - name: 'admin'
          permissions:
            - 'Overall/Administer'
  # http://yaml-multiline.info/
  systemMessage: |-
    Testing Matrix-Auth plugin

When I log into the newly created Jenkins instance (which I create from scratch each time, deleting everything in docker-compose and the docker volume), I see the following.

plugins.txt file fed into Docker Image: (see Dockerfile and docker-compose.yaml files)

saml:1.0.7
matrix-auth:2.3
pipeline-model-definition:1.3.1
docker-slaves:1.0.7
pipeline-utility-steps:2.1.0
configuration-as-code:0.10-alpha

docker-compose.yaml:

version: '3'

services:
  jenkins:
    restart: on-failure
    build:
      context: ./jenkins-master
    ports:
      - 8080:8080
    volumes:
      - jenkins_home:/var/jenkins_home
      - /etc/localtime:/etc/localtime:ro  # To ensure timezone matches host
    environment:
      - CASC_JENKINS_CONFIG=/usr/share/jenkins/ref/jenkins.yaml

volumes:
  jenkins_home:

jenkins-master/Dockerfile:

FROM jenkins/jenkins:lts
# LTS version!
# Requires something to execute `docker pull jenkins/jenkins:lts` every now and then, to stay up-to-date.
# Ref: https://hub.docker.com/r/jenkins/jenkins/tags/

# LABEL maintaner=""

COPY stb_plugins.txt /usr/share/jenkins/ref/stb_plugins.txt
RUN xargs /usr/local/bin/install-plugins.sh < /usr/share/jenkins/ref/stb_plugins.txt

# Jenkins Configuration-as-Code (JCasC) settings file:
COPY jenkins.yaml /usr/share/jenkins/ref/jenkins.yaml

[x10an14]

Seems to maybe be related to what @ndeloof is writing about in #352 and in https://github.com/jenkinsci/matrix-auth-plugin/pull/48.

[x10an14]

Oh, and I use

docker-compose down && docker-compose rm -v && docker volume rm stb-jcasc_jenkins_home
&& docker-compose up --build --detach

between each Jenkins iteration, in case that has any effect on reproducing my issue.

[jetersen]

Do you get an expect during startup?
Think you can just remove anonymous when you don't give it any permissions

[jetersen]

Also simplify your run command 😄

docker-compose down -v && docker-compose up --build -d

[x10an14]

@ndeloof Seems to be ahead of me with a potential solution already: #360.

@casz I get no exceptions during startup, no. Below is an example of startup logs with this setup:

Creating stb-jcasc_jenkins_1 ... done
Attaching to stb-jcasc_jenkins_1
jenkins_1  | Running from: /usr/share/jenkins/jenkins.war
jenkins_1  | webroot: EnvVars.masterEnvVars.get("JENKINS_HOME")
jenkins_1  | Jul 13, 2018 9:54:48 AM org.eclipse.jetty.util.log.Log initialized
jenkins_1  | INFO: Logging initialized @245ms to org.eclipse.jetty.util.log.JavaUtilLog
jenkins_1  | Jul 13, 2018 9:54:48 AM winstone.Logger logInternal
jenkins_1  | INFO: Beginning extraction from war file
jenkins_1  | Jul 13, 2018 9:54:49 AM org.eclipse.jetty.server.handler.ContextHandler setContextPath
jenkins_1  | WARNING: Empty contextPath
jenkins_1  | Jul 13, 2018 9:54:49 AM org.eclipse.jetty.server.Server doStart
jenkins_1  | INFO: jetty-9.4.z-SNAPSHOT, build timestamp: 2017-11-21T21:27:37Z, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
jenkins_1  | Jul 13, 2018 9:54:49 AM org.eclipse.jetty.webapp.StandardDescriptorProcessor visitServlet
jenkins_1  | INFO: NO JSP Support for /, did not find org.eclipse.jetty.jsp.JettyJspServlet
jenkins_1  | Jul 13, 2018 9:54:50 AM org.eclipse.jetty.server.session.DefaultSessionIdManager doStart
jenkins_1  | INFO: DefaultSessionIdManager workerName=node0
jenkins_1  | Jul 13, 2018 9:54:50 AM org.eclipse.jetty.server.session.DefaultSessionIdManager doStart
jenkins_1  | INFO: No SessionScavenger set, using defaults
jenkins_1  | Jul 13, 2018 9:54:50 AM org.eclipse.jetty.server.session.HouseKeeper startScavenging
jenkins_1  | INFO: Scavenging every 600000ms
jenkins_1  | Jenkins home directory: /var/jenkins_home found at: EnvVars.masterEnvVars.get("JENKINS_HOME")
jenkins_1  | Jul 13, 2018 9:54:50 AM org.eclipse.jetty.server.handler.ContextHandler doStart
jenkins_1  | INFO: Started w.@66971f6b{/,file:///var/jenkins_home/war/,AVAILABLE}{/var/jenkins_home/war}
jenkins_1  | Jul 13, 2018 9:54:50 AM org.eclipse.jetty.server.AbstractConnector doStart
jenkins_1  | INFO: Started ServerConnector@60cf80e7{HTTP/1.1,[http/1.1]}{0.0.0.0:8080}
jenkins_1  | Jul 13, 2018 9:54:50 AM org.eclipse.jetty.server.Server doStart
jenkins_1  | INFO: Started @1982ms
jenkins_1  | Jul 13, 2018 9:54:50 AM winstone.Logger logInternal
jenkins_1  | INFO: Winstone Servlet Engine v4.0 running: controlPort=disabled
jenkins_1  | Jul 13, 2018 9:54:51 AM jenkins.InitReactorRunner$1 onAttained
jenkins_1  | INFO: Started initialization
jenkins_1  | Jul 13, 2018 9:54:52 AM jenkins.InitReactorRunner$1 onAttained
jenkins_1  | INFO: Listed all plugins
jenkins_1  | Jul 13, 2018 9:54:55 AM jenkins.InitReactorRunner$1 onAttained
jenkins_1  | INFO: Prepared all plugins
jenkins_1  | Jul 13, 2018 9:54:55 AM jenkins.InitReactorRunner$1 onAttained
jenkins_1  | INFO: Started all plugins
jenkins_1  | Jul 13, 2018 9:54:55 AM jenkins.InitReactorRunner$1 onAttained
jenkins_1  | INFO: Augmented all extensions
jenkins_1  | Jul 13, 2018 9:54:55 AM org.jenkinsci.plugins.casc.impl.configurators.DataBoundConfigurator tryConstructor
jenkins_1  | INFO: Setting class org.jenkinsci.plugins.casc.integrations.globalmatrixauth.GroupPermissionDefinition.name = anonymous
jenkins_1  | Jul 13, 2018 9:54:55 AM org.jenkinsci.plugins.casc.impl.configurators.DataBoundConfigurator tryConstructor
jenkins_1  | INFO: Setting class org.jenkinsci.plugins.casc.integrations.globalmatrixauth.GroupPermissionDefinition.permissions = []
jenkins_1  | Jul 13, 2018 9:54:55 AM org.jenkinsci.plugins.casc.impl.configurators.DataBoundConfigurator tryConstructor
jenkins_1  | INFO: Setting class org.jenkinsci.plugins.casc.integrations.globalmatrixauth.GroupPermissionDefinition.name = authenticated
jenkins_1  | Jul 13, 2018 9:54:55 AM org.jenkinsci.plugins.casc.impl.configurators.DataBoundConfigurator tryConstructor
jenkins_1  | INFO: Setting class org.jenkinsci.plugins.casc.integrations.globalmatrixauth.GroupPermissionDefinition.permissions = [Overall/Read]
jenkins_1  | Jul 13, 2018 9:54:55 AM org.jenkinsci.plugins.casc.impl.configurators.DataBoundConfigurator tryConstructor
jenkins_1  | INFO: Setting class org.jenkinsci.plugins.casc.integrations.globalmatrixauth.GroupPermissionDefinition.name = admin
jenkins_1  | Jul 13, 2018 9:54:55 AM org.jenkinsci.plugins.casc.impl.configurators.DataBoundConfigurator tryConstructor
jenkins_1  | INFO: Setting class org.jenkinsci.plugins.casc.integrations.globalmatrixauth.GroupPermissionDefinition.permissions = [Overall/Administer]
jenkins_1  | Jul 13, 2018 9:54:55 AM org.jenkinsci.plugins.casc.BaseConfigurator configure
jenkins_1  | INFO: Setting [email protected] = hudson.security.GlobalMatrixAuthorizationStrategy@30ad76ba
jenkins_1  | Jul 13, 2018 9:54:55 AM org.jenkinsci.plugins.casc.BaseConfigurator configure
jenkins_1  | INFO: Setting [email protected] = Testing Matrix-Auth plugin
jenkins_1  | Jul 13, 2018 9:54:55 AM jenkins.InitReactorRunner$1 onAttained
jenkins_1  | INFO: Loaded all jobs
jenkins_1  | Jul 13, 2018 9:54:56 AM jenkins.util.groovy.GroovyHookScript execute
jenkins_1  | INFO: Executing /var/jenkins_home/init.groovy.d/tcp-slave-agent-port.groovy
jenkins_1  | Jul 13, 2018 9:54:56 AM hudson.model.AsyncPeriodicWork$1 run
jenkins_1  | INFO: Started Download metadata
jenkins_1  | Jul 13, 2018 9:54:56 AM org.springframework.context.support.AbstractApplicationContext prepareRefresh
jenkins_1  | INFO: Refreshing org.springframework.web.context.support.StaticWebApplicationContext@11ae3427: display name [Root WebApplicationContext]; startup date [Fri Jul 13 09:54:56 UTC 2018]; root of context hierarchy
jenkins_1  | Jul 13, 2018 9:54:56 AM org.springframework.context.support.AbstractApplicationContext obtainFreshBeanFactory
jenkins_1  | INFO: Bean factory for application context [org.springframework.web.context.support.StaticWebApplicationContext@11ae3427]: org.springframework.beans.factory.support.DefaultListableBeanFactory@161d433
jenkins_1  | Jul 13, 2018 9:54:56 AM org.springframework.beans.factory.support.DefaultListableBeanFactory preInstantiateSingletons
jenkins_1  | INFO: Pre-instantiating singletons in org.springframework.beans.factory.support.DefaultListableBeanFactory@161d433: defining beans [authenticationManager]; root of factory hierarchy
jenkins_1  | Jul 13, 2018 9:54:56 AM org.springframework.context.support.AbstractApplicationContext prepareRefresh
jenkins_1  | INFO: Refreshing org.springframework.web.context.support.StaticWebApplicationContext@6c9b1297: display name [Root WebApplicationContext]; startup date [Fri Jul 13 09:54:56 UTC 2018]; root of context hierarchy
jenkins_1  | Jul 13, 2018 9:54:56 AM org.springframework.context.support.AbstractApplicationContext obtainFreshBeanFactory
jenkins_1  | INFO: Bean factory for application context [org.springframework.web.context.support.StaticWebApplicationContext@6c9b1297]: org.springframework.beans.factory.support.DefaultListableBeanFactory@7a4c2088
jenkins_1  | Jul 13, 2018 9:54:56 AM org.springframework.beans.factory.support.DefaultListableBeanFactory preInstantiateSingletons
jenkins_1  | INFO: Pre-instantiating singletons in org.springframework.beans.factory.support.DefaultListableBeanFactory@7a4c2088: defining beans [filter,legacy]; root of factory hierarchy
jenkins_1  | Jul 13, 2018 9:54:56 AM jenkins.install.SetupWizard init
jenkins_1  | INFO: 
jenkins_1  | 
jenkins_1  | *************************************************************
jenkins_1  | *************************************************************
jenkins_1  | *************************************************************
jenkins_1  | 
jenkins_1  | Jenkins initial setup is required. An admin user has been created and a password generated.
jenkins_1  | Please use the following password to proceed to installation:
jenkins_1  | 
jenkins_1  | 711697b3f6164fb4aa07081f2f8744e7
jenkins_1  | 
jenkins_1  | This may also be found at: /var/jenkins_home/secrets/initialAdminPassword
jenkins_1  | 
jenkins_1  | *************************************************************
jenkins_1  | *************************************************************
jenkins_1  | *************************************************************
jenkins_1  | 
jenkins_1  | Jul 13, 2018 9:55:00 AM hudson.model.UpdateSite updateData
jenkins_1  | INFO: Obtained the latest update center data file for UpdateSource default
jenkins_1  | Jul 13, 2018 9:55:01 AM hudson.model.UpdateSite updateData
jenkins_1  | INFO: Obtained the latest update center data file for UpdateSource default
jenkins_1  | Jul 13, 2018 9:55:01 AM hudson.model.DownloadService$Downloadable load
jenkins_1  | INFO: Obtained the updated data file for hudson.tasks.Maven.MavenInstaller
jenkins_1  | Jul 13, 2018 9:55:01 AM hudson.model.AsyncPeriodicWork$1 run
jenkins_1  | INFO: Finished Download metadata. 5,209 ms
jenkins_1  | Jul 13, 2018 9:55:01 AM jenkins.InitReactorRunner$1 onAttained
jenkins_1  | INFO: Completed initialization
jenkins_1  | Jul 13, 2018 9:55:01 AM hudson.WebAppMain$3 run
jenkins_1  | INFO: Jenkins is fully up and running
jenkins_1  | --> setting agent port for jnlp
jenkins_1  | --> setting agent port for jnlp... done
jenkins_1  | Jul 13, 2018 10:04:52 AM hudson.model.AsyncAperiodicWork$1 run
jenkins_1  | INFO: Started Update IdP Metadata from URL PeriodicWork
jenkins_1  | Jul 13, 2018 10:04:52 AM hudson.model.AsyncAperiodicWork$1 run
jenkins_1  | INFO: Finished Update IdP Metadata from URL PeriodicWork. 1 ms
jenkins_1  | Jul 13, 2018 10:14:52 AM hudson.model.AsyncAperiodicWork$1 run
jenkins_1  | INFO: Started Update IdP Metadata from URL PeriodicWork
jenkins_1  | Jul 13, 2018 10:14:52 AM hudson.model.AsyncAperiodicWork$1 run
jenkins_1  | INFO: Finished Update IdP Metadata from URL PeriodicWork. 1 ms

[jetersen]

Oh you definitely want to disable the initial setup wizard...

Add this to your Dockerfile

RUN \
  echo $JENKINS_VERSION > /usr/share/jenkins/ref/jenkins.install.UpgradeWizard.state && \
  echo $JENKINS_VERSION > /usr/share/jenkins/ref/jenkins.install.InstallUtil.lastExecVersion

[x10an14]

@casz docker-compose down -v only deletes _anonymous_ volumes, not named ones =)

As such, my docker-compose rm -v is most likely unecessary, but I've put this all into a Makefile, and thought to just be on the safe side =)
What I'm _actually_ typing into my terminal is just make clean && make jenkins ;)

[jetersen]

From docker-compose down --help

-v, --volumes           Remove named volumes declared in the `volumes`
                            section of the Compose file and anonymous volumes
                            attached to containers.

[x10an14]

@casz Ahh, thanks! I was wondering about that!

I read somewhere that it wouldn't show if I had made some configuration of a user (like admin), so I was waiting on my SAML setup (or Matrix roles) to work before attempting to check that egg.

[x10an14]

Ahh, derp. That's what I get for trusting random stack overflow posts instead of checking the documentation the tool offers.

Thanks for the tip on how to disable the initial setup-wizard! =D

[jetersen]

This is ours and it seems to work 👍

  authorizationStrategy:
    projectMatrix:
      grantedPermissions:
        - name: Read Only
          permissions:
            - Overall/Read
        - name: Create Agents
          permissions:
            - Overall/Administer
        - name: Admins
          permissions:
            - Overall/Administer

[ndeloof]

You indeed should wait for configurator to move to matrix-auth plugin
Please note the configuration format will change, see for sample https://github.com/jenkinsci/matrix-auth-plugin/pull/48/files#diff-45a73ad3af57ec94b0420e8f03d55a46

[x10an14]

@casz Changing my yaml config as described in below git-diff had no effect.
I am greeted with the same settings when I visit security config-page.

diff --git a/jenkins-master/jenkins.yaml b/jenkins-master/jenkins.yaml
index 11063dd..0959278 100644
--- a/jenkins-master/jenkins.yaml
+++ b/jenkins-master/jenkins.yaml
@@ -15,16 +15,16 @@

 jenkins:
   authorizationStrategy:
-    globalMatrix:
+    projectMatrix:
       grantedPermissions:
-        - name: 'anonymous'
+        - name: anonymous
           permissions: []
-        - name: 'authenticated'
+        - name: authenticated
           permissions:
-            - 'Overall/Read'
-        - name: 'admin'
+            - Overall/Read
+        - name: admin
           permissions:
-            - 'Overall/Administer'
+            - Overall/Administer

I think I'll just wait and keep an eye on #360, like @ndeloof says I ought to =)
Thanks for both of your time and help! =D

[jetersen]

Well I did help you with a bunch of other stuff so all good @x10an14 😅

[x10an14]

@casz In case it's of any interest, I (personally/subjectively) think this is a little more readable Dockerfile RUN command for skipping initial setup-wizard:

# Skip first-setup install wizard:
# https://github.com/jenkinsci/configuration-as-code-plugin/issues/361#issuecomment-404793129
RUN echo $JENKINS_VERSION | tee \
    /usr/share/jenkins/ref/jenkins.install.UpgradeWizard.state \
    /usr/share/jenkins/ref/jenkins.install.InstallUtil.lastExecVersion

Thanks again for your assistance =) I've got some other questions I'd like to poke you with, but I'll do it on Gitter instead of polluting this thread ;)

[ndeloof]

when JCasC is enabled, setup wizard is automatically disabled (or should be, maybe we have a bug ?) so you don't have to handle this

[x10an14]

@ndeloof Sorry for taking a week to get back to you.

This (the initial setup wizard) is still happening to me, every time I start a fresh jenkins_volume with my previously given setup.

Would you like me to make an issue of this?

[ndeloof]

Please do. IIRC as we miss a programatic API to disable wizard this had been postponed, but we need to track this and investigate for a fix

[ndeloof]

closing this issue to follow up with initial discussion on https://github.com/jenkinsci/configuration-as-code-plugin/pull/360

[jluk-scms]

Hi, @x10an14 , I'm hitting the same issue with matrix-auth 2.5 and JCasC 1.34. Have you managed to overcome this somehow?

My case details for the record:

I'm configuring the per project matrix authorization with some user permissions.
Jenkins boots up with "Logged in users can do anything" authorization.
When I reload the JCasC config via UI, the auth configuration is fixed (project matrix is set with the appropriate user permissions stored in JCasC config).

Interestingly, if I switch the auth config manually via UI (without reloading JCasC), the user permissions defined in yaml appear properly, so the information is loaded and stored, just the auth setting is wrong.

[x10an14]

I haven't worked on this for quite some time - I no longer work where this was a challenge. I'd recommend you to check out their Gitter channel or make a new issue instead of re-opening a closed one that's a year-and-a-half old =)