Configuration-as-code-plugin: CasC stuck on an old version of a job defined by JobDSL

[timblaktu]

Description

The CasC engine appears to be "stuck" on an old version of a job which contains a syntax error, and I am therefore unable to fix the syntax error and move on.

My jenkins.yaml includes a jobs: list, and at one point in the past, one of the directly-referenced groovy files contained a syntax error which caused the below traceback.

If I comment out the - <my-syntactically-bad-job>.groovy line in my jenkins.yaml` and restart jenkins service, jenkins starts up fine, of course. ATM I have to do this to make the system operational so I can continue hacking on the jobDSL file to try make it right.

But _even after replacing the jobDSL (groovy) file with a version that fixes the syntax error, and reloading CasC (or restarting Jenkins service, or rebooting server), as long as that jobDSL file is in the jobs: list of my jenkins.yaml, I continue to get the same error_.

Maybe relevant: .groovy defines a simple, plain job whose syntax error is inside a shell step (which contains an inline bash script string). This jobDSL worked fine with this workflow up until this syntax error creeped in and now the workflow is ineffectual.

Do I need to try to force CasC to delete the job so it can be re-created properly without err? How do I do this?

Jenkins controller is on Debian 10.5, Jenkins v2.249.2, CasC v1.45, JobDSL v1.77.

org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
script: 101: unexpected char: '\' @ line 101, column 52.
   NCH=$(echo ${BRANCH} | sed 's/\//%2F/g' 
                                 ^

1 error

    at org.codehaus.groovy.control.ErrorCollector.failIfErrors(ErrorCollector.java:310)
    at org.codehaus.groovy.control.ErrorCollector.addFatalError(ErrorCollector.java:150)
    at org.codehaus.groovy.control.ErrorCollector.addError(ErrorCollector.java:120)
    at org.codehaus.groovy.control.ErrorCollector.addError(ErrorCollector.java:132)
    at org.codehaus.groovy.control.SourceUnit.addError(SourceUnit.java:350)
    at org.codehaus.groovy.antlr.AntlrParserPlugin.transformCSTIntoAST(AntlrParserPlugin.java:139)
    at org.codehaus.groovy.antlr.AntlrParserPlugin.parseCST(AntlrParserPlugin.java:110)
    at org.codehaus.groovy.control.SourceUnit.parse(SourceUnit.java:234)
    at org.codehaus.groovy.control.CompilationUnit$1.call(CompilationUnit.java:168)
    at org.codehaus.groovy.control.CompilationUnit.applyToSourceUnits(CompilationUnit.java:943)
    at org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:605)
    at org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:581)
    at org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:558)
    at groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:298)
    at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:268)
    at groovy.lang.GroovyShell.parseClass(GroovyShell.java:688)
    at groovy.lang.GroovyShell.parse(GroovyShell.java:700)
    at groovy.lang.GroovyShell$parse.call(Unknown Source)
    at javaposse.jobdsl.dsl.AbstractDslScriptLoader.parseScript(AbstractDslScriptLoader.groovy:134)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
    at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoCachedMethodSiteNoUnwrapNoCoerce.invoke(PogoMetaMethodSite.java:210)
    at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.callCurrent(PogoMetaMethodSite.java:59)
    at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:174)
    at javaposse.jobdsl.dsl.AbstractDslScriptLoader.runScriptEngine(AbstractDslScriptLoader.groovy:101)
Caused: javaposse.jobdsl.dsl.DslException: startup failed:
script: 101: unexpected char: '\' @ line 101, column 52.
   NCH=$(echo ${BRANCH} | sed 's/\//%2F/g' 
                                 ^

• [2.249.2] Jenkins version

• [CasC 1.45, JobDSL 1.77] Plugin version

• [Debian 10.5 ] OS

[jetersen]

Is the groovy script written inside yaml or loaded from a groovy file?

[jetersen]

if written inside groovy, you ought to escape it twice since since you most likely wanted to preserve the backslash for your bash script.

Honestly have nothing todo with JCasC 😅

[jetersen]

Basic groovy syntax: https://groovy-lang.org/syntax.html

[timblaktu]

Astute comments, @jetersen. The JobDSL is defined in a dedicated .groovy file which is directly references in the jobs: list.

In fact, I was just writing that this turned out to be cockpit error. This job is completely defined in this groovy file. As I reported, the job is composed of a shell JobDSL step that is completely defined in a string (which defines a bash script) in the shell step in the groovy file. _This_ is the crux of the problem, since we have so many layers of language and formatting going on here.

The problematic line was the comment line below from the groovy file:

        shell('''\
#!/bin/bash
.
.
        # ESCAPED_BRANCH=$(echo ${BRANCH} | sed 's/\//%2F/g' | sed 's/"//g')
.
.
''')

The problem that was reported by CasC was legitimate. I didn't escape the first backslash in the sed call. I.e. changing the above to this works:

        shell('''\
#!/bin/bash
.
.
        # ESCAPED_BRANCH=$(echo ${BRANCH} | sed 's/\\//%2F/g' | sed 's/"//g')
.
.
''')

Because this line was commented out (at the bash "layer") I fooled myself into thinking I was protecting myself from syntax problems in that line. But I was forgetting here we've got bash inside groovy. (and in my case I've got another outer layer which is the ansible/jinja2 template that's creating the groovy file to be placed on the server. Holy escape characters, Batman!)

Of course now I see this was not the case. CasC (or probably JobDSL plugin) was parsing this groovy file and failed the groovy syntax check.