Docker: jenkins-log.txt: Directory nonexistent

[minotaurrr]

Issues and Contributing

Please note that only issues related to this Docker image will be addressed here.

• If you have Docker related issues, please ask in the Docker user mailing list.
• If you have Jenkins related issues, please ask in the Jenkins mailing lists.
• If you are not sure, then this is probably not the place to create an issue and you should use any of the previously mentioned mailing lists.

If after going through the previous checklist you still think you should create an issue here please provide:

Docker commands that you execute

Actual result

[testing] Running shell script
sh: 1: cannot create /var/jenkins_home/workspace/testing@tmp/durable-518faec8/jenkins-log.txt: Directory nonexistent
sh: 1: cannot create /var/jenkins_home/workspace/testing@tmp/durable-518faec8/jenkins-result.txt: Directory nonexistent

Expected outcome

Have you tried a non-dockerized Jenkins and get the expected outcome?

Output of docker version

Docker version 17.12.0-ce, build c97c6d6

Other relevant information

Hi guys,

I'm trying to run this Jenkinsfile but it keeps giving me the above error in the test stage.

node {
    def app
    stage ('checkout') {
        checkout scm
    }

    stage ('build') {
        app = docker.build('testing')
    }

    stage ('test') {
        app.inside {
            sh 'npm test'
        } 
    }
}

Ran docker container like this

docker run -d -u root -p 8080:8080 -v /usr/local/bin:/user/bin -v /var/run/docker.sock:/var/run/docker.sock jenkins

Couldn't find any answer on Google or here. Can anyone help please?

[carlossg]

you are executing sh in a docker container that is actually in the host, I don't think that will work with docker pipeline from a docker container, only from host (unless you share the /var/jenkins_home/workspace between both containers).
You should ask in jenkins-user mailing list

[midN]

It was working until recent updates either on Docker or Jenkins plugins.

Now it looks like if you run Jenkins as a Container and inside Jenkins you run Docker.inside command from pipeline it will use host filesystem, while previously it used Jenkins container volume.

[minotaurrr]

@midN Same here.

  | Pipeline: DeclarativeAn opinionated, declarative Pipeline. | 1.2.6 |   |  
  | Pipeline: Declarative Extension Points APIAPIs for extension points used in Declarative Pipelines. | 1.2.6 |   |  
  | Pipeline: Model APIModel API for Declarative Pipeline. | 1.2.6 |   |  
  | Pipeline: Stage Tags MetadataLibrary plugin for Pipeline stage tag metadata. | 1.2.6

These were updated from 1.2.5 and it stopped working.

[midN]

My workaround to not do changes in every project:
I've had a separate folder on HOST shared to Jenkins container to store workspaces such as "/imafolder" and on Jenkins container it was booted under "/var/imafolder".

Since the folder is shared, i just changed HOST folder to match exactly same as Jenkins used "/var/imafolder".

With these changes once you run a new container from Jenkins container, it will use given folder on HOST but since it's shared to Jenkins on same location, there will be no problems whatsoever.

[minotaurrr]

@midN do you mind sharing the Docker command you ran?

[midN]

@minotaurrr
Here are steps i use:

1. On Jenkins HOST i created directory to hold workspaces for projects built with Jenkinsfile

# Jenkins workspaces
mkdir -p /var/jenkins_workspaces
chmod -R 777 /var/jenkins_workspaces

1. Mount given dir as volume to Jenkins container ( i run ECS, so just -v pretty much. )

- Name: workspaces_volume
  Host:
    SourcePath: /var/jenkins_workspaces

1. Mount volume to EXACTLY same path on Container

- ContainerPath: /var/jenkins_workspaces
  SourceVolume: workspaces_volume

1. Wrap commands ran in Jenkinsfile with your custom workspace

ws("/var/jenkins_workspaces/helloworld"){
  YOUR STAGES

  docker.image("x").inside("-u 0:0") {
  }
}

So basically you are telling your Jenkins container to use "/var/jenkins_workspaces" dir on Container which is really a volume on Host with same path "/var/jenkins_workspaces".
Now when i run a "docker.image.inside", it will use Hosts "/var/jenkins_workspaces" path, which is mounted to Jenkins.

[minotaurrr]

@midN thanks for sharing! I’ll try it and let you know later :)

[barbarosalp]

Having the same issue here.

[tao-zhang]

Same issue here. And tried @midN workaround, it works great!

[tao-zhang]

As we run Jenkins jnlp docker in ECS, it's really annoying that if we scale cluster instance. You have to create jenkins workspaces manually on each cluster instances.
Does anyone know with which (Docker or jenkins) version it still works with container volume instead of host volume? Maybe I can downgrade it.

[midN]

@tao-zhang I do it in user-data, so you don't need to create it manually everytime :)
Or if you don't use autoscaling, create custom AMI with folder created.

Or, you can use EFS to have shared workspace for all ec2 machines, although creating docker images on EFS was quite slow compared to EBS, that's why i moved my workspaces out to EBS

[zakariaboualaid]

I have the same issue. Docker on OSX.
Docker 17.12.0-ce
Docker plugin : 1.1.2
Jenkins : 2.104

Hi guys - is there any other workaround for this issue? @midN's didn't go well in my context.

[carlossg]

In order to avoid more _me too_ comments this is an issue with docker-workflow-plugin see https://issues.jenkins-ci.org/browse/JENKINS-49063 which is already merged in docker-workflow-plugin

See @midN comment https://github.com/jenkinsci/docker/issues/626#issuecomment-358331311 for workaround on ECS. Similar in other environments

[eduardorn]

Is this fixed? Without workarounds?
I'm still seeing issues, and I'm just using a sample Jenkins file from Jenkins website:

``` pipeline {
agent none
stages {
stage('Example Build') {
agent { docker 'maven:3-alpine' }
steps {
echo 'Hello, Maven'
sh 'mvn --version'
}
}
stage('Example Test') {
agent { docker 'openjdk:8-jre' }
steps {
echo 'Hello, JDK'
sh 'java -version'
}
}
}
}

Running shell script
sh: can't create /home/jenkins/workspace/test@tmp/durable-1878c51f/jenkins-log.txt: nonexistent directory
sh: can't create /home/jenkins/workspace/test@tmp/durable-1878c51f/jenkins-result.txt.tmp: nonexistent directory
mv: can't rename '/home/jenkins/workspace/test@tmp/durable-1878c51f/jenkins-result.txt.tmp': No such file or directory
```

[svc-cloudops-bose]

This is still not fixed, so why is it closed?

[7AC]

@carlossg @ndeloof is this supposed to be fixed by https://github.com/jenkinsci/docker-workflow-plugin/commit/f3428880a2780ad68f589ebcb732fbd2b862fcdf ? Looks like https://issues.jenkins-ci.org/browse/JENKINS-49063 points to https://github.com/jenkinsci/docker-workflow-plugin/pull/128 but on both this and the issue on jenkins-ci.org people are complaining that it's not fixed (that's what I'm seeing too).

[bcolferzd]

Your image may be running as root and not jenkins this means that any job that runs a jenkins may not be able to create a new file in the necessary directories.

[at-vinhhuynh]

Same issue, after few hours research, I got a magic fixed.
have you try to add label is slave machine running?

node (label: 'master') {
    ...
}

[Sadzeih]

We're having the issue too, with a descriptive pipeline and using the docker agent. The workaround proposed wouldn't work in our case though so it would be great if this was fixed.

[javiersanp]

Same issue here. As you can see in this output, Jenkins tries to write in "$WORKSPACE@tmp/durable-70e158c6" inside the alpine container while this directory don't exists, it was "$WORKSPACE@tmp/durable-8eefd86a" in the parent agent.

Running in Durability level: MAX_SURVIVABILITY
[Pipeline] Start of Pipeline
[Pipeline] node

Running on slave-000kjwxb51gdi on LocalDockerHost in /home/jenkins/workspace/prueba-pipeline
[Pipeline] {
[Pipeline] sh

+ id
uid=1000(jenkins) gid=1000(jenkins) groups=1000(jenkins)
[Pipeline] sh
+ ls -l /home/jenkins/workspace/prueba-pipeline@tmp
total 4
drwxr-sr-x    2 jenkins  jenkins       4096 Jul  4 11:04 durable-8eefd86a
[Pipeline] sh
+ docker inspect -f . alpine:3.9
.
[Pipeline] withDockerContainer
slave-000kjwxb51gdi on LocalDockerHost seems to be running inside container 862f601a536f9678f43a29d0dec0d3c3b89bb439cde8f174bcd61ecff3512cd3
but /home/jenkins/workspace/prueba-pipeline could not be found among [/git]
but /home/jenkins/workspace/prueba-pipeline@tmp could not be found among [/git]
$ docker run -t -d -u 1000:1000 -w /home/jenkins/workspace/prueba-pipeline -v /home/jenkins/workspace/prueba-pipeline:/home/jenkins/workspace/prueba-pipeline:rw,z -v /home/jenkins/workspace/prueba-pipeline@tmp:/home/jenkins/workspace/prueba-pipeline@tmp:rw,z -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** alpine:3.9 cat

$ docker top 74db144b89cbd0da0c9245c713d2fdf67d37e0c7eca5d9fcd2693d1e754c10e6 -eo pid,comm
[Pipeline] {
[Pipeline] echo
hola mundo
[Pipeline] sh
sh: can't create /home/jenkins/workspace/prueba-pipeline@tmp/durable-70e158c6/jenkins-log.txt: nonexistent directory
sh: can't create /home/jenkins/workspace/prueba-pipeline@tmp/durable-70e158c6/jenkins-result.txt.tmp: nonexistent directory
mv: can't rename '/home/jenkins/workspace/prueba-pipeline@tmp/durable-70e158c6/jenkins-result.txt.tmp': No such file or directory

[RobGoretsky]

I hit this issue as well after upgrading Jenkins and the Docker plugin. My workaround was to make similar changes as those suggested by @midN , but within the "Configure Jenkins" UI.
There, I changed the Container Template Working Directory from /home/jenkins/agent to just /home/jenkins so that the directory /home/jenkins/workspace is at the exact same location in the child and parent containers.