odo create fails with Access Denied on windows when used with --starter and existing devfile

Created on 17 May 2021 · 23Comments · Source: openshift/odo

/kind bug
/area windows

What versions of software are you using?

Operating System: windows 10

Output of odo version: master

How did you run odo exactly?

$ odo create ahsdlk --project qwerty --starter

Actual behavior

Manually

$ mkdir tempp && cd tempp
$ odo project create qwerty
$ cp $ODO_PROJECT/test/example/source/devfiles/springboot/devfile-with-subdir.yaml   devfile.yaml
$ odo create ahsdlk --project qwerty --starter

Devfile Object Validation
V  Creating a devfile component from devfile path: C:\Users\Admin\tempp\devfile.yaml [0ns]
Validating if devfile name is correct  ...
 V  Validating if devfile name is correct [0ns]
Starter Project

 X  Downloading starter project springbootproject from https://github.com/odo-devfiles/springboot-ex.git  
 X  CreateFile C:\Users\Admin\699881763\src\main: Access is denied.

Expected behavior

odo create should pass without any error while having a devfile in current directory in windows

Any logs, error output, etc?

logs from automated test

[ssh:Windows 10] [2] + Failure [2.698 seconds]
[ssh:Windows 10] [2] odo devfile create command tests
[ssh:Windows 10] [2] C:/Users/Admin/amqp_ci_rcv_odo-windows-e2e-pr-build_PR_4658/repo/tests/integration/devfile/cmd_devfile_create_test.go:19
[ssh:Windows 10] [2]   When executing odo create with devfile component, --starter flag and subDir has a valid value
[ssh:Windows 10] [2]   C:/Users/Admin/amqp_ci_rcv_odo-windows-e2e-pr-build_PR_4658/repo/tests/integration/devfile/cmd_devfile_create_test.go:457
[ssh:Windows 10] [2]     should only extract the specified path in the subDir field [It]
[ssh:Windows 10] [2]     C:/Users/Admin/amqp_ci_rcv_odo-windows-e2e-pr-build_PR_4658/repo/tests/integration/devfile/cmd_devfile_create_test.go:458
[ssh:Windows 10] [2] 
[ssh:Windows 10] [2]     No future change is possible.  Bailing out early after 0.635s.
[ssh:Windows 10] [2]     Running odo.exe with args [odo create zphazc --project cmd-devfile-create-test458pxg --starter]
[ssh:Windows 10] [2]     Expected
[ssh:Windows 10] [2]         <int>: 1
[ssh:Windows 10] [2]     to match exit code:
[ssh:Windows 10] [2]         <int>: 0
[ssh:Windows 10] [2] 
[ssh:Windows 10] [2]     C:/Users/Admin/amqp_ci_rcv_odo-windows-e2e-pr-build_PR_4658/repo/tests/helper/helper_run.go:35
[ssh:Windows 10] [2] ------------------------------
[ssh:Windows 10] [2] Created dir: C:\Users\Admin\391290992
[ssh:Windows 10] [2] Setting KUBECONFIG=C:\Users\Admin\391290992\config
[ssh:Windows 10] [2] Creating a new project: cmd-devfile-create-test458pxg
[ssh:Windows 10] [2] Running odo.exe with args [odo project create cmd-devfile-create-test458pxg -w -v4]
[ssh:Windows 10] [2] [odo] I0517 06:38:59.036342    8236 util.go:761] Cached response used.
[ssh:Windows 10] [2] [odo] I0517 06:38:58.999632    8236 preference.go:220] The path for preference file is \cygdrive\c\Users\Admin\amqp_ci_rcv_odo-windows-e2e-pr-build_PR_4658\repo\artifacts\.odo\preference.yaml
[ssh:Windows 10] [2] [odo] I0517 06:38:59.037499    8236 runnable.go:46] Skipping telemetry question because there is no terminal (tty)
[ssh:Windows 10] [2] [odo]  -  Waiting for project to come up  ...
[ssh:Windows 10] [2] [odo] I0517 06:38:59.066002    8236 kclient.go:202] Checking if "projects" resource supported
[ssh:Windows 10] [2] [odo] I0517 06:39:00.598975    8236 projects.go:190] Status of creation of project cmd-devfile-create-test458pxg is Active
[ssh:Windows 10] [2] [odo] I0517 06:39:00.598975    8236 projects.go:195] Project cmd-devfile-create-test458pxg now exists
[ssh:Windows 10] [2] [odo] I0517 06:39:00.605008    8236 namespace.go:181] Status of creation of service account &ServiceAccount{ObjectMeta:{default  cmd-devfile-create-test458pxg /api/v1/namespaces/cmd-devfile-create-test458pxg/serviceaccounts/default 0b1889b0-562f-4d01-93db-ec51e93e02b6 21179423 0 2021-05-17 06:36:57 +0000 GMT <nil> <nil> map[] map[] [] []  []},Secrets:[]ObjectReference{ObjectReference{Kind:,Namespace:,Name:default-token-ql47s,UID:,APIVersion:,ResourceVersion:,FieldPath:,},ObjectReference{Kind:,Namespace:,Name:default-dockercfg-2mphb,UID:,APIVersion:,ResourceVersion:,FieldPath:,},},ImagePullSecrets:[]LocalObjectReference{LocalObjectReference{Name:default-dockercfg-2mphb,},},AutomountServiceAccountToken:nil,} is ready
[ssh:Windows 10] [2] [odo] 
 V  Waiting for project to come up [2s]
[ssh:Windows 10] [2] [odo]  V  Project 'cmd-devfile-create-test458pxg' is ready for use
[ssh:Windows 10] [2] [odo]  V  New project created and now using project: cmd-devfile-create-test458pxg
[ssh:Windows 10] [2] [odo] I0517 06:39:00.620664    8236 segment.go:207] Sending telemetry disabled by ODO_DISABLE_TELEMETRY=true
[ssh:Windows 10] [2] [odo] I0517 06:39:00.620664    8236 odo.go:73] Could not get the latest release information in time. Never mind, exiting gracefully :)
[ssh:Windows 10] [2] Current working dir: C:\Users\Admin\amqp_ci_rcv_odo-windows-e2e-pr-build_PR_4658\repo\tests\integration\devfile
[ssh:Windows 10] [2] Setting current dir to: C:\Users\Admin\391290992
[ssh:Windows 10] [2] Created dir: C:\Users\Admin\250328335
[ssh:Windows 10] [2] Setting current dir to: C:\Users\Admin\250328335
[ssh:Windows 10] [2] Running odo.exe with args [odo create zphazc --project cmd-devfile-create-test458pxg --starter]
[ssh:Windows 10] [2] [odo] I0517 06:39:00.814679    6404 util.go:761] Cached response used.
[ssh:Windows 10] [2] [odo] I0517 06:39:00.785526    6404 util.go:422] path .odo\config.yaml doesn't exist, skipping it
[ssh:Windows 10] [2] [odo] I0517 06:39:00.814679    6404 util.go:422] path .odo\env\env.yaml doesn't exist, skipping it
[ssh:Windows 10] [2] [odo] I0517 06:39:00.814679    6404 util.go:422] path .odo\env\env.yaml doesn't exist, skipping it
[ssh:Windows 10] [2] [odo] Devfile Object Validation
[ssh:Windows 10] [2] [odo]  -  Creating a devfile component from devfile path: C:\Users\Admin\250328335\devfile.yaml  ...
[ssh:Windows 10] [2] [odo] 
 V  Creating a devfile component from devfile path: C:\Users\Admin\250328335\devfile.yaml [0ns]
[ssh:Windows 10] [2] [odo] Validation
[ssh:Windows 10] [2] [odo]  -  Validating if devfile name is correct  ...
[ssh:Windows 10] [2] [odo] 
 V  Validating if devfile name is correct [0ns]
[ssh:Windows 10] [2] [odo] I0517 06:39:00.815837    6404 content.go:33] converted devfile YAML to JSON
[ssh:Windows 10] [2] [odo] I0517 06:39:00.815837    6404 apiVersion.go:42] devfile schemaVersion: '2.0.0'
[ssh:Windows 10] [2] [odo] I0517 06:39:00.815837    6404 helper.go:41] devfile apiVersion '2.0.0' is supported
[ssh:Windows 10] [2] [odo] I0517 06:39:00.821609    6404 schema.go:46] validated devfile schema
[ssh:Windows 10] [2] [odo] I0517 06:39:00.822655    6404 validate.go:50] Successfully validated devfile sections
[ssh:Windows 10] [2] [odo] I0517 06:39:00.822655    6404 preference.go:220] The path for preference file is C:\Users\Admin\391290992\preference.yaml
[ssh:Windows 10] [2] [odo] 
[ssh:Windows 10] [2] [odo] Starter Project
[ssh:Windows 10] [2] [odo]  -  Downloading starter project springbootproject from https://github.com/odo-devfiles/springboot-ex.git  ...
[ssh:Windows 10] [2] [odo] 
 X  Downloading starter project springbootproject from https://github.com/odo-devfiles/springboot-ex.git [455ms]
[ssh:Windows 10] [2] [odo] I0517 06:39:01.278830    6404 segment.go:207] Sending telemetry disabled by ODO_DISABLE_TELEMETRY=true
[ssh:Windows 10] [2] [odo]  X  CreateFile C:\Users\Admin\699881763\src\main: Access is denied.
[ssh:Windows 10] [2] Setting current dir to: C:\Users\Admin\391290992
[ssh:Windows 10] [2] Deleting project: cmd-devfile-create-test458pxg
[ssh:Windows 10] [2] Running odo.exe with args [odo project delete cmd-devfile-create-test458pxg -f]
[ssh:Windows 10] [2] [odo] I0517 06:39:01.485862    9016 kclient.go:202] Checking if "projects" resource supported
[ssh:Windows 10] [2] [odo] I0517 06:39:01.499053    9016 util.go:761] Cached response used.
[ssh:Windows 10] [2] [odo] I0517 06:39:01.528603    9016 kclient.go:202] Checking if "projects" resource supported
[ssh:Windows 10] [2] [odo]  V  Deleted project : cmd-devfile-create-test458pxg
[ssh:Windows 10] [2] [odo]  !  Warning! Projects are deleted from the cluster asynchronously. Odo does its best to delete the project. Due to multi-tenant clusters, the project may still exist on a different node.
[ssh:Windows 10] [2] [odo] I0517 06:39:01.552926    9016 segment.go:207] Sending telemetry disabled by ODO_DISABLE_TELEMETRY=true
[ssh:Windows 10] [2] [odo] I0517 06:39:01.552926    9016 odo.go:73] Could not get the latest release information in time. Never mind, exiting gracefully :)
[ssh:Windows 10] [2] Setting current dir to: C:\Users\Admin\amqp_ci_rcv_odo-windows-e2e-pr-build_PR_4658\repo\tests\integration\devfile
[ssh:Windows 10] [2] Deleting dir: C:\Users\Admin\391290992

areWindows kinbug

Source

anandrkskd

All 23 comments

Trying on master build manually
Note: Redacted step to ssh into a windows node

First let's try with as it happens in the example given

$ cp -avrf ../tests/examples/source/devfiles/springboot/devfile-with-subDir.yaml ./devfile.yaml
'../tests/examples/source/devfiles/springboot/devfile-with-subDir.yaml' -> './devfile.yaml'
$ ../odo create ahsdlk --starter
Devfile Object Validation
 V  Creating a devfile component from devfile path: C:\Users\Admin\test\odo\prj\devfile.yaml [0ns]
Validation
 V  Validating if devfile name is correct [0ns]

Starter Project
 X  Downloading starter project springbootproject from https://github.com/odo-devfiles/springboot-ex.git [493ms]
 X  CreateFile C:\cygwin\tmp\957292071\src\main: Access is denied.


$ rm -rf * ./.*

Now lets try with different devfile eg

$ cp -avrf ../tests/examples/source/devfiles/nodejs/devfile.yaml ./devfile.yaml
'../tests/examples/source/devfiles/nodejs/devfile.yaml' -> './devfile.yaml'

$ ../odo create ahsdlk --starter
Devfile Object Validation
 V  Creating a devfile component from devfile path: C:\Users\Admin\test\odo\prj\devfile.yaml [26ms]
Validation
 V  Validating if devfile name is correct [0ns]

Starter Project
 V  Downloading starter project nodejs-starter from https://github.com/odo-devfiles/nodejs-ex.git [523ms]

Please use `odo push` command to create the component with source deployed

$ rm -rf * ./.*

What about a different sprintboot devfile

$ cp -avrf ../tests/examples/source/devfiles/springboot/devfile.yaml ./devfile.yaml
'../tests/examples/source/devfiles/springboot/devfile.yaml' -> './devfile.yaml'

$ ../odo create ahsdlk --starter
Devfile Object Validation
 V  Creating a devfile component from devfile path: C:\Users\Admin\test\odo\prj\devfile.yaml [13ms]
Validation
 V  Validating if devfile name is correct [60ms]

Starter Project
 V  Downloading starter project springbootproject from https://github.com/odo-devfiles/springboot-ex.git [796ms]

Please use `odo push` command to create the component with source deployed

The normal command seems to work fine

$ ../odo create nodejs --starter
Devfile Object Validation
 V  Checking devfile existence [0ns]
 V  Creating a devfile component from registry: DefaultDevfileRegistry [1ms]
Validation
 V  Validating if devfile name is correct [0ns]

Starter Project
 V  Downloading starter project nodejs-starter from https://github.com/odo-devfiles/nodejs-ex.git [291ms]

Please use `odo push` command to create the component with source deployed

With this, we can conclude this is specifically happening in subdir devfile on windows

mohammedzee1000 on 26 May 2021

Further investigation reveals that the subdir is getting copied

$ ls
devfile.yaml  java  resources

This is what is present in https://github.com/odo-devfiles/springboot-ex/tree/master/src/main.
I verified that all files are present

mohammedzee1000 on 26 May 2021

Right now i have build an odo bin with log.infof in every other line. Once i can get this into win m/c and run it, il paste the logs here

mohammedzee1000 on 26 May 2021

Additional logging:

$ ../odo create ahsdlk --starter -v 9
I0526 11:54:04.589815    3804 preference.go:220] The path for preference file is C:\Users\Admin\.odo\preference.yaml
I0526 11:54:04.631025    3804 util.go:423] path .odo\config.yaml doesn't exist, skipping it
I0526 11:54:04.631025    3804 util.go:423] path .odo\env\env.yaml doesn't exist, skipping it
I0526 11:54:04.631025    3804 util.go:423] path .odo\env\env.yaml doesn't exist, skipping it
Devfile Object Validation
 -  Creating a devfile component from devfile path: C:\Users\Admin\test\odo\prj\devfile.yaml  ...
 V  Creating a devfile component from devfile path: C:\Users\Admin\test\odo\prj\devfile.yaml [7ms]
Validation
 -  Validating if devfile name is correct  ...
 V  Validating if devfile name is correct [4ms]
I0526 11:54:04.660137    3804 content.go:33] converted devfile YAML to JSON
I0526 11:54:04.661192    3804 apiVersion.go:42] devfile schemaVersion: '2.0.0'
I0526 11:54:04.661192    3804 helper.go:41] devfile apiVersion '2.0.0' is supported
I0526 11:54:04.671089    3804 schema.go:46] validated devfile schema
I0526 11:54:04.673704    3804 validate.go:50] Successfully validated devfile sections
I0526 11:54:04.673704    3804 preference.go:220] The path for preference file is C:\Users\Admin\.odo\preference.yaml

Starter Project
Downloading git based starter project
 -  Downloading starter project springbootproject from https://github.com/odo-devfiles/springboot-ex.git  ...
subdir /src/main detected
project will be temporarily cloned to C:\cygwin\tmp\016220831
I0526 11:54:04.691574    3804 util.go:762] Cached response used.
going for git based subdir
gitsubdir cloned at C:\cygwin\tmp\016220831 to be cloned to C:\Users\Admin\test\odo\prj with subdir /src/main
opening subdir in srcpath
opened successful
reading subdir contents
done successfully
working on file java
old path of file C:\cygwin\tmp\016220831\src\main\java
attempting to copy as dir to dest
getting src info for C:\cygwin\tmp\016220831\src\main\java
done, now attempting to create all dirs at dst
done, now Reading dir from src
working on dir/file named com
attempting to copy dir C:\cygwin\tmp\016220831\src\main\java/com to C:\Users\Admin\test\odo\prj\java/com
getting src info for C:\cygwin\tmp\016220831\src\main\java/com
done, now attempting to create all dirs at dst
done, now Reading dir from src
working on dir/file named example
attempting to copy dir C:\cygwin\tmp\016220831\src\main\java/com/example to C:\Users\Admin\test\odo\prj\java/com/example
getting src info for C:\cygwin\tmp\016220831\src\main\java/com/example
done, now attempting to create all dirs at dst
done, now Reading dir from src
working on dir/file named demo
attempting to copy dir C:\cygwin\tmp\016220831\src\main\java/com/example/demo to C:\Users\Admin\test\odo\prj\java/com/example/
demo
getting src info for C:\cygwin\tmp\016220831\src\main\java/com/example/demo
done, now attempting to create all dirs at dst
done, now Reading dir from src
working on dir/file named DemoApplication.java
attempting to copy file C:\cygwin\tmp\016220831\src\main\java/com/example/demo/DemoApplication.java to C:\Users\Admin\test\odo
\prj\java/com/example/demo/DemoApplication.java
opening src file C:\cygwin\tmp\016220831\src\main\java/com/example/demo/DemoApplication.java
creating dest file at C:\Users\Admin\test\odo\prj\java/com/example/demo/DemoApplication.java
attempting to copy file C:\cygwin\tmp\016220831\src\main\java/com/example/demo/DemoApplication.java to C:\Users\Admin\test\odo
\prj\java/com/example/demo/DemoApplication.java
done
done
done
done
working on file resources
old path of file C:\cygwin\tmp\016220831\src\main\resources
attempting to copy as dir to dest
getting src info for C:\cygwin\tmp\016220831\src\main\resources
done, now attempting to create all dirs at dst
done, now Reading dir from src
working on dir/file named application.properties
attempting to copy file C:\cygwin\tmp\016220831\src\main\resources/application.properties to C:\Users\Admin\test\odo\prj\resou
rces/application.properties
opening src file C:\cygwin\tmp\016220831\src\main\resources/application.properties
creating dest file at C:\Users\Admin\test\odo\prj\resources/application.properties
attempting to copy file C:\cygwin\tmp\016220831\src\main\resources/application.properties to C:\Users\Admin\test\odo\prj\resou
rces/application.properties
done
 X  Downloading starter project springbootproject from https://github.com/odo-devfiles/springboot-ex.git [498ms]
 X  CreateFile C:\cygwin\tmp\016220831\src\main: Access is denied.

mohammedzee1000 on 26 May 2021

Scoping is still in progress

mohammedzee1000 on 26 May 2021

Ok one of my attempts to gather more info succeeded. So we can say its a flake
ie

$ ../odo create ahsdlk --starter -v 9
...
done looping over all files, now attempting to remove files from srcpath C:\cygwin\tmp\289222079
done with git subdir action
 V  Downloading starter project springbootproject from https://github.com/odo-devfiles/springboot-ex.git [526ms]
done downloading git starter project

Please use `odo push` command to create the component with source deployed
I0527 06:42:31.653013    8972 odo.go:75] Could not get the latest release information in time. Never mind, exiting gracefully 
:)

mohammedzee1000 on 27 May 2021

Another attempt failed and here are the logs:

$ ../odo create ahsdlk --starter -v 9
I0527 06:43:49.767331    9108 preference.go:220] The path for preference file is C:\Users\Admin\.odo\preference.yaml
I0527 06:43:49.800486    9108 util.go:762] Cached response used.
I0527 06:43:49.801549    9108 util.go:423] path .odo\config.yaml doesn't exist, skipping it
I0527 06:43:49.802613    9108 util.go:423] path .odo\env\env.yaml doesn't exist, skipping it
I0527 06:43:49.802613    9108 util.go:423] path .odo\env\env.yaml doesn't exist, skipping it
Devfile Object Validation
 -  Creating a devfile component from devfile path: C:\Users\Admin\mzee1000\odo_check\odo\prj\devfile.yaml  ...
 V  Creating a devfile component from devfile path: C:\Users\Admin\mzee1000\odo_check\odo\prj\devfile.yaml [8ms]
Validation
 -  Validating if devfile name is correct  ...
 V  Validating if devfile name is correct [2ms]
I0527 06:43:49.833603    9108 content.go:33] converted devfile YAML to JSON
I0527 06:43:49.833603    9108 apiVersion.go:42] devfile schemaVersion: '2.0.0'
I0527 06:43:49.834616    9108 helper.go:41] devfile apiVersion '2.0.0' is supported
I0527 06:43:49.841839    9108 schema.go:46] validated devfile schema
I0527 06:43:49.843438    9108 validate.go:50] Successfully validated devfile sections
I0527 06:43:49.844435    9108 preference.go:220] The path for preference file is C:\Users\Admin\.odo\preference.yaml

Starter Project
Downloading git based starter project
 -  Downloading starter project springbootproject from https://github.com/odo-devfiles/springboot-ex.git  ...
subdir /src/main detected
project will be temporarily cloned to C:\cygwin\tmp\165401215
going for git based subdir
gitsubdir cloned at C:\cygwin\tmp\165401215 to be cloned to C:\Users\Admin\mzee1000\odo_check\odo\prj with subdir /src/main
opening subdir in srcpath
opened successful
reading subdir contents
done successfully, now looping over files and copying subdir /src/main to destination
working on file java
old path of file C:\cygwin\tmp\165401215\src\main\java
attempting to copy as dir to dest
getting src info for C:\cygwin\tmp\165401215\src\main\java
done, now attempting to create all dirs at dst
done, now Reading dir from src
working on dir/file named com
attempting to copy dir C:\cygwin\tmp\165401215\src\main\java/com to C:\Users\Admin\mzee1000\odo_check\odo\prj\java/com
getting src info for C:\cygwin\tmp\165401215\src\main\java/com
done, now attempting to create all dirs at dst
done, now Reading dir from src
working on dir/file named example
attempting to copy dir C:\cygwin\tmp\165401215\src\main\java/com/example to C:\Users\Admin\mzee1000\odo_check\odo\prj\java/com
/example
getting src info for C:\cygwin\tmp\165401215\src\main\java/com/example
done, now attempting to create all dirs at dst
done, now Reading dir from src
working on dir/file named demo
attempting to copy dir C:\cygwin\tmp\165401215\src\main\java/com/example/demo to C:\Users\Admin\mzee1000\odo_check\odo\prj\jav
a/com/example/demo
getting src info for C:\cygwin\tmp\165401215\src\main\java/com/example/demo
done, now attempting to create all dirs at dst
done, now Reading dir from src
working on dir/file named DemoApplication.java
attempting to copy file C:\cygwin\tmp\165401215\src\main\java/com/example/demo/DemoApplication.java to C:\Users\Admin\mzee1000
\odo_check\odo\prj\java/com/example/demo/DemoApplication.java
opening src file C:\cygwin\tmp\165401215\src\main\java/com/example/demo/DemoApplication.java
creating dest file at C:\Users\Admin\mzee1000\odo_check\odo\prj\java/com/example/demo/DemoApplication.java
attempting to copy file C:\cygwin\tmp\165401215\src\main\java/com/example/demo/DemoApplication.java to C:\Users\Admin\mzee1000
\odo_check\odo\prj\java/com/example/demo/DemoApplication.java
done
done
done
done
working on file resources
old path of file C:\cygwin\tmp\165401215\src\main\resources
attempting to copy as dir to dest
getting src info for C:\cygwin\tmp\165401215\src\main\resources
done, now attempting to create all dirs at dst
done, now Reading dir from src
working on dir/file named application.properties
attempting to copy file C:\cygwin\tmp\165401215\src\main\resources/application.properties to C:\Users\Admin\mzee1000\odo_check
\odo\prj\resources/application.properties
opening src file C:\cygwin\tmp\165401215\src\main\resources/application.properties
creating dest file at C:\Users\Admin\mzee1000\odo_check\odo\prj\resources/application.properties
attempting to copy file C:\cygwin\tmp\165401215\src\main\resources/application.properties to C:\Users\Admin\mzee1000\odo_check
\odo\prj\resources/application.properties
done
done looping over all files, now attempting to remove files from srcpath C:\cygwin\tmp\165401215
 X  Downloading starter project springbootproject from https://github.com/odo-devfiles/springboot-ex.git [574ms]
 X  CreateFile C:\cygwin\tmp\165401215\src\main: Access is denied.

Based on this, I can conclude that sometimes there is a problem deleting the subdir in temp location

mohammedzee1000 on 27 May 2021

Summary: Based on above logs, I can conclude that

The error condition is a flake.
It fails while removing content from tmpdir i.e feliure happens on the line below
https://github.com/openshift/odo/blob/3196871acd1ee55dcc96d2bf8b1d6bfb247dfc8b/pkg/util/util.go#L1448

mohammedzee1000 on 27 May 2021

What I don't get tho is why the error says CreateFile C:\cygwin\tmp\165401215\src\main: Access is denied.. in the RemoveAll function. That does not make sense why it is trying to create something while removing

mohammedzee1000 on 27 May 2021

attempting to copy file C:\cygwin\tmp\165401215\src\main\resources/application.properties to C:\Users\Admin\mzee1000\odo_check
\odo\prj\resources/application.properties

CreateFile C:\cygwin\tmp\165401215\src\main: Access is denied.

It seems to me it is creating the file C:\Users\Admin\mzee1000\odo_check\odo\prj\resources/application.properties
but cannot read access the file C:\cygwin\tmp\165401215\src\main\resources/application.properties

Where is the code displaying attempting to copy file ... ?

feloy on 27 May 2021

attempting to copy file C:\cygwin\tmp\165401215\src\main\resources/application.properties to C:\Users\Admin\mzee1000\odo_check
\odo\prj\resources/application.properties

CreateFile C:\cygwin\tmp\165401215\src\main: Access is denied.
It seems to me it is creating the file C:\Users\Admin\mzee1000\odo_check\odo\prj\resources/application.properties
but cannot read access the file C:\cygwin\tmp\165401215\src\main\resources/application.properties

Where is the code displaying attempting to copy file ... ?

It is there

attempting to copy file C:\cygwin\tmp\165401215\src\main\resources/application.properties to C:\Users\Admin\mzee1000\odo_check
\odo\prj\resources/application.properties
done

If it failed to copy any of the files, it would never reach line done looping over all files, now attempting to remove files from srcpath blah as the code is littered with return err
`

mohammedzee1000 on 27 May 2021

Either way, something is interfering randomly with the process

mohammedzee1000 on 27 May 2021

It might be because the directory is not closed before deleting https://github.com/openshift/odo/blob/3196871acd1ee55dcc96d2bf8b1d6bfb247dfc8b/pkg/util/util.go#L1418 Please try adding defer outputDirRead.Close() after this line.

I faced this problem recently and noticed that linux can handle deletions without closing the file, most of the time, but windows system cannot.

mik-dass on 27 May 2021

👍1

hmm ok lemme try th

It might be because the directory is not closed before deleting

https://github.com/openshift/odo/blob/3196871acd1ee55dcc96d2bf8b1d6bfb247dfc8b/pkg/util/util.go#L1418

Please try adding defer outputDirRead.Close() after this line.
I recently faced this problem and noticed that linux can handle deletion with closing the file but windows system cannot.

ok lemme try that

mohammedzee1000 on 27 May 2021

btw here are logs with klog

I0527 08:31:47.269855    2988 preference.go:220] The path for preference file is C:\Users\Admin\.odo\preference.yaml
I0527 08:31:47.316780    2988 util.go:423] path .odo\config.yaml doesn't exist, skipping it
I0527 08:31:47.316780    2988 util.go:423] path .odo\env\env.yaml doesn't exist, skipping it
I0527 08:31:47.316780    2988 util.go:423] path .odo\env\env.yaml doesn't exist, skipping it
Devfile Object Validation
 -  Creating a devfile component from devfile path: C:\Users\Admin\mzee1000\odo_check\odo\prj\devfile.yaml  ...
 V  Creating a devfile component from devfile path: C:\Users\Admin\mzee1000\odo_check\odo\prj\devfile.yaml [0ns]
Validation
 -  Validating if devfile name is correct  ...
 V  Validating if devfile name is correct [0ns]
I0527 08:31:47.317790    2988 content.go:33] converted devfile YAML to JSON
I0527 08:31:47.317790    2988 apiVersion.go:42] devfile schemaVersion: '2.0.0'
I0527 08:31:47.317790    2988 helper.go:41] devfile apiVersion '2.0.0' is supported
I0527 08:31:47.326527    2988 schema.go:46] validated devfile schema
I0527 08:31:47.328542    2988 validate.go:50] Successfully validated devfile sections
I0527 08:31:47.328542    2988 preference.go:220] The path for preference file is C:\Users\Admin\.odo\preference.yaml

Starter Project
Downloading git based starter project
 -  Downloading starter project springbootproject from https://github.com/odo-devfiles/springboot-ex.git  ...
subdir /src/main detected
project will be temporarily cloned to C:\cygwin\tmp\906619339
I0527 08:31:47.384663    2988 util.go:762] Cached response used.
going for git based subdir
gitsubdir cloned at C:\cygwin\tmp\906619339 to be cloned to C:\Users\Admin\mzee1000\odo_check\odo\prj with subdir /src/main
I0527 08:31:47.590190    2988 util.go:1431] opening subdir in srcpath
I0527 08:31:47.591190    2988 util.go:1436] opened successful
reading subdir contents
I0527 08:31:47.591190    2988 util.go:1444] done successfully, now looping over files and copying subdir /src/main to destination
I0527 08:31:47.591190    2988 util.go:1452] gitsubdir: working on file java
gitsubdir: old path of file C:\cygwin\tmp\906619339\src\main\java
I0527 08:31:47.591190    2988 util.go:1457] attempting to copy as dir to dest
I0527 08:31:47.591190    2988 util.go:1321] getting src info for C:\cygwin\tmp\906619339\src\main\java
I0527 08:31:47.591190    2988 util.go:1325] done, now attempting to create all dirs at dst
I0527 08:31:47.591190    2988 util.go:1329] done, now Reading dir from src
I0527 08:31:47.591190    2988 util.go:1334] working on dir/file named com
I0527 08:31:47.591190    2988 util.go:1338] attempting to copy dir C:\cygwin\tmp\906619339\src\main\java/com to C:\Users\Admin\mzee1000\odo_check\odo\prj\java/com
I0527 08:31:47.591190    2988 util.go:1321] getting src info for C:\cygwin\tmp\906619339\src\main\java/com
I0527 08:31:47.591190    2988 util.go:1325] done, now attempting to create all dirs at dst
I0527 08:31:47.591190    2988 util.go:1329] done, now Reading dir from src
I0527 08:31:47.591190    2988 util.go:1334] working on dir/file named example
I0527 08:31:47.591190    2988 util.go:1338] attempting to copy dir C:\cygwin\tmp\906619339\src\main\java/com/example to C:\Users\Admin\mzee1000\odo_check\odo\prj\java/com/example
I0527 08:31:47.591190    2988 util.go:1321] getting src info for C:\cygwin\tmp\906619339\src\main\java/com/example
I0527 08:31:47.591190    2988 util.go:1325] done, now attempting to create all dirs at dst
I0527 08:31:47.592205    2988 util.go:1329] done, now Reading dir from src
I0527 08:31:47.592205    2988 util.go:1334] working on dir/file named demo
I0527 08:31:47.592205    2988 util.go:1338] attempting to copy dir C:\cygwin\tmp\906619339\src\main\java/com/example/demo to C:\Users\Admin\mzee1000\odo_check\odo\prj\java/com/example/demo
I0527 08:31:47.592205    2988 util.go:1321] getting src info for C:\cygwin\tmp\906619339\src\main\java/com/example/demo
I0527 08:31:47.592205    2988 util.go:1325] done, now attempting to create all dirs at dst
I0527 08:31:47.592205    2988 util.go:1329] done, now Reading dir from src
I0527 08:31:47.592205    2988 util.go:1334] working on dir/file named DemoApplication.java
I0527 08:31:47.592205    2988 util.go:1344] attempting to copy file C:\cygwin\tmp\906619339\src\main\java/com/example/demo/DemoApplication.java to C:\Users\Admin\mzee1000\odo_check\odo\prj\java/com/example/demo/DemoApplication.java
I0527 08:31:47.592205    2988 util.go:1276] opening src file C:\cygwin\tmp\906619339\src\main\java/com/example/demo/DemoApplication.java
I0527 08:31:47.592205    2988 util.go:1287] creating dest file at C:\Users\Admin\mzee1000\odo_check\odo\prj\java/com/example/demo/DemoApplication.java
I0527 08:31:47.592205    2988 util.go:1299] attempting to copy file C:\cygwin\tmp\906619339\src\main\java/com/example/demo/DemoApplication.java to C:\Users\Admin\mzee1000\odo_check\odo\prj\java/com/example/demo/DemoApplication.java
I0527 08:31:47.593236    2988 util.go:1306] done copying
I0527 08:31:47.593236    2988 util.go:1348] done with file
I0527 08:31:47.593236    2988 util.go:1342] done with dir
I0527 08:31:47.593236    2988 util.go:1342] done with dir
I0527 08:31:47.593236    2988 util.go:1342] done with dir
I0527 08:31:47.593236    2988 util.go:1452] gitsubdir: working on file resources
gitsubdir: old path of file C:\cygwin\tmp\906619339\src\main\resources
I0527 08:31:47.593236    2988 util.go:1457] attempting to copy as dir to dest
I0527 08:31:47.593236    2988 util.go:1321] getting src info for C:\cygwin\tmp\906619339\src\main\resources
I0527 08:31:47.593236    2988 util.go:1325] done, now attempting to create all dirs at dst
I0527 08:31:47.593767    2988 util.go:1329] done, now Reading dir from src
I0527 08:31:47.593767    2988 util.go:1334] working on dir/file named application.properties
I0527 08:31:47.593767    2988 util.go:1344] attempting to copy file C:\cygwin\tmp\906619339\src\main\resources/application.properties to C:\Users\Admin\mzee1000\odo_check\odo\prj\resources/application.properties
I0527 08:31:47.593767    2988 util.go:1276] opening src file C:\cygwin\tmp\906619339\src\main\resources/application.properties
I0527 08:31:47.593767    2988 util.go:1287] creating dest file at C:\Users\Admin\mzee1000\odo_check\odo\prj\resources/application.properties
I0527 08:31:47.593767    2988 util.go:1299] attempting to copy file C:\cygwin\tmp\906619339\src\main\resources/application.properties to C:\Users\Admin\mzee1000\odo_check\odo\prj\resources/application.properties
I0527 08:31:47.593767    2988 util.go:1306] done copying
I0527 08:31:47.593767    2988 util.go:1348] done with file
I0527 08:31:47.593767    2988 util.go:1467] done looping over all files, now attempting to remove files from srcpath C:\cygwin\tmp\906619339
 X  Downloading starter project springbootproject from https://github.com/odo-devfiles/springboot-ex.git [269ms]
 X  CreateFile C:\cygwin\tmp\906619339\src\main: Access is denied.

mohammedzee1000 on 27 May 2021

hmm ok lemme try th

It might be because the directory is not closed before deleting
https://github.com/openshift/odo/blob/3196871acd1ee55dcc96d2bf8b1d6bfb247dfc8b/pkg/util/util.go#L1418

Please try adding defer outputDirRead.Close() after this line.
I recently faced this problem and noticed that linux can handle deletion with closing the file but windows system cannot.

ok lemme try that

If you add a defer Close() here, it will be executed after removeAll(). Instead, you can make a new function that handles all the copy stuff, defering the Close at the end of this sub-function:

func gitSubDir(srcPath, destinationPath, subDir string, fs filesystem.Filesystem) error {
    go StartSignalWatcher(func() {
        err := cleanDir(destinationPath, map[string]bool{
            "devfile.yaml": true,
        }, fs)
        if err != nil {
            klog.V(4).Infof("error %v occurred while calling handleInterruptedSubDir", err)
        }
        err = fs.RemoveAll(srcPath)
        if err != nil {
            klog.V(4).Infof("error %v occurred during temp folder clean up", err)
        }
    })

        copyDirectory(...)

    return fs.RemoveAll(srcPath)
}

feloy on 27 May 2021

hmm ok lemme try th

It might be because the directory is not closed before deleting
https://github.com/openshift/odo/blob/3196871acd1ee55dcc96d2bf8b1d6bfb247dfc8b/pkg/util/util.go#L1418

Please try adding defer outputDirRead.Close() after this line.
I recently faced this problem and noticed that linux can handle deletion with closing the file but windows system cannot.

ok lemme try that

If you add a defer Close() here, it will be executed after removeAll(). Instead, you can make a new function that handles all the copy stuff, defering the Close at the end of this sub-function:
func gitSubDir(srcPath, destinationPath, subDir string, fs filesystem.Filesystem) error {
  go StartSignalWatcher(func() {
      err := cleanDir(destinationPath, map[string]bool{
          "devfile.yaml": true,
      }, fs)
      if err != nil {
          klog.V(4).Infof("error %v occurred while calling handleInterruptedSubDir", err)
      }
      err = fs.RemoveAll(srcPath)
      if err != nil {
          klog.V(4).Infof("error %v occurred during temp folder clean up", err)
      }
  })

        copyDirectory(...)

  return fs.RemoveAll(srcPath)
}

Hmm yea that actually makes sense. Tho as that code is not repeated, I guess we could just do anonymous func

    func() {
        //copy code here
    }()

wdyt @feloy ?

mohammedzee1000 on 27 May 2021

👍1

Ok updated code, will give it spin and report shortly

mohammedzee1000 on 27 May 2021

Awesome seeing consistent success across 15 runs we should be good.

I0527 09:03:39.797868    9396 util.go:762] Cached response used.
I0527 09:03:39.762601    9396 preference.go:220] The path for preference file is C:\Users\Admin\.odo\preference.yaml
I0527 09:03:39.798941    9396 util.go:423] path .odo\config.yaml doesn't exist, skipping it
I0527 09:03:39.798941    9396 util.go:423] path .odo\env\env.yaml doesn't exist, skipping it
I0527 09:03:39.798941    9396 util.go:423] path .odo\env\env.yaml doesn't exist, skipping it
Devfile Object Validation
 -  Creating a devfile component from devfile path: C:\Users\Admin\mzee1000\odo_check\odo\prj\devfile.yaml  ...
 V  Creating a devfile component from devfile path: C:\Users\Admin\mzee1000\odo_check\odo\prj\devfile.yaml [1ms]
Validation
 -  Validating if devfile name is correct  ...
 V  Validating if devfile name is correct [0ns]
I0527 09:03:39.800967    9396 content.go:33] converted devfile YAML to JSON
I0527 09:03:39.800967    9396 apiVersion.go:42] devfile schemaVersion: '2.0.0'
I0527 09:03:39.800967    9396 helper.go:41] devfile apiVersion '2.0.0' is supported
I0527 09:03:39.811503    9396 schema.go:46] validated devfile schema
I0527 09:03:39.813547    9396 validate.go:50] Successfully validated devfile sections
I0527 09:03:39.813547    9396 preference.go:220] The path for preference file is C:\Users\Admin\.odo\preference.yaml

Starter Project
Downloading git based starter project
 -  Downloading starter project springbootproject from https://github.com/odo-devfiles/springboot-ex.git  ...
subdir /src/main detected
project will be temporarily cloned to C:\cygwin\tmp\737665123
going for git based subdir
gitsubdir cloned at C:\cygwin\tmp\737665123 to be cloned to C:\Users\Admin\mzee1000\odo_check\odo\prj with subdir /src/main
I0527 09:03:40.071321    9396 util.go:1432] opening subdir in srcpath
I0527 09:03:40.071321    9396 util.go:1438] opened successful
reading subdir contents
I0527 09:03:40.071321    9396 util.go:1446] done successfully, now looping over files and copying subdir /src/main to destination
I0527 09:03:40.071321    9396 util.go:1454] gitsubdir: working on file java
gitsubdir: old path of file C:\cygwin\tmp\737665123\src\main\java
I0527 09:03:40.071321    9396 util.go:1459] attempting to copy as dir to dest
I0527 09:03:40.071321    9396 util.go:1321] getting src info for C:\cygwin\tmp\737665123\src\main\java
I0527 09:03:40.071321    9396 util.go:1325] done, now attempting to create all dirs at dst
I0527 09:03:40.071321    9396 util.go:1329] done, now Reading dir from src
I0527 09:03:40.071321    9396 util.go:1334] working on dir/file named com
I0527 09:03:40.071321    9396 util.go:1338] attempting to copy dir C:\cygwin\tmp\737665123\src\main\java/com to C:\Users\Admin\mzee1000\odo_check\odo\prj\java/com
I0527 09:03:40.071321    9396 util.go:1321] getting src info for C:\cygwin\tmp\737665123\src\main\java/com
I0527 09:03:40.072358    9396 util.go:1325] done, now attempting to create all dirs at dst
I0527 09:03:40.072358    9396 util.go:1329] done, now Reading dir from src
I0527 09:03:40.072934    9396 util.go:1334] working on dir/file named example
I0527 09:03:40.072934    9396 util.go:1338] attempting to copy dir C:\cygwin\tmp\737665123\src\main\java/com/example to C:\Users\Admin\mzee1000\odo_check\odo\prj\java/com/example
I0527 09:03:40.072934    9396 util.go:1321] getting src info for C:\cygwin\tmp\737665123\src\main\java/com/example
I0527 09:03:40.072934    9396 util.go:1325] done, now attempting to create all dirs at dst
I0527 09:03:40.072934    9396 util.go:1329] done, now Reading dir from src
I0527 09:03:40.072934    9396 util.go:1334] working on dir/file named demo
I0527 09:03:40.072934    9396 util.go:1338] attempting to copy dir C:\cygwin\tmp\737665123\src\main\java/com/example/demo to C:\Users\Admin\mzee1000\odo_check\odo\prj\java/com/example/demo
I0527 09:03:40.072934    9396 util.go:1321] getting src info for C:\cygwin\tmp\737665123\src\main\java/com/example/demo
I0527 09:03:40.072934    9396 util.go:1325] done, now attempting to create all dirs at dst
I0527 09:03:40.072934    9396 util.go:1329] done, now Reading dir from src
I0527 09:03:40.073948    9396 util.go:1334] working on dir/file named DemoApplication.java
I0527 09:03:40.073948    9396 util.go:1344] attempting to copy file C:\cygwin\tmp\737665123\src\main\java/com/example/demo/DemoApplication.java to C:\Users\Admin\mzee1000\odo_check\odo\prj\java/com/example/demo/DemoApplication.java
I0527 09:03:40.073948    9396 util.go:1276] opening src file C:\cygwin\tmp\737665123\src\main\java/com/example/demo/DemoApplication.java
I0527 09:03:40.073948    9396 util.go:1287] creating dest file at C:\Users\Admin\mzee1000\odo_check\odo\prj\java/com/example/demo/DemoApplication.java
I0527 09:03:40.073948    9396 util.go:1299] attempting to copy file C:\cygwin\tmp\737665123\src\main\java/com/example/demo/DemoApplication.java to C:\Users\Admin\mzee1000\odo_check\odo\prj\java/com/example/demo/DemoApplication.java
I0527 09:03:40.073948    9396 util.go:1306] done copying
I0527 09:03:40.074995    9396 util.go:1348] done with file
I0527 09:03:40.074995    9396 util.go:1342] done with dir
I0527 09:03:40.074995    9396 util.go:1342] done with dir
I0527 09:03:40.074995    9396 util.go:1342] done with dir
I0527 09:03:40.074995    9396 util.go:1454] gitsubdir: working on file resources
gitsubdir: old path of file C:\cygwin\tmp\737665123\src\main\resources
I0527 09:03:40.074995    9396 util.go:1459] attempting to copy as dir to dest
I0527 09:03:40.074995    9396 util.go:1321] getting src info for C:\cygwin\tmp\737665123\src\main\resources
I0527 09:03:40.074995    9396 util.go:1325] done, now attempting to create all dirs at dst
I0527 09:03:40.074995    9396 util.go:1329] done, now Reading dir from src
I0527 09:03:40.074995    9396 util.go:1334] working on dir/file named application.properties
I0527 09:03:40.074995    9396 util.go:1344] attempting to copy file C:\cygwin\tmp\737665123\src\main\resources/application.properties to C:\Users\Admin\mzee1000\odo_check\odo\prj\resources/application.properties
I0527 09:03:40.074995    9396 util.go:1276] opening src file C:\cygwin\tmp\737665123\src\main\resources/application.properties
I0527 09:03:40.074995    9396 util.go:1287] creating dest file at C:\Users\Admin\mzee1000\odo_check\odo\prj\resources/application.properties
I0527 09:03:40.076026    9396 util.go:1299] attempting to copy file C:\cygwin\tmp\737665123\src\main\resources/application.properties to C:\Users\Admin\mzee1000\odo_check\odo\prj\resources/application.properties
I0527 09:03:40.076026    9396 util.go:1306] done copying
I0527 09:03:40.076026    9396 util.go:1348] done with file
I0527 09:03:40.076026    9396 util.go:1469] done looping over all files, now attempting to remove files from srcpath C:\cygwin\tmp\737665123
done with git subdir action
 V  Downloading starter project springbootproject from https://github.com/odo-devfiles/springboot-ex.git [269ms]
done downloading git starter project

Please use `odo push` command to create the component with source deployed
I0527 09:03:40.088569    9396 odo.go:75] Could not get the latest release information in time. Never mind, exiting gracefully :)

Let me cleanout the extra logs and i should have pr up for fhis
thanks @feloy @mik-dass

mohammedzee1000 on 27 May 2021

If you add a defer Close() here, it will be executed after removeAll(). Instead, you can make a new function that handles all the copy stuff, defering the Close at the end of this sub-function:

Sorry missed the deletion at the end of the function. Closing the file after the directory read or before the folder deletion might work as well.

mik-dass on 27 May 2021

👍1

That is by far a simpler solution

mohammedzee1000 on 27 May 2021

Though I feel there is one drawback. defer ensures execution of close() in all cases except an actual panic and that could leave us with dangling file pointers in cases of error

mohammedzee1000 on 27 May 2021

Though I feel there is one drawback. defer ensures execution of close() in all cases except an actual panic and that could leave us with dangling file pointers in cases of error

Yes, the advantage os defer is to write the Close() only once, and not for all error/return cases. You also have to think that error/return cases will be added afterwards by future devs, and the risk of forgetting some Close() on these future cases is high. So the introduction of the defer, which is very useful

feloy on 27 May 2021

Was this page helpful?

0 / 5 - 0 ratings