Pkg.jl: `SystemError` when testing package - only occurs when package is added by `Pkg.add`, does not occur when package is added by `Pkg.develop`

Created on 7 Dec 2019 · 3Comments · Source: JuliaLang/Pkg.jl

I can reproduce this on both Julia 1.3 and Julia master on both macOS and Linux.

I have a package in which Pkg.test will pass if I add the package with Pkg.develop and will fail if I add the package with Pkg.add.

This is a pretty nasty bug for me, because it occurs with many of my packages, and it causes those packages to fail NewPkgEval.

MWE

Delete your ~/.julia folder, and then run the following commands:

julia> import Pkg
julia> Pkg.develop(Pkg.PackageSpec(name = "PredictMDAPI"))
julia> Pkg.test("PredictMDAPI") # this works fine

Delete your ~/.julia folder, and then run the following commands:

julia> import Pkg
julia> Pkg.add(Pkg.PackageSpec(name = "PredictMDAPI", rev = "master"))
julia> Pkg.test("PredictMDAPI")
   Testing PredictMDAPI
 Resolving package versions...
  Updating `/private/var/folders/48/n09p39y53bx6dnbnf244s2jc0000gp/T/jl_UtizW9/Project.toml`
  [df4d2c64] + PredictMDAPI v0.3.0-DEV #master (https://github.com/bcbi/PredictMDAPI.jl.git)
  [8dfed614] + Test
 Resolving package versions...
  Updating `/private/var/folders/48/n09p39y53bx6dnbnf244s2jc0000gp/T/jl_UtizW9/Project.toml`
  [df4d2c64] + PredictMDAPI v0.3.0-DEV [`~/.julia/packages/PredictMDAPI/OBU19`]
  [8dfed614] + Test
ERROR: SystemError: opening file "/private/var/folders/48/n09p39y53bx6dnbnf244s2jc0000gp/T/jl_UtizW9/Project.toml": Permission denied
Stacktrace:
 [1] #systemerror#44(::Nothing, ::typeof(systemerror), ::String, ::Bool) at ./error.jl:134
 [2] systemerror at ./error.jl:134 [inlined]
 [3] #open#512(::Nothing, ::Nothing, ::Nothing, ::Bool, ::Nothing, ::typeof(open), ::String) at ./iostream.jl:254
 [4] #open at ./none:0 [inlined]
 [5] #open#271(::Base.Iterators.Pairs{Symbol,Bool,Tuple{Symbol},NamedTuple{(:truncate,),Tuple{Bool}}}, ::typeof(open), ::Pkg.Types.var"#41#43"{Base.GenericIOBuffer{Array{UInt8,1}}}, ::String) at ./io.jl:296
 [6] #open at ./none:0 [inlined]
 [7] write_project at /Users/dilum/dev/julia-releases/julia-1.3.0/usr/share/julia/stdlib/v1.3/Pkg/src/project.jl:176 [inlined]
 [8] #write_project#44(::Bool, ::typeof(Pkg.Types.write_project), ::Pkg.Types.Project, ::Pkg.Types.EnvCache, ::Pkg.Types.EnvCache, ::Pkg.Types.Context) at /Users/dilum/dev/julia-releases/julia-1.3.0/usr/share/julia/stdlib/v1.3/Pkg/src/project.jl:187
 [9] #write_project at ./none:0 [inlined]
 [10] #write_env#141 at /Users/dilum/dev/julia-releases/julia-1.3.0/usr/share/julia/stdlib/v1.3/Pkg/src/Types.jl:1375 [inlined]
 [11] write_env at /Users/dilum/dev/julia-releases/julia-1.3.0/usr/share/julia/stdlib/v1.3/Pkg/src/Types.jl:1373 [inlined]
 [12] #develop#115(::Bool, ::Pkg.BinaryPlatforms.MacOS, ::typeof(Pkg.Operations.develop), ::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}, ::Array{Base.UUID,1}) at /Users/dilum/dev/julia-releases/julia-1.3.0/usr/share/julia/stdlib/v1.3/Pkg/src/Operations.jl:1040
 [13] #develop at /Users/dilum/dev/julia-releases/julia-1.3.0/usr/share/julia/stdlib/v1.3/Pkg/src/API.jl:0 [inlined]
 [14] #develop#17(::Bool, ::Bool, ::Pkg.BinaryPlatforms.MacOS, ::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::typeof(Pkg.API.develop), ::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) at /Users/dilum/dev/julia-releases/julia-1.3.0/usr/share/julia/stdlib/v1.3/Pkg/src/API.jl:61
 [15] develop(::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) at /Users/dilum/dev/julia-releases/julia-1.3.0/usr/share/julia/stdlib/v1.3/Pkg/src/API.jl:41
 [16] #develop#16(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::typeof(Pkg.API.develop), ::Array{Pkg.Types.PackageSpec,1}) at /Users/dilum/dev/julia-releases/julia-1.3.0/usr/share/julia/stdlib/v1.3/Pkg/src/API.jl:38
 [17] develop(::Array{Pkg.Types.PackageSpec,1}) at /Users/dilum/dev/julia-releases/julia-1.3.0/usr/share/julia/stdlib/v1.3/Pkg/src/API.jl:38
 [18] #develop#12(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::typeof(Pkg.API.develop), ::Pkg.Types.PackageSpec) at /Users/dilum/dev/julia-releases/julia-1.3.0/usr/share/julia/stdlib/v1.3/Pkg/src/API.jl:35
 [19] develop at /Users/dilum/dev/julia-releases/julia-1.3.0/usr/share/julia/stdlib/v1.3/Pkg/src/API.jl:35 [inlined]
 [20] (::Pkg.Operations.var"#128#130"{String,String,Pkg.Operations.var"#132#133"{Bool,Nothing,Cmd,Cmd,Pkg.Types.Context,Array{String,1},String,Pkg.Types.PackageSpec},String})() at /Users/dilum/dev/julia-releases/julia-1.3.0/usr/share/julia/stdlib/v1.3/Pkg/src/Operations.jl:1290
 [21] with_temp_env(::Pkg.Operations.var"#128#130"{String,String,Pkg.Operations.var"#132#133"{Bool,Nothing,Cmd,Cmd,Pkg.Types.Context,Array{String,1},String,Pkg.Types.PackageSpec},String}, ::String) at /Users/dilum/dev/julia-releases/julia-1.3.0/usr/share/julia/stdlib/v1.3/Pkg/src/Operations.jl:1237
 [22] (::Pkg.Operations.var"#127#129"{Pkg.Operations.var"#132#133"{Bool,Nothing,Cmd,Cmd,Pkg.Types.Context,Array{String,1},String,Pkg.Types.PackageSpec},Pkg.Types.Context,Pkg.Types.PackageSpec,String,String,String})(::String) at /Users/dilum/dev/julia-releases/julia-1.3.0/usr/share/julia/stdlib/v1.3/Pkg/src/Operations.jl:1284
 [23] #mktempdir#18(::String, ::typeof(mktempdir), ::Pkg.Operations.var"#127#129"{Pkg.Operations.var"#132#133"{Bool,Nothing,Cmd,Cmd,Pkg.Types.Context,Array{String,1},String,Pkg.Types.PackageSpec},Pkg.Types.Context,Pkg.Types.PackageSpec,String,String,String}, ::String) at ./file.jl:634
 [24] mktempdir(::Function, ::String) at ./file.jl:632 (repeats 2 times)
 [25] sandbox(::Function, ::Pkg.Types.Context, ::Pkg.Types.PackageSpec, ::String, ::String) at /Users/dilum/dev/julia-releases/julia-1.3.0/usr/share/julia/stdlib/v1.3/Pkg/src/Operations.jl:1271
 [26] #test#131(::Bool, ::Nothing, ::Cmd, ::Cmd, ::typeof(Pkg.Operations.test), ::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) at /Users/dilum/dev/julia-releases/julia-1.3.0/usr/share/julia/stdlib/v1.3/Pkg/src/Operations.jl:1355
 [27] #test at ./none:0 [inlined]
 [28] #test#62(::Bool, ::Nothing, ::Cmd, ::Cmd, ::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::typeof(Pkg.API.test), ::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) at /Users/dilum/dev/julia-releases/julia-1.3.0/usr/share/julia/stdlib/v1.3/Pkg/src/API.jl:253
 [29] test(::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) at /Users/dilum/dev/julia-releases/julia-1.3.0/usr/share/julia/stdlib/v1.3/Pkg/src/API.jl:239
 [30] #test#61 at /Users/dilum/dev/julia-releases/julia-1.3.0/usr/share/julia/stdlib/v1.3/Pkg/src/API.jl:233 [inlined]
 [31] test at /Users/dilum/dev/julia-releases/julia-1.3.0/usr/share/julia/stdlib/v1.3/Pkg/src/API.jl:233 [inlined]
 [32] #test#60 at /Users/dilum/dev/julia-releases/julia-1.3.0/usr/share/julia/stdlib/v1.3/Pkg/src/API.jl:232 [inlined]
 [33] test at /Users/dilum/dev/julia-releases/julia-1.3.0/usr/share/julia/stdlib/v1.3/Pkg/src/API.jl:232 [inlined]
 [34] #test#59(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::typeof(Pkg.API.test), ::String) at /Users/dilum/dev/julia-releases/julia-1.3.0/usr/share/julia/stdlib/v1.3/Pkg/src/API.jl:231
 [35] test(::String) at /Users/dilum/dev/julia-releases/julia-1.3.0/usr/share/julia/stdlib/v1.3/Pkg/src/API.jl:231
 [36] top-level scope at REPL[3]:1
caused by [exception 1]
SystemError: opening file "/private/var/folders/48/n09p39y53bx6dnbnf244s2jc0000gp/T/jl_UtizW9/Project.toml": Permission denied
Stacktrace:
 [1] #systemerror#44(::Nothing, ::typeof(systemerror), ::String, ::Bool) at ./error.jl:134
 [2] systemerror at ./error.jl:134 [inlined]
 [3] #open#512(::Nothing, ::Nothing, ::Nothing, ::Bool, ::Nothing, ::typeof(open), ::String) at ./iostream.jl:254
 [4] #open at ./none:0 [inlined]
 [5] #open#271(::Base.Iterators.Pairs{Symbol,Bool,Tuple{Symbol},NamedTuple{(:truncate,),Tuple{Bool}}}, ::typeof(open), ::Pkg.Types.var"#41#43"{Base.GenericIOBuffer{Array{UInt8,1}}}, ::String) at ./io.jl:296
 [6] #open at ./none:0 [inlined]
 [7] write_project at /Users/dilum/dev/julia-releases/julia-1.3.0/usr/share/julia/stdlib/v1.3/Pkg/src/project.jl:176 [inlined]
 [8] #write_project#44(::Bool, ::typeof(Pkg.Types.write_project), ::Pkg.Types.Project, ::Pkg.Types.EnvCache, ::Pkg.Types.EnvCache, ::Pkg.Types.Context) at /Users/dilum/dev/julia-releases/julia-1.3.0/usr/share/julia/stdlib/v1.3/Pkg/src/project.jl:187
 [9] #write_project at ./none:0 [inlined]
 [10] #write_env#141 at /Users/dilum/dev/julia-releases/julia-1.3.0/usr/share/julia/stdlib/v1.3/Pkg/src/Types.jl:1375 [inlined]
 [11] write_env at /Users/dilum/dev/julia-releases/julia-1.3.0/usr/share/julia/stdlib/v1.3/Pkg/src/Types.jl:1373 [inlined]
 [12] #develop#115(::Bool, ::Pkg.BinaryPlatforms.MacOS, ::typeof(Pkg.Operations.develop), ::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}, ::Array{Base.UUID,1}) at /Users/dilum/dev/julia-releases/julia-1.3.0/usr/share/julia/stdlib/v1.3/Pkg/src/Operations.jl:1040
 [13] #develop at /Users/dilum/dev/julia-releases/julia-1.3.0/usr/share/julia/stdlib/v1.3/Pkg/src/API.jl:0 [inlined]
 [14] #develop#17(::Bool, ::Bool, ::Pkg.BinaryPlatforms.MacOS, ::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::typeof(Pkg.API.develop), ::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) at /Users/dilum/dev/julia-releases/julia-1.3.0/usr/share/julia/stdlib/v1.3/Pkg/src/API.jl:61
 [15] (::Pkg.API.var"#kw##develop")(::NamedTuple{(:strict,),Tuple{Bool}}, ::typeof(Pkg.API.develop), ::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) at ./none:0
 [16] #develop#16(::Base.Iterators.Pairs{Symbol,Bool,Tuple{Symbol},NamedTuple{(:strict,),Tuple{Bool}}}, ::typeof(Pkg.API.develop), ::Array{Pkg.Types.PackageSpec,1}) at /Users/dilum/dev/julia-releases/julia-1.3.0/usr/share/julia/stdlib/v1.3/Pkg/src/API.jl:38
 [17] (::Pkg.API.var"#kw##develop")(::NamedTuple{(:strict,),Tuple{Bool}}, ::typeof(Pkg.API.develop), ::Array{Pkg.Types.PackageSpec,1}) at ./none:0
 [18] #develop#12(::Base.Iterators.Pairs{Symbol,Bool,Tuple{Symbol},NamedTuple{(:strict,),Tuple{Bool}}}, ::typeof(Pkg.API.develop), ::Pkg.Types.PackageSpec) at /Users/dilum/dev/julia-releases/julia-1.3.0/usr/share/julia/stdlib/v1.3/Pkg/src/API.jl:35
 [19] #develop at ./none:0 [inlined]
 [20] (::Pkg.Operations.var"#128#130"{String,String,Pkg.Operations.var"#132#133"{Bool,Nothing,Cmd,Cmd,Pkg.Types.Context,Array{String,1},String,Pkg.Types.PackageSpec},String})() at /Users/dilum/dev/julia-releases/julia-1.3.0/usr/share/julia/stdlib/v1.3/Pkg/src/Operations.jl:1286
 [21] with_temp_env(::Pkg.Operations.var"#128#130"{String,String,Pkg.Operations.var"#132#133"{Bool,Nothing,Cmd,Cmd,Pkg.Types.Context,Array{String,1},String,Pkg.Types.PackageSpec},String}, ::String) at /Users/dilum/dev/julia-releases/julia-1.3.0/usr/share/julia/stdlib/v1.3/Pkg/src/Operations.jl:1237
 [22] (::Pkg.Operations.var"#127#129"{Pkg.Operations.var"#132#133"{Bool,Nothing,Cmd,Cmd,Pkg.Types.Context,Array{String,1},String,Pkg.Types.PackageSpec},Pkg.Types.Context,Pkg.Types.PackageSpec,String,String,String})(::String) at /Users/dilum/dev/julia-releases/julia-1.3.0/usr/share/julia/stdlib/v1.3/Pkg/src/Operations.jl:1284
 [23] #mktempdir#18(::String, ::typeof(mktempdir), ::Pkg.Operations.var"#127#129"{Pkg.Operations.var"#132#133"{Bool,Nothing,Cmd,Cmd,Pkg.Types.Context,Array{String,1},String,Pkg.Types.PackageSpec},Pkg.Types.Context,Pkg.Types.PackageSpec,String,String,String}, ::String) at ./file.jl:634
 [24] mktempdir(::Function, ::String) at ./file.jl:632 (repeats 2 times)
 [25] sandbox(::Function, ::Pkg.Types.Context, ::Pkg.Types.PackageSpec, ::String, ::String) at /Users/dilum/dev/julia-releases/julia-1.3.0/usr/share/julia/stdlib/v1.3/Pkg/src/Operations.jl:1271
 [26] #test#131(::Bool, ::Nothing, ::Cmd, ::Cmd, ::typeof(Pkg.Operations.test), ::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) at /Users/dilum/dev/julia-releases/julia-1.3.0/usr/share/julia/stdlib/v1.3/Pkg/src/Operations.jl:1355
 [27] #test at ./none:0 [inlined]
 [28] #test#62(::Bool, ::Nothing, ::Cmd, ::Cmd, ::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::typeof(Pkg.API.test), ::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) at /Users/dilum/dev/julia-releases/julia-1.3.0/usr/share/julia/stdlib/v1.3/Pkg/src/API.jl:253
 [29] test(::Pkg.Types.Context, ::Array{Pkg.Types.PackageSpec,1}) at /Users/dilum/dev/julia-releases/julia-1.3.0/usr/share/julia/stdlib/v1.3/Pkg/src/API.jl:239
 [30] #test#61 at /Users/dilum/dev/julia-releases/julia-1.3.0/usr/share/julia/stdlib/v1.3/Pkg/src/API.jl:233 [inlined]
 [31] test at /Users/dilum/dev/julia-releases/julia-1.3.0/usr/share/julia/stdlib/v1.3/Pkg/src/API.jl:233 [inlined]
 [32] #test#60 at /Users/dilum/dev/julia-releases/julia-1.3.0/usr/share/julia/stdlib/v1.3/Pkg/src/API.jl:232 [inlined]
 [33] test at /Users/dilum/dev/julia-releases/julia-1.3.0/usr/share/julia/stdlib/v1.3/Pkg/src/API.jl:232 [inlined]
 [34] #test#59(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::typeof(Pkg.API.test), ::String) at /Users/dilum/dev/julia-releases/julia-1.3.0/usr/share/julia/stdlib/v1.3/Pkg/src/API.jl:231
 [35] test(::String) at /Users/dilum/dev/julia-releases/julia-1.3.0/usr/share/julia/stdlib/v1.3/Pkg/src/API.jl:231
 [36] top-level scope at REPL[3]:1

bug sandbox

Source

DilumAluthge

All 3 comments

The plot thickens. It seems that if I delete my test/Project.toml file, then the error disappears.

Delete your ~/.julia folder, and then run the following commands:

julia> import Pkg
julia> Pkg.add(Pkg.PackageSpec(name = "PredictMDAPI", rev = "da/delete-test-project-toml"))
julia> Pkg.test("PredictMDAPI") # works just fine

This doesn't really make sense. The contents of my test/Project.toml file are:

[deps]
Test = "8dfed614-e22c-5e08-85e1-65c5234f0b40"

So I think that this is a bug in Pkg.

DilumAluthge on 7 Dec 2019

It is a bug.

The error happens because Pkg uses cp to copy over the test Project.toml to the temporary test directory. So this bug is triggered because (all?) the files in packages are readonly. chmod should take care of this.