Flux.jl: Dense on GPU causes LLVM error: Cannot cast between two non-generic address spaces

Created on 18 Oct 2020 · 4Comments · Source: FluxML/Flux.jl

I get the following LLVM error:

   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.4.1
 _/ |\__'_|_|_|\__'_|  |  Ubuntu ⛬  julia/1.4.1+dfsg-1
|__/                   |

julia> using Flux, CUDA

julia> m = Dense(10,5) |> gpu
Dense(10, 5)

julia> x = rand(10) |> gpu
10-element CuArray{Float32,1}:
 0.34053323
 0.4008895
 0.00021552182
 0.7614378
 0.36244386
 0.26973456
 0.48306096
 0.5690147
 0.15642975
 0.54924625

julia> m(x)
ERROR: LLVM error: Cannot cast between two non-generic address spaces
Stacktrace:
 [1] handle_error(::Cstring) at /home/jungd/.julia/packages/LLVM/T8ZBA/src/core/context.jl:105
 [2] macro expansion at /home/jungd/.julia/packages/LLVM/T8ZBA/src/util.jl:109 [inlined]
 [3] LLVMTargetMachineEmitToMemoryBuffer(::LLVM.TargetMachine, ::LLVM.Module, ::LLVM.API.LLVMCodeGenFileType, ::Base.RefValue{Cstring}, ::Base.RefValue{Ptr{LLVM.API.LLVMOpaqueMemoryBuffer}}) at /home/jungd/.julia/packages/LLVM/T8ZBA/lib/libLLVM_h.jl:3512
 [4] emit(::LLVM.TargetMachine, ::LLVM.Module, ::LLVM.API.LLVMCodeGenFileType) at /home/jungd/.julia/packages/LLVM/T8ZBA/src/targetmachine.jl:43
 [5] mcgen at /home/jungd/.julia/packages/GPUCompiler/GKp4B/src/mcgen.jl:73 [inlined]
 [6] macro expansion at /home/jungd/.julia/packages/TimerOutputs/dVnaw/src/TimerOutput.jl:206 [inlined]
 [7] macro expansion at /home/jungd/.julia/packages/GPUCompiler/GKp4B/src/driver.jl:254 [inlined]
 [8] macro expansion at /home/jungd/.julia/packages/TimerOutputs/dVnaw/src/TimerOutput.jl:206 [inlined]
 [9] codegen(::Symbol, ::GPUCompiler.CompilerJob{GPUCompiler.PTXCompilerTarget,CUDA.CUDACompilerParams}; libraries::Bool, deferred_codegen::Bool, optimize::Bool, strip::Bool, validate::Bool, only_entry::Bool) at /home/jungd/.julia/packages/GPUCompiler/GKp4B/src/driver.jl:250
 [10] compile(::Symbol, ::GPUCompiler.CompilerJob{GPUCompiler.PTXCompilerTarget,CUDA.CUDACompilerParams}; libraries::Bool, deferred_codegen::Bool, optimize::Bool, strip::Bool, validate::Bool, only_entry::Bool) at /home/jungd/.julia/packages/GPUCompiler/GKp4B/src/driver.jl:39
 [11] compile at /home/jungd/.julia/packages/GPUCompiler/GKp4B/src/driver.jl:35 [inlined]
 [12] _cufunction(::GPUCompiler.FunctionSpec{GPUArrays.var"#broadcast_kernel#14",Tuple{CUDA.CuKernelContext,CuDeviceArray{Float32,1,CUDA.AS.Global},Base.Broadcast.Broadcasted{Nothing,Tuple{Base.OneTo{Int64}},typeof(identity),Tuple{Base.Broadcast.Broadcasted{CUDA.CuArrayStyle{1},Nothing,typeof(+),Tuple{Base.Broadcast.Extruded{CuDeviceArray{Float32,1,CUDA.AS.Global},Tuple{Bool},Tuple{Int64}},Base.Broadcast.Extruded{CuDeviceArray{Float32,1,CUDA.AS.Global},Tuple{Bool},Tuple{Int64}}}}}},Int64}}; kwargs::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}) at /home/jungd/.julia/packages/CUDA/dZvbp/src/compiler/execution.jl:310
 [13] _cufunction at /home/jungd/.julia/packages/CUDA/dZvbp/src/compiler/execution.jl:304 [inlined]
 [14] check_cache(::typeof(CUDA._cufunction), ::GPUCompiler.FunctionSpec{GPUArrays.var"#broadcast_kernel#14",Tuple{CUDA.CuKernelContext,CuDeviceArray{Float32,1,CUDA.AS.Global},Base.Broadcast.Broadcasted{Nothing,Tuple{Base.OneTo{Int64}},typeof(identity),Tuple{Base.Broadcast.Broadcasted{CUDA.CuArrayStyle{1},Nothing,typeof(+),Tuple{Base.Broadcast.Extruded{CuDeviceArray{Float32,1,CUDA.AS.Global},Tuple{Bool},Tuple{Int64}},Base.Broadcast.Extruded{CuDeviceArray{Float32,1,CUDA.AS.Global},Tuple{Bool},Tuple{Int64}}}}}},Int64}}, ::UInt64; kwargs::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}) at /home/jungd/.julia/packages/GPUCompiler/GKp4B/src/cache.jl:24
 [15] broadcast_kernel at /home/jungd/.julia/packages/GPUArrays/uaFZh/src/host/broadcast.jl:60 [inlined]
 [16] cached_compilation(::typeof(CUDA._cufunction), ::GPUCompiler.FunctionSpec{GPUArrays.var"#broadcast_kernel#14",Tuple{CUDA.CuKernelContext,CuDeviceArray{Float32,1,CUDA.AS.Global},Base.Broadcast.Broadcasted{Nothing,Tuple{Base.OneTo{Int64}},typeof(identity),Tuple{Base.Broadcast.Broadcasted{CUDA.CuArrayStyle{1},Nothing,typeof(+),Tuple{Base.Broadcast.Extruded{CuDeviceArray{Float32,1,CUDA.AS.Global},Tuple{Bool},Tuple{Int64}},Base.Broadcast.Extruded{CuDeviceArray{Float32,1,CUDA.AS.Global},Tuple{Bool},Tuple{Int64}}}}}},Int64}}, ::UInt64; kwargs::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}) at /home/jungd/.julia/packages/GPUCompiler/GKp4B/src/cache.jl:0
 [17] cached_compilation at /home/jungd/.julia/packages/GPUCompiler/GKp4B/src/cache.jl:44 [inlined]
 [18] cufunction(::GPUArrays.var"#broadcast_kernel#14", ::Type{Tuple{CUDA.CuKernelContext,CuDeviceArray{Float32,1,CUDA.AS.Global},Base.Broadcast.Broadcasted{Nothing,Tuple{Base.OneTo{Int64}},typeof(identity),Tuple{Base.Broadcast.Broadcasted{CUDA.CuArrayStyle{1},Nothing,typeof(+),Tuple{Base.Broadcast.Extruded{CuDeviceArray{Float32,1,CUDA.AS.Global},Tuple{Bool},Tuple{Int64}},Base.Broadcast.Extruded{CuDeviceArray{Float32,1,CUDA.AS.Global},Tuple{Bool},Tuple{Int64}}}}}},Int64}}; name::Nothing, kwargs::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}) at /home/jungd/.julia/packages/CUDA/dZvbp/src/compiler/execution.jl:298
 [19] cufunction at /home/jungd/.julia/packages/CUDA/dZvbp/src/compiler/execution.jl:293 [inlined]
 [20] #launch_heuristic#838 at /home/jungd/.julia/packages/CUDA/dZvbp/src/gpuarrays.jl:19 [inlined]
 [21] launch_heuristic at /home/jungd/.julia/packages/CUDA/dZvbp/src/gpuarrays.jl:17 [inlined]
 [22] copyto! at /home/jungd/.julia/packages/GPUArrays/uaFZh/src/host/broadcast.jl:66 [inlined]
 [23] copyto!(::CuArray{Float32,1}, ::Base.Broadcast.Broadcasted{CUDA.CuArrayStyle{1},Tuple{Base.OneTo{Int64}},typeof(identity),Tuple{Base.Broadcast.Broadcasted{CUDA.CuArrayStyle{1},Nothing,typeof(+),Tuple{CuArray{Float32,1},CuArray{Float32,1}}}}}) at ./broadcast.jl:864
 [24] copy(::Base.Broadcast.Broadcasted{CUDA.CuArrayStyle{1},Tuple{Base.OneTo{Int64}},typeof(identity),Tuple{Base.Broadcast.Broadcasted{CUDA.CuArrayStyle{1},Nothing,typeof(+),Tuple{CuArray{Float32,1},CuArray{Float32,1}}}}}) at ./broadcast.jl:840
 [25] materialize at ./broadcast.jl:820 [inlined]
 [26] Dense at /home/jungd/.julia/packages/Flux/05b38/src/layers/basic.jl:123 [inlined]
 [27] (::Dense{typeof(identity),CuArray{Float32,2},CuArray{Float32,1}})(::CuArray{Float32,1}) at /home/jungd/.julia/packages/Flux/05b38/src/layers/basic.jl:134
 [28] top-level scope at REPL[4]:1

I just installed Flux & CUDA from the Julia package prompt. The nvidia-smi command reports I have CUDA 11.0 (on Ubuntu focal/20.04 x86 desktop).

jungd@sinope:~$ nvidia-smi 
Sun Oct 18 15:50:03 2020       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 450.80.02    Driver Version: 450.80.02    CUDA Version: 11.0     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  GeForce GPU         Off  | 00000000:01:00.0  On |                  N/A |
| N/A   49C    P8    N/A /  N/A |    753MiB /  1999MiB |      4%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

jungd@sinope:~$ nvcc -V
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2019 NVIDIA Corporation
Built on Sun_Jul_28_19:07:16_PDT_2019
Cuda compilation tools, release 10.1, V10.1.243

pytorch works fine, so I don't think it is a CUDA installation issue:

jungd@sinope:~$ python3
Python 3.8.5 (default, Jul 28 2020, 12:59:40) 
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
>>> dev = torch.device("cuda") if torch.cuda.is_available() else torch.device("cpu")
>>> t1 = torch.randn(1,2)
>>> t1 = t1.to(dev)
>>> print(t1.is_cuda) # True
True

Source

davidljung

Most helpful comment

Maybe this is a documentation issue: I uninstalled Julia and reinstalled Julia 1.5.1 and the error doesn't occur. It says at the top of the Flux docs that Julia 1.0 or later is required, then suggests doing 'add CUDA' if a GPU is present. I was using the Julia shipped with Ubuntu 20.04 (1.4.1). However, looking at the docs for the Julia CUDA package, it indicates that Julia 1.5 is required.

davidljung on 19 Oct 2020

👍3

All 4 comments

Could you provide with the version of the Julia packages installed in the environment?
Like with:

] st

jeremiedb on 19 Oct 2020

(@v1.4) pkg> st
Status `~/.julia/environments/v1.4/Project.toml`
  [052768ef] CUDA v1.3.3
  [587475ba] Flux v0.11.1

davidljung on 19 Oct 2020

👍3

You may have also just gotten unlucky with the versions of Julia and CUDA that you used; there have been a good number of changes around the kinds of LLVM pointer types used to represent GPU memory, and they have slightly different semantics at the LLVM level (which is where this error is coming from). Generally, I would advise to use the latest release of Julia (1.5.3 currently) and CUDA.jl that you can for anything relating to the GPU.