Nixpkgs: Blender segfaults on OpenCL render
Describe the bug
Blender segfaults when an OpenCL render is attempted.
To Reproduce
Steps to reproduce the behavior:
blender- Edit > Settings > System > Click on OpenCL, _don't close the settings window_
- Render Properties (right panel) > Select Cycles, change device to GPU Compute
- Render > Render Image (F12)
Expected behavior
It should render.
Additional context
May be related to #97311 and #75868
Notify maintainers
@cillianderoiste @veprbl
Metadata
- system:
"x86_64-linux" - host os:
Linux 5.4.60, NixOS, 20.03.20200904.51d115a (Markhor) - multi-user?:
yes - sandbox:
yes - version:
nix-env (Nix) 2.4pre20200721_ff314f1 - channels(root):
"nixos-20.03.2910.42674051d12, nixpkgs" - nixpkgs:
/nix/var/nix/profiles/per-user/root/channels/nixos
Maintainer information:
# a list of nixpkgs attributes affected by the problem
attribute:
# a list of nixos modules affected by the problem
module:
Blender gdb backtrace:
gdb /nix/store/2ld5kp0x0psrrlnqjn2p1ll4q92kfvm8-blender-2.83.5/bin/.blender-wrapped
Thread 72 ".blender-wrappe" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fffaeefe700 (LWP 16100)]
0x00007fff9422b7cd in llvm::TargetPassConfig::addPass(llvm::Pass*, bool, bool) () from /nix/store/0fdlazsmnl47m199x7zd76xbmm386kf7-rocm-llvm-3.7.0-lib/lib/libLLVM-11git.so
(gdb) bt
#0 0x00007fff9422b7cd in llvm::TargetPassConfig::addPass(llvm::Pass*, bool, bool) () from /nix/store/0fdlazsmnl47m199x7zd76xbmm386kf7-rocm-llvm-3.7.0-lib/lib/libLLVM-11git.so
#1 0x00007fff9422e388 in llvm::TargetPassConfig::addRegAssignmentOptimized() () from /nix/store/0fdlazsmnl47m199x7zd76xbmm386kf7-rocm-llvm-3.7.0-lib/lib/libLLVM-11git.so
#2 0x00007fff9422e725 in llvm::TargetPassConfig::addOptimizedRegAlloc() () from /nix/store/0fdlazsmnl47m199x7zd76xbmm386kf7-rocm-llvm-3.7.0-lib/lib/libLLVM-11git.so
#3 0x00007fff9422ecde in llvm::TargetPassConfig::addMachinePasses() () from /nix/store/0fdlazsmnl47m199x7zd76xbmm386kf7-rocm-llvm-3.7.0-lib/lib/libLLVM-11git.so
#4 0x00007fff93f723f0 in addPassesToGenerateCode(llvm::LLVMTargetMachine&, llvm::legacy::PassManagerBase&, bool, llvm::MachineModuleInfoWrapperPass&) () from /nix/store/0fdlazsmnl47m199x7zd76xbmm386kf7-rocm-llvm-3.7.0-lib/lib/libLLVM-11git.so
#5 0x00007fff93f775f1 in llvm::LLVMTargetMachine::addPassesToEmitFile(llvm::legacy::PassManagerBase&, llvm::raw_pwrite_stream&, llvm::raw_pwrite_stream*, llvm::CodeGenFileType, bool, llvm::MachineModuleInfoWrapperPass*) () from /nix/store/0fdlazsmnl47m199x7zd76xbmm386kf7-rocm-llvm-3.7.0-lib/lib/libLLVM-11git.so
#6 0x00007fff97890797 in (anonymous namespace)::EmitAssemblyHelper::AddEmitPasses(llvm::legacy::PassManager&, clang::BackendAction, llvm::raw_pwrite_stream&, llvm::raw_pwrite_stream*) [clone .constprop.0] () from /nix/store/4b8wzp5pg2w5cj0clq9xwmb44sng1nfk-rocm-comgr-3.7.0/lib/libamd_comgr.so.1
#7 0x00007fff97894bf0 in (anonymous namespace)::EmitAssemblyHelper::EmitAssembly(clang::BackendAction, std::unique_ptr<llvm::raw_pwrite_stream, std::default_delete<llvm::raw_pwrite_stream> >) () from /nix/store/4b8wzp5pg2w5cj0clq9xwmb44sng1nfk-rocm-comgr-3.7.0/lib/libamd_comgr.so.1
#8 0x00007fff978966db in clang::EmitBackendOutput(clang::DiagnosticsEngine&, clang::HeaderSearchOptions const&, clang::CodeGenOptions const&, clang::TargetOptions const&, clang::LangOptions const&, llvm::DataLayout const&, llvm::Module*, clang::BackendAction, std::unique_ptr<llvm::raw_pwrite_stream, std::default_delete<llvm::raw_pwrite_stream> >) () from /nix/store/4b8wzp5pg2w5cj0clq9xwmb44sng1nfk-rocm-comgr-3.7.0/lib/libamd_comgr.so.1
#9 0x00007fff97836aff in clang::CodeGenAction::ExecuteAction() [clone .part.0] () from /nix/store/4b8wzp5pg2w5cj0clq9xwmb44sng1nfk-rocm-comgr-3.7.0/lib/libamd_comgr.so.1
#10 0x00007fff98616181 in clang::FrontendAction::Execute() () from /nix/store/4b8wzp5pg2w5cj0clq9xwmb44sng1nfk-rocm-comgr-3.7.0/lib/libamd_comgr.so.1
#11 0x00007fff985d0623 in clang::CompilerInstance::ExecuteAction(clang::FrontendAction&) () from /nix/store/4b8wzp5pg2w5cj0clq9xwmb44sng1nfk-rocm-comgr-3.7.0/lib/libamd_comgr.so.1
#12 0x00007fff97829c3b in clang::ExecuteCompilerInvocation(clang::CompilerInstance*) () from /nix/store/4b8wzp5pg2w5cj0clq9xwmb44sng1nfk-rocm-comgr-3.7.0/lib/libamd_comgr.so.1
#13 0x00007fff975e9656 in COMGR::InProcessDriver::execute(llvm::ArrayRef<char const*>) () from /nix/store/4b8wzp5pg2w5cj0clq9xwmb44sng1nfk-rocm-comgr-3.7.0/lib/libamd_comgr.so.1
#14 0x00007fff975ef82d in COMGR::AMDGPUCompiler::processFile(char const*, char const*) () from /nix/store/4b8wzp5pg2w5cj0clq9xwmb44sng1nfk-rocm-comgr-3.7.0/lib/libamd_comgr.so.1
#15 0x00007fff975efc84 in COMGR::AMDGPUCompiler::processFiles(amd_comgr_data_kind_s, char const*) () from /nix/store/4b8wzp5pg2w5cj0clq9xwmb44sng1nfk-rocm-comgr-3.7.0/lib/libamd_comgr.so.1
#16 0x00007fff9762c29f in amd_comgr_do_action () from /nix/store/4b8wzp5pg2w5cj0clq9xwmb44sng1nfk-rocm-comgr-3.7.0/lib/libamd_comgr.so.1
#17 0x00007fffa04e94e9 in device::Program::compileAndLinkExecutable(amd_comgr_data_set_s, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, amd::option::Options*, char**, unsigned long*) () from /nix/store/4w67bbiximrfn899mgjibjzy76nc2cb9-rocm-opencl-runtime-3.7.0/lib/libamdocl64.so
#18 0x00007fffa04ea014 in device::Program::linkImplLC(amd::option::Options*) () from /nix/store/4w67bbiximrfn899mgjibjzy76nc2cb9-rocm-opencl-runtime-3.7.0/lib/libamdocl64.so
#19 0x00007fffa04ef6e2 in device::Program::build(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, char const*, amd::option::Options*) () from /nix/store/4w67bbiximrfn899mgjibjzy76nc2cb9-rocm-opencl-runtime-3.7.0/lib/libamdocl64.so
#20 0x00007fffa04a528c in amd::Program::build(std::vector<amd::Device*, std::allocator<amd::Device*> > const&, char const*, void (*)(_cl_program*, void*), void*, bool, bool) () from /nix/store/4w67bbiximrfn899mgjibjzy76nc2cb9-rocm-opencl-runtime-3.7.0/lib/libamdocl64.so
#21 0x00007fffa0496de4 in amd::Device::BlitProgram::create(amd::Device*, char const*, char const*) () from /nix/store/4w67bbiximrfn899mgjibjzy76nc2cb9-rocm-opencl-runtime-3.7.0/lib/libamdocl64.so
#22 0x00007fffa04c8650 in roc::Device::createBlitProgram() () from /nix/store/4w67bbiximrfn899mgjibjzy76nc2cb9-rocm-opencl-runtime-3.7.0/lib/libamdocl64.so
#23 0x00007fffa05052e8 in roc::KernelBlitManager::createProgram(roc::Device&) () from /nix/store/4w67bbiximrfn899mgjibjzy76nc2cb9-rocm-opencl-runtime-3.7.0/lib/libamdocl64.so
#24 0x00007fffa04d778d in roc::VirtualGPU::create() () from /nix/store/4w67bbiximrfn899mgjibjzy76nc2cb9-rocm-opencl-runtime-3.7.0/lib/libamdocl64.so
#25 0x00007fffa04c7b70 in roc::Device::createVirtualDevice(amd::CommandQueue*) () from /nix/store/4w67bbiximrfn899mgjibjzy76nc2cb9-rocm-opencl-runtime-3.7.0/lib/libamdocl64.so
#26 0x00007fffa04a8877 in amd::HostQueue::Thread::run(void*) () from /nix/store/4w67bbiximrfn899mgjibjzy76nc2cb9-rocm-opencl-runtime-3.7.0/lib/libamdocl64.so
#27 0x00007fffa04540f7 in amd::Thread::main() () from /nix/store/4w67bbiximrfn899mgjibjzy76nc2cb9-rocm-opencl-runtime-3.7.0/lib/libamdocl64.so
#28 0x00007fffa04aa253 in amd::Thread::entry(amd::Thread*) () from /nix/store/4w67bbiximrfn899mgjibjzy76nc2cb9-rocm-opencl-runtime-3.7.0/lib/libamdocl64.so
#29 0x00007ffff7cc0ead in start_thread () from /nix/store/2pi6zgkwnr3zdslvlv16nixpzvbyjx1n-glibc-2.31/lib/libpthread.so.0
#30 0x00007ffff10f2d2f in clone () from /nix/store/2pi6zgkwnr3zdslvlv16nixpzvbyjx1n-glibc-2.31/lib/libc.so.6
I'm using a Radeon VII with:
hardware = {
opengl = {
enable = true;
driSupport = true;
driSupport32Bit = true;
extraPackages32 = with pkgs.pkgsi686Linux; [ libva ];
extraPackages = with pkgs.unstable; [
amdvlk
rocm-opencl-icd
rocm-runtime
];
};
pulseaudio.support32Bit = true;
};
clinfo gives me:
Number of platforms 1
Platform Name AMD Accelerated Parallel Processing
Platform Vendor Advanced Micro Devices, Inc.
Platform Version OpenCL 2.0 AMD-APP (3182.0)
Platform Profile FULL_PROFILE
Platform Extensions cl_khr_icd cl_amd_event_callback
Platform Extensions function suffix AMD
Platform Name AMD Accelerated Parallel Processing
Number of devices 1
Device Name gfx906
Device Vendor Advanced Micro Devices, Inc.
Device Vendor ID 0x1002
Device Version OpenCL 2.0
Driver Version 3182.0 (HSA1.1,LC)
Device OpenCL C Version OpenCL C 2.0
Device Type GPU
Device Board Name (AMD) Device 66af
Device Topology (AMD) PCI-E, 2b:00.0
Device Profile FULL_PROFILE
Device Available Yes
Compiler Available Yes
Linker Available Yes
Max compute units 60
SIMD per compute unit (AMD) 4
SIMD width (AMD) 16
SIMD instruction width (AMD) 1
Max clock frequency 1801MHz
Graphics IP (AMD) 9.6
Device Partition (core)
Max number of sub-devices 60
Supported partition types None
Supported affinity domains (n/a)
Max work item dimensions 3
Max work item sizes 1024x1024x1024
Max work group size 256
Preferred work group size (AMD) 256
Max work group size (AMD) 1024
Preferred work group size multiple 64
Wavefront width (AMD) 64
Preferred / native vector sizes
char 4 / 4
short 2 / 2
int 1 / 1
long 1 / 1
half 1 / 1 (cl_khr_fp16)
float 1 / 1
double 1 / 1 (cl_khr_fp64)
Half-precision Floating-point support (cl_khr_fp16)
Denormals No
Infinity and NANs No
Round to nearest No
Round to zero No
Round to infinity No
IEEE754-2008 fused multiply-add No
Support is emulated in software No
Single-precision Floating-point support (core)
Denormals Yes
Infinity and NANs Yes
Round to nearest Yes
Round to zero Yes
Round to infinity Yes
IEEE754-2008 fused multiply-add Yes
Support is emulated in software No
Correctly-rounded divide and sqrt operations Yes
Double-precision Floating-point support (cl_khr_fp64)
Denormals Yes
Infinity and NANs Yes
Round to nearest Yes
Round to zero Yes
Round to infinity Yes
IEEE754-2008 fused multiply-add Yes
Support is emulated in software No
Address bits 64, Little-Endian
Global memory size 17163091968 (15.98GiB)
Global free memory (AMD) 16760832 (15.98GiB)
Global memory channels (AMD) 128
Global memory banks per channel (AMD) 4
Global memory bank width (AMD) 256 bytes
Error Correction support No
Max memory allocation 14588628172 (13.59GiB)
Unified memory for Host and Device No
Shared Virtual Memory (SVM) capabilities (core)
Coarse-grained buffer sharing Yes
Fine-grained buffer sharing Yes
Fine-grained system sharing No
Atomics No
Minimum alignment for any data type 128 bytes
Alignment of base address 1024 bits (128 bytes)
Preferred alignment for atomics
SVM 0 bytes
Global 0 bytes
Local 0 bytes
Max size for global variable 14588628172 (13.59GiB)
Preferred total size of global vars 17163091968 (15.98GiB)
Global Memory cache type Read/Write
Global Memory cache size 16384 (16KiB)
Global Memory cache line size 64 bytes
Image support Yes
Max number of samplers per kernel 26287
Max size for 1D images from buffer 65536 pixels
Max 1D or 2D image array size 2048 images
Base address alignment for 2D image buffers 256 bytes
Pitch alignment for 2D image buffers 256 pixels
Max 2D image size 16384x16384 pixels
Max 3D image size 2048x2048x2048 pixels
Max number of read image args 128
Max number of write image args 8
Max number of read/write image args 64
Max number of pipe args 16
Max active pipe reservations 16
Max pipe packet size 1703726284 (1.587GiB)
Local memory type Local
Local memory size 65536 (64KiB)
Local memory syze per CU (AMD) 65536 (64KiB)
Local memory banks (AMD) 32
Max number of constant args 8
Max constant buffer size 14588628172 (13.59GiB)
Preferred constant buffer size (AMD) 16384 (16KiB)
Max size of kernel argument 1024
Queue properties (on host)
Out-of-order execution No
Profiling Yes
Queue properties (on device)
Out-of-order execution Yes
Profiling Yes
Preferred size 262144 (256KiB)
Max size 8388608 (8MiB)
Max queues on device 1
Max events on device 1024
Prefer user sync for interop Yes
Number of P2P devices (AMD) 0
P2P devices (AMD) <printDeviceInfo:147: get number of CL_DEVICE_P2P_DEVICES_AMD : error -30>
Profiling timer resolution 1ns
Profiling timer offset since Epoch (AMD) 0ns (Wed Dec 31 16:00:00 1969)
Execution capabilities
Run OpenCL kernels Yes
Run native kernels No
Thread trace supported (AMD) No
Number of async queues (AMD) 8
Max real-time compute queues (AMD) 8
Max real-time compute units (AMD) 60
printf() buffer size 4194304 (4MiB)
Built-in kernels (n/a)
Device Extensions cl_khr_fp64 cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_khr_int64_base_atomics cl_khr_int64_extended_atomics cl_khr_3d_image_writes cl_khr_byte_addressable_store cl_khr_fp16 cl_khr_gl_sharing cl_amd_device_attribute_query cl_amd_media_ops cl_amd_media_ops2 cl_khr_image2d_from_buffer cl_khr_subgroups cl_khr_depth_images cl_amd_copy_buffer_p2p cl_amd_assembly_program
NULL platform behavior
clGetPlatformInfo(NULL, CL_PLATFORM_NAME, ...) AMD Accelerated Parallel Processing
clGetDeviceIDs(NULL, CL_DEVICE_TYPE_ALL, ...) Success [AMD]
clCreateContext(NULL, ...) [default] Success [AMD]
clCreateContextFromType(NULL, CL_DEVICE_TYPE_DEFAULT) Success (1)
Platform Name AMD Accelerated Parallel Processing
Device Name gfx906
clCreateContextFromType(NULL, CL_DEVICE_TYPE_CPU) No devices found in platform
clCreateContextFromType(NULL, CL_DEVICE_TYPE_GPU) Success (1)
Platform Name AMD Accelerated Parallel Processing
Device Name gfx906
clCreateContextFromType(NULL, CL_DEVICE_TYPE_ACCELERATOR) No devices found in platform
clCreateContextFromType(NULL, CL_DEVICE_TYPE_CUSTOM) No devices found in platform
clCreateContextFromType(NULL, CL_DEVICE_TYPE_ALL) Success (1)
Platform Name AMD Accelerated Parallel Processing
Device Name gfx906
ICD loader properties
ICD loader Name OpenCL ICD Loader
ICD loader Vendor OCL Icd free software
ICD loader Version 2.2.10
ICD loader Profile OpenCL 2.1
billop
All 15 comments
cc @acowley @danieldk
veprbl
on 8 Sep 2020
This seems like an upstream issue. The question is which upstream? Does the crash also occur with e.g. the Intel OpenCL runtime?
danieldk
on 8 Sep 2020
FWIW, I also see the segfault on an rx580 following the repro instructions. It does seem like others have Blender working with ROCm OpenCL, so the crash may well be due to our packaging.
@danieldk did such a nice job cleaning up our LLVM derivations that there's much less room for funny business now than with previous releases. I see that we have a debugVersion option we could give LLVM which might reveal a bit more information?
acowley
on 8 Sep 2020
I can also reproduce the problem on a RX580.
Here somebody has the same issue, with a plausible explanation:
https://github.com/RadeonOpenCompute/ROCm/issues/992
The conclusion there is that it occurs when two different LLVM versions are used in a process. Which would be the case here, since Blender uses OpenGL.
Currently our Mesa is still on LLVM 9, whereas ROCm uses LLVM 11.
danieldk
on 9 Sep 2020
Yep, this is probably the issue. If I compile the blender derivation with a mesa built against llvmPackages_11, I don't get the segfault.
I am not sure yet what the solution is, since we cannot always update the LLVM version that Mesa uses to that of ROCm. I wonder how this is dealt with with ROCm on other distributions. Do they override the system Mesa?
danieldk
on 9 Sep 2020
I guess I should have turned up that link first given my prior involvement! 馃槣
@danieldk Could we substitute a mesa_rocm at runtime instead? Perhaps via ICD, or LD_LIBRARY_PATH, or even LD_PRELOAD?
acowley
on 9 Sep 2020
There is hardware.opengl.package, which we could use to override system package. I guess if it works, we could add a module along the lines of the amdgpu-pro support, which would use mesa_rocm as the OpenGL package and could even add the ICD definition to extraPackages. But we should probably first test how well building mesa against the LLVM fork works. At least it builds ;).
I might have some time to look at this after work.
danieldk
on 9 Sep 2020
@danieldk If you have a chance, could you elaborate on how you did this in nixos?:
If I compile the blender derivation with a mesa built against llvmPackages_11, I don't get the segfault.
I understand that it's not the permanent fix, but I'm quite curious to try it.
valencik
on 14 Sep 2020
@danieldk If you have a chance, could you elaborate on how you did this in nixos?:
If I compile the blender derivation with a mesa built against llvmPackages_11, I don't get the segfault.
I understand that it's not the permanent fix, but I'm quite curious to try it.
I modified nixpkgs in-place, but you could do something like:
mesa-rocm = mesa.override { llvmPackages = llvmPackages_rocm; };
And then:
blender-rocm = blender.override { mesa = mesa-rocm; };
A more proper solution would be to switch to a llvmPackages_rocm-built Mesa for /run/opengl. I do have a branch for this:
https://github.com/danieldk/nixpkgs/commits/mesa-rocm
or the two commits:
https://github.com/danieldk/nixpkgs/commit/48b866ebb6c27b1897d92ae65ca69b8b83e89009
https://github.com/danieldk/nixpkgs/commit/4ff2ccbee86201e35e24d38765d05bd5639a6644
But I ran into another issue in Blender, and I didn't have time to investigate. The system (including GNOME, etc.) worked fine with Mesa built with llvmPackages_rocm.
danieldk
on 14 Sep 2020
blender-rocm = blender.override { mesa = mesa-rocm; };
Blender doesn't seem to have a mesa input?
billop
on 14 Sep 2020
blender-rocm = blender.override { mesa = mesa-rocm; };Blender doesn't seem to have a mesa input?
Sorry. I misremember. I may have rebuilt my system with
mesa = callPackage ../development/libraries/mesa {
llvmPackages = llvmPackages_rocm;
inherit (darwin.apple_sdk.frameworks) OpenGL;
inherit (darwin.apple_sdk.libs) Xplugin;
};
Which should be equivalent to having an overlay mesa = self.mesa.override { llvmPackages = llvmPackages_rocm; };
But I am a bit nervous about switching Mesa to a vendor-forked (IIRC prerelease) of LLVM.
We should just document this incompatibility for 20.09 and then perhaps land something like the proposed branch/module in master/21.03. But that definitely requires more testing.
danieldk
on 15 Sep 2020
@danieldk i tried your amdgpu-rocm thing but now blender crashes with this error when opening user preferences:
Read prefs: /home/ash/.config/blender/2.83/config/userpref.blend
mesa: CommandLine Error: Option 'h' registered more than once!
LLVM ERROR: inconsistency in registered CommandLine options
zsh: abort (core dumped) blender
is this the error you mentioned you didn't have time to debug?
ashkitten
on 18 Sep 2020
CommandLine Error: Option 'h' registered more than once!
That's the one. I am afraid though that I can't help with any further debugging. I had to replace the RX580 in the machine I was testing on by an NVIDIA card, because there were too many issues with training machine learning models on the RX580 :confused: .
danieldk
on 18 Sep 2020
this seems very similar to https://github.com/JuliaGPU/OpenCL.jl/issues/125 and replies there would seem to indicate that it's still some LLVM incompatibility, but i'm not sure what is incompatible here... is it blender itself?
ashkitten
on 18 Sep 2020
I think the core problem is that both OpenGL and OpenCL (through winding paths) each load their own instances of LLVM into the process, and those instances of LLVM somehow end up sharing some global data, with usually disastrous results.
athas
on 22 Sep 2020
Related issues
matthiasbeyer
路
3Comments
yawnt
路
3Comments
rzetterberg
路
3Comments
chris-martin
路
3Comments
retrry
路
3Comments