Ml-agents: Failing to get proper visual observation on a Linux build using HDRP

[automata]

Describe the bug
While using the HDRP pipeline, we're able to create and train following the steps from https://github.com/Unity-Technologies/ml-agents/blob/latest_release/docs/Learning-Environment-Create-New.md.

We can even run a Jupyter notebook, send actions and get both vector and visual observations from the running environment inside Unity Editor.

However, when we build for Linux and try to call the binary from the Jupyter notebook, we're able to run actions but resulting visual observation is the right camera shot the first time but just dark images (with some incomplete representations as next results), as you can see in the following screenshot.

We tested the same project on Windows 10 and it works, from inside Unity Editor or a built binary for Windows.

To Reproduce
Steps to reproduce the behavior:

1. Create an Unity project using HDRP
2. Follow the steps on https://github.com/Unity-Technologies/ml-agents/blob/latest_release/docs/Learning-Environment-Create-New.md to create a new learning environment
3. Capture visual observations from the main camera
4. Build a Linux binary
5. Run the Jupyter notebook from https://github.com/Unity-Technologies/ml-agents/blob/latest_release/notebooks/getting-started.ipynb but using the binary you just built
6. Check the visual observations

Console logs / stack traces
We don't see any error or other relevant information on console logs.

Screenshots

Environment:

• OS + version: Ubuntu Linux 18.04
• _ML-Agents version_: 0.15.1
• _TensorFlow version_: 1.14.0
• _Python_: 3.6.9
• _Environment_: RollerAgent (built from docs in a new HDRP project)

[vincentpierre]

Hi,

What are the graphics settings you have on Linux ? (Are you using Vulkan ?)
Are you able to run the environment on Linux without issues ?
Are you running in batchmode or with the no-graphics command line ?
Do you have some Player logs that could help us debug this?

[automata]

Hey!

Yes, I'm using Vulkan, on a GTX 1060 GPU.

Yes, I can run the environment with zero issues on Linux.

I tried to run it just by calling ./hdrp_cam.x86_64 -force-vulkan --mlagents-port 5004 and then running the Python code with UnityEnvironment(file_name=None, side_channels = [engine_configuration_channel], base_port=5004). Also tried running specifying the file_name. Both get me the same issue. Only runs flawlessly when inside Unity Editor ((2019.3.10f1).

The Player logs was a good call! Here you go:

Mono path[0] = '/media/data/tmp/unity/hdrp_cam_Data/Managed'
Mono config path = '/media/data/tmp/unity/hdrp_cam_Data/MonoBleedingEdge/etc'
PlayerConnection initialized from /media/data/tmp/unity/hdrp_cam_Data (debug = 0)
PlayerConnection initialized network socket : 0.0.0.0 55495
Multi-casting "[IP] 192.168.0.10 [Port] 55495 [Flags] 2 [Guid] 1673654108 [EditorId] 0 [Version] 1048832 [Id] LinuxPlayer(192.168.0.10) [Debug] 0 [PackageName] LinuxPlayer [ProjectName] <no name>" to [225.0.0.222:54997]...
Started listening to [0.0.0.0:55495]
Preloaded 'lib_burst_generated.so'
Preloaded 'libgrpc_csharp_ext.x64.so'
PlayerConnection already initialized - listening to [0.0.0.0:55495]
Display 0 'LG ULTRAWIDE 29"': 2560x1080 (primary device).
Desktop is 2560 x 1080 @ 60 Hz
Invalid initial resolution 80 x 80 - forcing to 100 x 100
[Vulkan init] extensions: count=18
[Vulkan init] extensions: name=VK_KHR_device_group_creation, enabled=0
[Vulkan init] extensions: name=VK_KHR_external_fence_capabilities, enabled=0
[Vulkan init] extensions: name=VK_KHR_external_memory_capabilities, enabled=0
[Vulkan init] extensions: name=VK_KHR_external_semaphore_capabilities, enabled=0
[Vulkan init] extensions: name=VK_KHR_get_display_properties2, enabled=0
[Vulkan init] extensions: name=VK_KHR_get_physical_device_properties2, enabled=0
[Vulkan init] extensions: name=VK_KHR_get_surface_capabilities2, enabled=0
[Vulkan init] extensions: name=VK_KHR_surface, enabled=1
[Vulkan init] extensions: name=VK_KHR_surface_protected_capabilities, enabled=0
[Vulkan init] extensions: name=VK_KHR_wayland_surface, enabled=1
[Vulkan init] extensions: name=VK_KHR_xcb_surface, enabled=0
[Vulkan init] extensions: name=VK_KHR_xlib_surface, enabled=1
[Vulkan init] extensions: name=VK_KHR_display, enabled=1
[Vulkan init] extensions: name=VK_EXT_direct_mode_display, enabled=0
[Vulkan init] extensions: name=VK_EXT_acquire_xlib_display, enabled=0
[Vulkan init] extensions: name=VK_EXT_display_surface_counter, enabled=0
[Vulkan init] extensions: name=VK_EXT_debug_report, enabled=1
[Vulkan init] extensions: name=VK_EXT_debug_utils, enabled=0
Vulkan detection: 2
Initialize engine version: 2019.3.10f1 (5968d7f82152)
[Subsystems] Discovering subsystems at path /media/data/tmp/unity/hdrp_cam_Data/UnitySubsystems
Forcing GfxDevice: Vulkan
GfxDevice: creating device client; threaded=1
[Vulkan init] extensions: count=18
[Vulkan init] extensions: name=VK_KHR_device_group_creation, enabled=0
[Vulkan init] extensions: name=VK_KHR_external_fence_capabilities, enabled=0
[Vulkan init] extensions: name=VK_KHR_external_memory_capabilities, enabled=0
[Vulkan init] extensions: name=VK_KHR_external_semaphore_capabilities, enabled=0
[Vulkan init] extensions: name=VK_KHR_get_display_properties2, enabled=0
[Vulkan init] extensions: name=VK_KHR_get_physical_device_properties2, enabled=0
[Vulkan init] extensions: name=VK_KHR_get_surface_capabilities2, enabled=0
[Vulkan init] extensions: name=VK_KHR_surface, enabled=1
[Vulkan init] extensions: name=VK_KHR_surface_protected_capabilities, enabled=0
[Vulkan init] extensions: name=VK_KHR_wayland_surface, enabled=1
[Vulkan init] extensions: name=VK_KHR_xcb_surface, enabled=0
[Vulkan init] extensions: name=VK_KHR_xlib_surface, enabled=1
[Vulkan init] extensions: name=VK_KHR_display, enabled=1
[Vulkan init] extensions: name=VK_EXT_direct_mode_display, enabled=0
[Vulkan init] extensions: name=VK_EXT_acquire_xlib_display, enabled=0
[Vulkan init] extensions: name=VK_EXT_display_surface_counter, enabled=0
[Vulkan init] extensions: name=VK_EXT_debug_report, enabled=1
[Vulkan init] extensions: name=VK_EXT_debug_utils, enabled=0
[Vulkan init] extensions: count=89
[Vulkan init] extensions: name=VK_KHR_8bit_storage, enabled=0
[Vulkan init] extensions: name=VK_KHR_16bit_storage, enabled=0
[Vulkan init] extensions: name=VK_KHR_bind_memory2, enabled=0
[Vulkan init] extensions: name=VK_KHR_create_renderpass2, enabled=0
[Vulkan init] extensions: name=VK_KHR_dedicated_allocation, enabled=1
[Vulkan init] extensions: name=VK_KHR_depth_stencil_resolve, enabled=0
[Vulkan init] extensions: name=VK_KHR_descriptor_update_template, enabled=1
[Vulkan init] extensions: name=VK_KHR_device_group, enabled=0
[Vulkan init] extensions: name=VK_KHR_draw_indirect_count, enabled=0
[Vulkan init] extensions: name=VK_KHR_driver_properties, enabled=0
[Vulkan init] extensions: name=VK_KHR_external_fence, enabled=0
[Vulkan init] extensions: name=VK_KHR_external_fence_fd, enabled=0
[Vulkan init] extensions: name=VK_KHR_external_memory, enabled=0
[Vulkan init] extensions: name=VK_KHR_external_memory_fd, enabled=0
[Vulkan init] extensions: name=VK_KHR_external_semaphore, enabled=0
[Vulkan init] extensions: name=VK_KHR_external_semaphore_fd, enabled=0
[Vulkan init] extensions: name=VK_KHR_get_memory_requirements2, enabled=1
[Vulkan init] extensions: name=VK_KHR_image_format_list, enabled=1
[Vulkan init] extensions: name=VK_KHR_imageless_framebuffer, enabled=0
[Vulkan init] extensions: name=VK_KHR_maintenance1, enabled=1
[Vulkan init] extensions: name=VK_KHR_maintenance2, enabled=0
[Vulkan init] extensions: name=VK_KHR_maintenance3, enabled=0
[Vulkan init] extensions: name=VK_KHR_multiview, enabled=1
[Vulkan init] extensions: name=VK_KHR_pipeline_executable_properties, enabled=0
[Vulkan init] extensions: name=VK_KHR_push_descriptor, enabled=0
[Vulkan init] extensions: name=VK_KHR_relaxed_block_layout, enabled=0
[Vulkan init] extensions: name=VK_KHR_sampler_mirror_clamp_to_edge, enabled=1
[Vulkan init] extensions: name=VK_KHR_sampler_ycbcr_conversion, enabled=0
[Vulkan init] extensions: name=VK_KHR_shader_atomic_int64, enabled=0
[Vulkan init] extensions: name=VK_KHR_shader_draw_parameters, enabled=0
[Vulkan init] extensions: name=VK_KHR_shader_float16_int8, enabled=0
[Vulkan init] extensions: name=VK_KHR_shader_float_controls, enabled=0
[Vulkan init] extensions: name=VK_KHR_storage_buffer_storage_class, enabled=0
[Vulkan init] extensions: name=VK_KHR_swapchain, enabled=1
[Vulkan init] extensions: name=VK_KHR_swapchain_mutable_format, enabled=0
[Vulkan init] extensions: name=VK_KHR_timeline_semaphore, enabled=0
[Vulkan init] extensions: name=VK_KHR_uniform_buffer_standard_layout, enabled=0
[Vulkan init] extensions: name=VK_KHR_variable_pointers, enabled=0
[Vulkan init] extensions: name=VK_KHR_vulkan_memory_model, enabled=0
[Vulkan init] extensions: name=VK_EXT_blend_operation_advanced, enabled=0
[Vulkan init] extensions: name=VK_EXT_buffer_device_address, enabled=0
[Vulkan init] extensions: name=VK_EXT_calibrated_timestamps, enabled=0
[Vulkan init] extensions: name=VK_EXT_conditional_rendering, enabled=0
[Vulkan init] extensions: name=VK_EXT_conservative_rasterization, enabled=0
[Vulkan init] extensions: name=VK_EXT_depth_clip_enable, enabled=0
[Vulkan init] extensions: name=VK_EXT_depth_range_unrestricted, enabled=0
[Vulkan init] extensions: name=VK_EXT_descriptor_indexing, enabled=0
[Vulkan init] extensions: name=VK_EXT_discard_rectangles, enabled=0
[Vulkan init] extensions: name=VK_EXT_display_control, enabled=0
[Vulkan init] extensions: name=VK_EXT_fragment_shader_interlock, enabled=0
[Vulkan init] extensions: name=VK_EXT_global_priority, enabled=0
[Vulkan init] extensions: name=VK_EXT_host_query_reset, enabled=0
[Vulkan init] extensions: name=VK_EXT_index_type_uint8, enabled=0
[Vulkan init] extensions: name=VK_EXT_inline_uniform_block, enabled=0
[Vulkan init] extensions: name=VK_EXT_line_rasterization, enabled=0
[Vulkan init] extensions: name=VK_EXT_memory_budget, enabled=0
[Vulkan init] extensions: name=VK_EXT_pci_bus_info, enabled=0
[Vulkan init] extensions: name=VK_EXT_pipeline_creation_feedback, enabled=0
[Vulkan init] extensions: name=VK_EXT_post_depth_coverage, enabled=0
[Vulkan init] extensions: name=VK_EXT_sample_locations, enabled=0
[Vulkan init] extensions: name=VK_EXT_sampler_filter_minmax, enabled=0
[Vulkan init] extensions: name=VK_EXT_scalar_block_layout, enabled=0
[Vulkan init] extensions: name=VK_EXT_separate_stencil_usage, enabled=0
[Vulkan init] extensions: name=VK_EXT_shader_demote_to_helper_invocation, enabled=0
[Vulkan init] extensions: name=VK_EXT_shader_subgroup_ballot, enabled=0
[Vulkan init] extensions: name=VK_EXT_shader_subgroup_vote, enabled=0
[Vulkan init] extensions: name=VK_EXT_shader_viewport_index_layer, enabled=0
[Vulkan init] extensions: name=VK_EXT_subgroup_size_control, enabled=0
[Vulkan init] extensions: name=VK_EXT_texel_buffer_alignment, enabled=0
[Vulkan init] extensions: name=VK_EXT_transform_feedback, enabled=0
[Vulkan init] extensions: name=VK_EXT_vertex_attribute_divisor, enabled=0
[Vulkan init] extensions: name=VK_EXT_ycbcr_image_arrays, enabled=0
[Vulkan init] extensions: name=VK_NV_clip_space_w_scaling, enabled=0
[Vulkan init] extensions: name=VK_NV_dedicated_allocation, enabled=0
[Vulkan init] extensions: name=VK_NV_dedicated_allocation_image_aliasing, enabled=0
[Vulkan init] extensions: name=VK_NV_device_diagnostic_checkpoints, enabled=0
[Vulkan init] extensions: name=VK_NV_fill_rectangle, enabled=0
[Vulkan init] extensions: name=VK_NV_fragment_coverage_to_color, enabled=0
[Vulkan init] extensions: name=VK_NV_framebuffer_mixed_samples, enabled=0
[Vulkan init] extensions: name=VK_NV_coverage_reduction_mode, enabled=0
[Vulkan init] extensions: name=VK_NV_geometry_shader_passthrough, enabled=0
[Vulkan init] extensions: name=VK_NV_sample_mask_override_coverage, enabled=0
[Vulkan init] extensions: name=VK_NV_shader_sm_builtins, enabled=0
[Vulkan init] extensions: name=VK_NV_shader_subgroup_partitioned, enabled=0
[Vulkan init] extensions: name=VK_NV_viewport_array2, enabled=0
[Vulkan init] extensions: name=VK_NV_viewport_swizzle, enabled=0
[Vulkan init] extensions: name=VK_NVX_device_generated_commands, enabled=0
[Vulkan init] extensions: name=VK_NVX_multiview_per_view_attributes, enabled=0
[Vulkan init] extensions: name=VK_NV_ray_tracing, enabled=0
Vulkan API version 1.1.0 (1.1.119 supported by driver)
Vulkan vendor=[NVIDIA] id=[10de]
Vulkan renderer=[GeForce GTX 1060 6GB] id=[1c03]
Vulkan device type 2
Vulkan driverversion=[0x6e148000] uint=[1846837248]
Vulkan PSO: cache data not found [path=/tmp/DefaultCompany/training-dots-hdrp/vulkan_pso_cache.bin]
Begin MonoManager ReloadAssembly
- Completed reload, in  0.067 seconds
WARNING: Shader Unsupported: 'HDRP/Lit' - All passes removed
WARNING: Shader Did you use #pragma only_renderers and omit this platform?
requesting resize 80 x 80
resizing window to 80 x 80
Desktop is 2560 x 1080 @ 60 Hz
InitializeOrResetSwapChain 80x80 hdr=0 samples=1
UnloadTime: 0.455892 ms
Fallback handler could not load library /media/data/tmp/unity/hdrp_cam_Data/Mono/libcoreclr.so
Fallback handler could not load library /media/data/tmp/unity/hdrp_cam_Data/Mono/libcoreclr.so
Fallback handler could not load library /media/data/tmp/unity/hdrp_cam_Data/Mono/libcoreclr.so
Fallback handler could not load library /media/data/tmp/unity/hdrp_cam_Data/Mono/libSystem.dylib
Fallback handler could not load library /media/data/tmp/unity/hdrp_cam_Data/Mono/libSystem.dylib.so
Fallback handler could not load library /media/data/tmp/unity/hdrp_cam_Data/Mono/libSystem.dylib
Fallback handler could not load library /media/data/tmp/unity/hdrp_cam_Data/Mono/libcoreclr.so
Fallback handler could not load library /media/data/tmp/unity/hdrp_cam_Data/Mono/libcoreclr.so
Fallback handler could not load library /media/data/tmp/unity/hdrp_cam_Data/Mono/libcoreclr.so
Fallback handler could not load library /media/data/tmp/unity/hdrp_cam_Data/Mono/libSystem.dylib
Fallback handler could not load library /media/data/tmp/unity/hdrp_cam_Data/Mono/libSystem.dylib.so
Fallback handler could not load library /media/data/tmp/unity/hdrp_cam_Data/Mono/libSystem.dylib
requesting resize 80 x 80
resizing window to 80 x 80
Desktop is 2560 x 1080 @ 60 Hz
InitializeOrResetSwapChain 80x80 hdr=0 samples=1
Setting up 2 worker threads for Enlighten.
  Thread -> id: 7f18935fe700 -> priority: 1 
  Thread -> id: 7f1892dfd700 -> priority: 1 
NullReferenceException: Object reference not set to an instance of an object
  at UnityEngine.Rendering.HighDefinition.HDProbeSystemInternal.QueryCullResults (UnityEngine.Rendering.HighDefinition.HDProbeCullState state, UnityEngine.Rendering.HighDefinition.HDProbeCullingResults& results) [0x00050] in <39762aba73064047936cd8a048e58147>:0 
  at UnityEngine.Rendering.HighDefinition.HDProbeSystem.QueryCullResults (UnityEngine.Rendering.HighDefinition.HDProbeCullState state, UnityEngine.Rendering.HighDefinition.HDProbeCullingResults& results) [0x00000] in <39762aba73064047936cd8a048e58147>:0 
  at UnityEngine.Rendering.HighDefinition.HDRenderPipeline.TryCull (UnityEngine.Camera camera, UnityEngine.Rendering.HighDefinition.HDCamera hdCamera, UnityEngine.Rendering.ScriptableRenderContext renderContext, UnityEngine.Rendering.HighDefinition.SkyManager skyManager, UnityEngine.Rendering.ScriptableCullingParameters cullingParams, UnityEngine.Rendering.HighDefinition.HDRenderPipelineAsset hdrp, UnityEngine.Rendering.HighDefinition.HDRenderPipeline+HDCullingResults& cullingResults) [0x0011a] in <39762aba73064047936cd8a048e58147>:0 
  at UnityEngine.Rendering.HighDefinition.HDRenderPipeline.Render (UnityEngine.Rendering.ScriptableRenderContext renderContext, UnityEngine.Camera[] cameras) [0x002a8] in <39762aba73064047936cd8a048e58147>:0 
  at UnityEngine.Rendering.RenderPipeline.InternalRender (UnityEngine.Rendering.ScriptableRenderContext context, UnityEngine.Camera[] cameras) [0x0001c] in <2e20a3f09314483f9ecf28d1adc29392>:0 
  at UnityEngine.Rendering.RenderPipelineManager.DoRenderLoop_Internal (UnityEngine.Rendering.RenderPipelineAsset pipe, System.IntPtr loopPtr) [0x0003d] in <2e20a3f09314483f9ecf28d1adc29392>:0 
UnityEngine.Camera:Render()
MLAgents.Sensors.CameraSensor:ObservationToTexture(Camera, Int32, Int32)
MLAgents.Sensors.CameraSensor:GetCompressedObservation()
MLAgents.GrpcExtensions:GetObservationProto(ISensor, WriteAdapter)
MLAgents.RpcCommunicator:PutObservations(String, AgentInfo, List`1)
MLAgents.Policies.RemotePolicy:RequestDecision(AgentInfo, List`1)
MLAgents.Agent:NotifyAgentDone(DoneReason)
MLAgents.Agent:OnDisable()

(Filename: <39762aba73064047936cd8a048e58147> Line: 0)

Vulkan PSO: Inconsistent pipeline cache header size[0], newDataSize[678036]
##utp:{"type":"MemoryLeaks","version":2,"phase":"Immediate","time":1587677058598,"processId":17033,"allocatedMemory":75156,"memoryLabels":[{"Default":40},{"Permanent":40},{"NewDelete":15487},{"Thread":800},{"Manager":1744},{"GfxDevice":11536},{"Physics":32},{"Serialization":40},{"String":553},{"DynamicArray":30792},{"Sprites":88},{"GI":12644},{"VR":2016},{"Subsystems":-656}]}

Thanks in advance Vincent!

[automata]

So it seems related with occlusion culling?

[automata]

OK I disabled ambient occlusion just in case and all post processing.

Run it one more time and got on logs:

Mono path[0] = '/media/data/tmp/unity/hdrp_cam5_Data/Managed'
Mono config path = '/media/data/tmp/unity/hdrp_cam5_Data/MonoBleedingEdge/etc'
PlayerConnection initialized from /media/data/tmp/unity/hdrp_cam5_Data (debug = 0)
PlayerConnection initialized network socket : 0.0.0.0 55044
Multi-casting "[IP] 192.168.0.10 [Port] 55044 [Flags] 2 [Guid] 3739687994 [EditorId] 4223096421 [Version] 1048832 [Id] LinuxPlayer(192.168.0.10) [Debug] 0 [PackageName] LinuxPlayer [ProjectName] training-dots-hdrp" to [225.0.0.222:54997]...
Started listening to [0.0.0.0:55044]
Preloaded 'lib_burst_generated.so'
Preloaded 'libgrpc_csharp_ext.x64.so'
PlayerConnection already initialized - listening to [0.0.0.0:55044]
Display 0 'LG ULTRAWIDE 29"': 2560x1080 (primary device).
Desktop is 2560 x 1080 @ 60 Hz
Invalid initial resolution 80 x 80 - forcing to 100 x 100
[Vulkan init] extensions: count=18
[Vulkan init] extensions: name=VK_KHR_device_group_creation, enabled=0
[Vulkan init] extensions: name=VK_KHR_external_fence_capabilities, enabled=0
[Vulkan init] extensions: name=VK_KHR_external_memory_capabilities, enabled=0
[Vulkan init] extensions: name=VK_KHR_external_semaphore_capabilities, enabled=0
[Vulkan init] extensions: name=VK_KHR_get_display_properties2, enabled=0
[Vulkan init] extensions: name=VK_KHR_get_physical_device_properties2, enabled=0
[Vulkan init] extensions: name=VK_KHR_get_surface_capabilities2, enabled=0
[Vulkan init] extensions: name=VK_KHR_surface, enabled=1
[Vulkan init] extensions: name=VK_KHR_surface_protected_capabilities, enabled=0
[Vulkan init] extensions: name=VK_KHR_wayland_surface, enabled=1
[Vulkan init] extensions: name=VK_KHR_xcb_surface, enabled=0
[Vulkan init] extensions: name=VK_KHR_xlib_surface, enabled=1
[Vulkan init] extensions: name=VK_KHR_display, enabled=1
[Vulkan init] extensions: name=VK_EXT_direct_mode_display, enabled=0
[Vulkan init] extensions: name=VK_EXT_acquire_xlib_display, enabled=0
[Vulkan init] extensions: name=VK_EXT_display_surface_counter, enabled=0
[Vulkan init] extensions: name=VK_EXT_debug_report, enabled=1
[Vulkan init] extensions: name=VK_EXT_debug_utils, enabled=0
Vulkan detection: 2
Initialize engine version: 2019.3.10f1 (5968d7f82152)
[Subsystems] Discovering subsystems at path /media/data/tmp/unity/hdrp_cam5_Data/UnitySubsystems
GfxDevice: creating device client; threaded=1
[Vulkan init] extensions: count=18
[Vulkan init] extensions: name=VK_KHR_device_group_creation, enabled=0
[Vulkan init] extensions: name=VK_KHR_external_fence_capabilities, enabled=0
[Vulkan init] extensions: name=VK_KHR_external_memory_capabilities, enabled=0
[Vulkan init] extensions: name=VK_KHR_external_semaphore_capabilities, enabled=0
[Vulkan init] extensions: name=VK_KHR_get_display_properties2, enabled=0
[Vulkan init] extensions: name=VK_KHR_get_physical_device_properties2, enabled=0
[Vulkan init] extensions: name=VK_KHR_get_surface_capabilities2, enabled=0
[Vulkan init] extensions: name=VK_KHR_surface, enabled=1
[Vulkan init] extensions: name=VK_KHR_surface_protected_capabilities, enabled=0
[Vulkan init] extensions: name=VK_KHR_wayland_surface, enabled=1
[Vulkan init] extensions: name=VK_KHR_xcb_surface, enabled=0
[Vulkan init] extensions: name=VK_KHR_xlib_surface, enabled=1
[Vulkan init] extensions: name=VK_KHR_display, enabled=1
[Vulkan init] extensions: name=VK_EXT_direct_mode_display, enabled=0
[Vulkan init] extensions: name=VK_EXT_acquire_xlib_display, enabled=0
[Vulkan init] extensions: name=VK_EXT_display_surface_counter, enabled=0
[Vulkan init] extensions: name=VK_EXT_debug_report, enabled=1
[Vulkan init] extensions: name=VK_EXT_debug_utils, enabled=0
[Vulkan init] extensions: count=89
[Vulkan init] extensions: name=VK_KHR_8bit_storage, enabled=0
[Vulkan init] extensions: name=VK_KHR_16bit_storage, enabled=0
[Vulkan init] extensions: name=VK_KHR_bind_memory2, enabled=0
[Vulkan init] extensions: name=VK_KHR_create_renderpass2, enabled=0
[Vulkan init] extensions: name=VK_KHR_dedicated_allocation, enabled=1
[Vulkan init] extensions: name=VK_KHR_depth_stencil_resolve, enabled=0
[Vulkan init] extensions: name=VK_KHR_descriptor_update_template, enabled=1
[Vulkan init] extensions: name=VK_KHR_device_group, enabled=0
[Vulkan init] extensions: name=VK_KHR_draw_indirect_count, enabled=0
[Vulkan init] extensions: name=VK_KHR_driver_properties, enabled=0
[Vulkan init] extensions: name=VK_KHR_external_fence, enabled=0
[Vulkan init] extensions: name=VK_KHR_external_fence_fd, enabled=0
[Vulkan init] extensions: name=VK_KHR_external_memory, enabled=0
[Vulkan init] extensions: name=VK_KHR_external_memory_fd, enabled=0
[Vulkan init] extensions: name=VK_KHR_external_semaphore, enabled=0
[Vulkan init] extensions: name=VK_KHR_external_semaphore_fd, enabled=0
[Vulkan init] extensions: name=VK_KHR_get_memory_requirements2, enabled=1
[Vulkan init] extensions: name=VK_KHR_image_format_list, enabled=1
[Vulkan init] extensions: name=VK_KHR_imageless_framebuffer, enabled=0
[Vulkan init] extensions: name=VK_KHR_maintenance1, enabled=1
[Vulkan init] extensions: name=VK_KHR_maintenance2, enabled=0
[Vulkan init] extensions: name=VK_KHR_maintenance3, enabled=0
[Vulkan init] extensions: name=VK_KHR_multiview, enabled=1
[Vulkan init] extensions: name=VK_KHR_pipeline_executable_properties, enabled=0
[Vulkan init] extensions: name=VK_KHR_push_descriptor, enabled=0
[Vulkan init] extensions: name=VK_KHR_relaxed_block_layout, enabled=0
[Vulkan init] extensions: name=VK_KHR_sampler_mirror_clamp_to_edge, enabled=1
[Vulkan init] extensions: name=VK_KHR_sampler_ycbcr_conversion, enabled=0
[Vulkan init] extensions: name=VK_KHR_shader_atomic_int64, enabled=0
[Vulkan init] extensions: name=VK_KHR_shader_draw_parameters, enabled=0
[Vulkan init] extensions: name=VK_KHR_shader_float16_int8, enabled=0
[Vulkan init] extensions: name=VK_KHR_shader_float_controls, enabled=0
[Vulkan init] extensions: name=VK_KHR_storage_buffer_storage_class, enabled=0
[Vulkan init] extensions: name=VK_KHR_swapchain, enabled=1
[Vulkan init] extensions: name=VK_KHR_swapchain_mutable_format, enabled=0
[Vulkan init] extensions: name=VK_KHR_timeline_semaphore, enabled=0
[Vulkan init] extensions: name=VK_KHR_uniform_buffer_standard_layout, enabled=0
[Vulkan init] extensions: name=VK_KHR_variable_pointers, enabled=0
[Vulkan init] extensions: name=VK_KHR_vulkan_memory_model, enabled=0
[Vulkan init] extensions: name=VK_EXT_blend_operation_advanced, enabled=0
[Vulkan init] extensions: name=VK_EXT_buffer_device_address, enabled=0
[Vulkan init] extensions: name=VK_EXT_calibrated_timestamps, enabled=0
[Vulkan init] extensions: name=VK_EXT_conditional_rendering, enabled=0
[Vulkan init] extensions: name=VK_EXT_conservative_rasterization, enabled=0
[Vulkan init] extensions: name=VK_EXT_depth_clip_enable, enabled=0
[Vulkan init] extensions: name=VK_EXT_depth_range_unrestricted, enabled=0
[Vulkan init] extensions: name=VK_EXT_descriptor_indexing, enabled=0
[Vulkan init] extensions: name=VK_EXT_discard_rectangles, enabled=0
[Vulkan init] extensions: name=VK_EXT_display_control, enabled=0
[Vulkan init] extensions: name=VK_EXT_fragment_shader_interlock, enabled=0
[Vulkan init] extensions: name=VK_EXT_global_priority, enabled=0
[Vulkan init] extensions: name=VK_EXT_host_query_reset, enabled=0
[Vulkan init] extensions: name=VK_EXT_index_type_uint8, enabled=0
[Vulkan init] extensions: name=VK_EXT_inline_uniform_block, enabled=0
[Vulkan init] extensions: name=VK_EXT_line_rasterization, enabled=0
[Vulkan init] extensions: name=VK_EXT_memory_budget, enabled=0
[Vulkan init] extensions: name=VK_EXT_pci_bus_info, enabled=0
[Vulkan init] extensions: name=VK_EXT_pipeline_creation_feedback, enabled=0
[Vulkan init] extensions: name=VK_EXT_post_depth_coverage, enabled=0
[Vulkan init] extensions: name=VK_EXT_sample_locations, enabled=0
[Vulkan init] extensions: name=VK_EXT_sampler_filter_minmax, enabled=0
[Vulkan init] extensions: name=VK_EXT_scalar_block_layout, enabled=0
[Vulkan init] extensions: name=VK_EXT_separate_stencil_usage, enabled=0
[Vulkan init] extensions: name=VK_EXT_shader_demote_to_helper_invocation, enabled=0
[Vulkan init] extensions: name=VK_EXT_shader_subgroup_ballot, enabled=0
[Vulkan init] extensions: name=VK_EXT_shader_subgroup_vote, enabled=0
[Vulkan init] extensions: name=VK_EXT_shader_viewport_index_layer, enabled=0
[Vulkan init] extensions: name=VK_EXT_subgroup_size_control, enabled=0
[Vulkan init] extensions: name=VK_EXT_texel_buffer_alignment, enabled=0
[Vulkan init] extensions: name=VK_EXT_transform_feedback, enabled=0
[Vulkan init] extensions: name=VK_EXT_vertex_attribute_divisor, enabled=0
[Vulkan init] extensions: name=VK_EXT_ycbcr_image_arrays, enabled=0
[Vulkan init] extensions: name=VK_NV_clip_space_w_scaling, enabled=0
[Vulkan init] extensions: name=VK_NV_dedicated_allocation, enabled=0
[Vulkan init] extensions: name=VK_NV_dedicated_allocation_image_aliasing, enabled=0
[Vulkan init] extensions: name=VK_NV_device_diagnostic_checkpoints, enabled=0
[Vulkan init] extensions: name=VK_NV_fill_rectangle, enabled=0
[Vulkan init] extensions: name=VK_NV_fragment_coverage_to_color, enabled=0
[Vulkan init] extensions: name=VK_NV_framebuffer_mixed_samples, enabled=0
[Vulkan init] extensions: name=VK_NV_coverage_reduction_mode, enabled=0
[Vulkan init] extensions: name=VK_NV_geometry_shader_passthrough, enabled=0
[Vulkan init] extensions: name=VK_NV_sample_mask_override_coverage, enabled=0
[Vulkan init] extensions: name=VK_NV_shader_sm_builtins, enabled=0
[Vulkan init] extensions: name=VK_NV_shader_subgroup_partitioned, enabled=0
[Vulkan init] extensions: name=VK_NV_viewport_array2, enabled=0
[Vulkan init] extensions: name=VK_NV_viewport_swizzle, enabled=0
[Vulkan init] extensions: name=VK_NVX_device_generated_commands, enabled=0
[Vulkan init] extensions: name=VK_NVX_multiview_per_view_attributes, enabled=0
[Vulkan init] extensions: name=VK_NV_ray_tracing, enabled=0
Vulkan API version 1.1.0 (1.1.119 supported by driver)
Vulkan vendor=[NVIDIA] id=[10de]
Vulkan renderer=[GeForce GTX 1060 6GB] id=[1c03]
Vulkan device type 2
Vulkan driverversion=[0x6e148000] uint=[1846837248]
Vulkan PSO: cache data not found [path=/tmp/DefaultCompany/training-dots-hdrp/vulkan_pso_cache.bin]
Begin MonoManager ReloadAssembly
- Completed reload, in  0.719 seconds
WARNING: Shader Unsupported: 'HDRP/Lit' - All passes removed
WARNING: Shader Did you use #pragma only_renderers and omit this platform?
requesting resize 80 x 80
resizing window to 80 x 80
Desktop is 2560 x 1080 @ 60 Hz
InitializeOrResetSwapChain 80x80 hdr=0 samples=1
UnloadTime: 0.484660 ms
Fallback handler could not load library /media/data/tmp/unity/hdrp_cam5_Data/Mono/libcoreclr.so
Fallback handler could not load library /media/data/tmp/unity/hdrp_cam5_Data/Mono/libcoreclr.so
Fallback handler could not load library /media/data/tmp/unity/hdrp_cam5_Data/Mono/libcoreclr.so
Fallback handler could not load library /media/data/tmp/unity/hdrp_cam5_Data/Mono/libSystem.dylib
Fallback handler could not load library /media/data/tmp/unity/hdrp_cam5_Data/Mono/libSystem.dylib.so
Fallback handler could not load library /media/data/tmp/unity/hdrp_cam5_Data/Mono/libSystem.dylib
Fallback handler could not load library /media/data/tmp/unity/hdrp_cam5_Data/Mono/libcoreclr.so
Fallback handler could not load library /media/data/tmp/unity/hdrp_cam5_Data/Mono/libcoreclr.so
Fallback handler could not load library /media/data/tmp/unity/hdrp_cam5_Data/Mono/libcoreclr.so
Fallback handler could not load library /media/data/tmp/unity/hdrp_cam5_Data/Mono/libSystem.dylib
Fallback handler could not load library /media/data/tmp/unity/hdrp_cam5_Data/Mono/libSystem.dylib.so
Fallback handler could not load library /media/data/tmp/unity/hdrp_cam5_Data/Mono/libSystem.dylib
requesting resize 80 x 80
resizing window to 80 x 80
Desktop is 2560 x 1080 @ 60 Hz
InitializeOrResetSwapChain 80x80 hdr=0 samples=1
Compute dispatch: missing UAV ID %d 
(Filename:  Line: 0)

Compute dispatch: missing texture ID %d 
(Filename:  Line: 0)

Compute dispatch: missing texture ID %d 
(Filename:  Line: 0)

Compute dispatch: missing texture ID %d 
(Filename:  Line: 0)

Compute dispatch: missing texture ID %d 
(Filename:  Line: 0)

Compute dispatch: missing texture ID %d 
(Filename:  Line: 0)

Compute dispatch: missing texture ID %d 
(Filename:  Line: 0)

Compute dispatch: missing UAV ID %d 
(Filename:  Line: 0)

Compute dispatch: missing texture ID %d 
(Filename:  Line: 0)

Compute dispatch: missing texture ID %d 
(Filename:  Line: 0)

Compute dispatch: missing texture ID %d 
(Filename:  Line: 0)

Compute dispatch: missing texture ID %d 
(Filename:  Line: 0)

Compute dispatch: missing texture ID %d 
(Filename:  Line: 0)

Compute dispatch: missing UAV ID %d 
(Filename:  Line: 0)

Compute dispatch: missing texture ID %d 
(Filename:  Line: 0)

Compute dispatch: missing texture ID %d 
(Filename:  Line: 0)

Compute dispatch: missing texture ID %d 
(Filename:  Line: 0)

Compute dispatch: missing texture ID %d 
(Filename:  Line: 0)

Compute dispatch: missing UAV ID %d 
(Filename:  Line: 0)

Compute dispatch: missing texture ID %d 
(Filename:  Line: 0)

Compute dispatch: missing texture ID %d 
(Filename:  Line: 0)

Compute dispatch: missing texture ID %d 
(Filename:  Line: 0)

Compute dispatch: missing UAV ID %d 
(Filename:  Line: 0)

Compute dispatch: missing texture ID %d 
(Filename:  Line: 0)

Compute dispatch: missing texture ID %d 
(Filename:  Line: 0)

Compute dispatch: missing texture ID %d 
(Filename:  Line: 0)

Compute dispatch: missing texture ID %d 
(Filename:  Line: 0)

Setting up 2 worker threads for Enlighten.
  Thread -> id: 7fc629979700 -> priority: 1 
  Thread -> id: 7fc629178700 -> priority: 1 
AssertionException: Assertion failure. Value was Null
Expected: Value was not Null
Culling was not prepared, please prepare cull before performing it.
  at UnityEngine.Assertions.Assert.Fail (System.String message, System.String userMessage) [0x0003c] in /home/builduser/buildslave/unity/build/Runtime/Export/Assertions/Assert/AssertBase.cs:29 
  at UnityEngine.Assertions.Assert.IsNotNull[T] (T value, System.String message) [0x00042] in /home/builduser/buildslave/unity/build/Runtime/Export/Assertions/Assert/AssertNull.cs:50 
  at UnityEngine.Rendering.HighDefinition.HDProbeSystemInternal.QueryCullResults (UnityEngine.Rendering.HighDefinition.HDProbeCullState state, UnityEngine.Rendering.HighDefinition.HDProbeCullingResults& results) [0x00001] in /media/data/vilson/src/extend/continuum/render/kb/expt/training-dots-hdrp/Library/PackageCache/[email protected]/Runtime/Lighting/Reflection/HDProbeSystem.cs:263 
  at UnityEngine.Rendering.HighDefinition.HDProbeSystem.QueryCullResults (UnityEngine.Rendering.HighDefinition.HDProbeCullState state, UnityEngine.Rendering.HighDefinition.HDProbeCullingResults& results) [0x00000] in /media/data/vilson/src/extend/continuum/render/kb/expt/training-dots-hdrp/Library/PackageCache/[email protected]/Runtime/Lighting/Reflection/HDProbeSystem.cs:80 
  at UnityEngine.Rendering.HighDefinition.HDRenderPipeline.TryCull (UnityEngine.Camera camera, UnityEngine.Rendering.HighDefinition.HDCamera hdCamera, UnityEngine.Rendering.ScriptableRenderContext renderContext, UnityEngine.Rendering.HighDefinition.SkyManager skyManager, UnityEngine.Rendering.ScriptableCullingParameters cullingParams, UnityEngine.Rendering.HighDefinition.HDRenderPipelineAsset hdrp, UnityEngine.Rendering.HighDefinition.HDRenderPipeline+HDCullingResults& cullingResults) [0x0013c] in /media/data/vilson/src/extend/continuum/render/kb/expt/training-dots-hdrp/Library/PackageCache/[email protected]/Runtime/RenderPipeline/HDRenderPipeline.cs:2683 
  at UnityEngine.Rendering.HighDefinition.HDRenderPipeline.Render (UnityEngine.Rendering.ScriptableRenderContext renderContext, UnityEngine.Camera[] cameras) [0x00315] in /media/data/vilson/src/extend/continuum/render/kb/expt/training-dots-hdrp/Library/PackageCache/[email protected]/Runtime/RenderPipeline/HDRenderPipeline.cs:1311 
  at UnityEngine.Rendering.RenderPipeline.InternalRender (UnityEngine.Rendering.ScriptableRenderContext context, UnityEngine.Camera[] cameras) [0x0001c] in /home/builduser/buildslave/unity/build/Runtime/Export/RenderPipeline/RenderPipeline.cs:34 
  at UnityEngine.Rendering.RenderPipelineManager.DoRenderLoop_Internal (UnityEngine.Rendering.RenderPipelineAsset pipe, System.IntPtr loopPtr) [0x00038] in /home/builduser/buildslave/unity/build/Runtime/Export/RenderPipeline/RenderPipelineManager.cs:86 
UnityEngine.Camera:Render()
MLAgents.Sensors.CameraSensor:ObservationToTexture(Camera, Int32, Int32) (at /media/data/vilson/src/extend/continuum/render/kb/expt/training-dots-hdrp/com.unity.ml-agents/Runtime/Sensors/CameraSensor.cs:145)
MLAgents.Sensors.CameraSensor:GetCompressedObservation() (at /media/data/vilson/src/extend/continuum/render/kb/expt/training-dots-hdrp/com.unity.ml-agents/Runtime/Sensors/CameraSensor.cs:85)
MLAgents.GrpcExtensions:GetObservationProto(ISensor, WriteAdapter) (at /media/data/vilson/src/extend/continuum/render/kb/expt/training-dots-hdrp/com.unity.ml-agents/Runtime/Communicator/GrpcExtensions.cs:232)
MLAgents.RpcCommunicator:PutObservations(String, AgentInfo, List`1) (at /media/data/vilson/src/extend/continuum/render/kb/expt/training-dots-hdrp/com.unity.ml-agents/Runtime/Communicator/RpcCommunicator.cs:287)
MLAgents.Policies.RemotePolicy:RequestDecision(AgentInfo, List`1) (at /media/data/vilson/src/extend/continuum/render/kb/expt/training-dots-hdrp/com.unity.ml-agents/Runtime/Policies/RemotePolicy.cs:33)
MLAgents.Agent:NotifyAgentDone(DoneReason) (at /media/data/vilson/src/extend/continuum/render/kb/expt/training-dots-hdrp/com.unity.ml-agents/Runtime/Agent.cs:324)
MLAgents.Agent:OnDisable() (at /media/data/vilson/src/extend/continuum/render/kb/expt/training-dots-hdrp/com.unity.ml-agents/Runtime/Agent.cs:311)

(Filename: /home/builduser/buildslave/unity/build/Runtime/Export/Assertions/Assert/AssertBase.cs Line: 29)

Vulkan PSO: Inconsistent pipeline cache header size[0], newDataSize[673224]
##utp:{"type":"MemoryLeaks","version":2,"phase":"Immediate","time":1587678800201,"processId":25753,"allocatedMemory":75157,"memoryLabels":[{"Default":40},{"Permanent":40},{"NewDelete":15487},{"Thread":800},{"Manager":1744},{"GfxDevice":11536},{"Physics":32},{"Serialization":40},{"String":554},{"DynamicArray":30792},{"Sprites":88},{"GI":12644},{"VR":2016},{"Subsystems":-656}]}

While searching about that new error Compute dispatch: missing texture ID %d (Filename: Line: 0) I got this https://forum.unity.com/threads/screen-setresolution-causing-materials-to-stop-rendering.804681/. Maybe related?

[vincentpierre]

Maybe this is due to our use Screen.SetResolution in the code. Can you try commenting this line out and try again?

[automata]

Great catch Vincent! I commented that out and now it works perfectly:

The error message related with culling is still there but no more complains about missing textures.

Inspired by your suggestion I tried to change settings to allow resizable window like:

However that didn't work neither.

I put all the post processing back and your solution works.

Thank you!

[automata]

Should I open a PR for that?

[surfnerd]

Hi @automata,
This looks like a graphics bug within unity itself, not ML-Agents. We are working with our graphics team to see if it's already fixed in a newer version, or if we can file a bug for this so they can fix it. Thank you for finding this!

[vincentpierre]

@surfnerd , We could do

# if not PLATFORM_LINUX and not EDITOR and VERSION_2019_3
Screen.SetResolution(width, height)
#endif

to make sure we do not set the resolution on this platform.
We could also not set the screen resolution by default in learn.py

[vincentpierre]

Tracking this bug internally : MLA-961
In the mean time, we will now not set the resolution by default when doing training #3867