Gfx: [mtl] Memory leak - quad example

[unconed]

Short info header:

• GFX version: current master
• OS: macOS 10.14.3 (18D109)
• GPU: AMD Radeon R9 M370X

Running the quad example with the Metal backend shows the following objects leaking per frame, creating a memory pressure of ~1MB/s:

BronzeMtlRenderCmdEncoder 12.50 KiB AMDMTLBronzeDriver -[BronzeMtlCmdBuffer renderCommandEncoderWithDescriptor:]

MTLRenderPassStencilAttachmentDescriptorInternal 176 Bytes Metal +[MTLRenderPassStencilAttachmentDescriptor allocWithZone:]

MTLRenderPassDepthAttachmentDescriptorInternal 176 Bytes Metal +[MTLRenderPassDepthAttachmentDescriptor allocWithZone:]

MTLRenderPassColorAttachmentDescriptorInternal 176 Bytes Metal +[MTLRenderPassColorAttachmentDescriptor allocWithZone:]

MTLRenderPassDescriptorInternal 128 Bytes Metal +[MTLRenderPassDescriptor allocWithZone:]

MTLRenderPassColorAttachmentDescriptorArrayInternal 96 Bytes Metal +[MTLRenderPassColorAttachmentDescriptorArray alloc]

[kvark]

Thanks! Can you point out where those render pass descriptors are getting allocated?
Looking through the code, I see them kept in strong references, so I'd expect them to be cleaned up when going out of scope (like everything else owned, coming from metal-rs).

[unconed]

Example of a stencil descriptor, the others are similar:

   0 libsystem_malloc.dylib malloc_zone_calloc
   1 libsystem_malloc.dylib calloc
   2 libobjc.A.dylib class_createInstance
   3 libobjc.A.dylib +[NSObject allocWithZone:]
   4 Metal +[MTLRenderPassStencilAttachmentDescriptor allocWithZone:]
   5 Metal -[MTLRenderPassDescriptorInternal stencilAttachment]
   6 AMDMTLBronzeDriver amdMtlBronzeHandleLoadActions(BronzeDeviceMembersRec*, BronzeSurfSyncMgrRec*, BronzeCmdBufInfoRec*, AMDBitVectorRec*, MTLRenderPassDescriptor const*, BronzeLoadStoreInfo*, ATI_BOOL)
   7 AMDMTLBronzeDriver -[BronzeMtlRenderCmdEncoder initWithCommandBuffer:descriptor:]
   8 AMDMTLBronzeDriver -[BronzeMtlCmdBuffer renderCommandEncoderWithDescriptor:]
   9 quad _$LT$$LP$A$C$$RP$$u20$as$u20$objc..message..MessageArguments$GT$::invoke::h0bb243a869d081db .cargo/registry/src/github.com-1ecc6299db9ec823/objc-0.2.5/src/message/mod.rs:128
  10 quad objc::message::platform::send_unverified::he0232bdecfbbbed3 .cargo/registry/src/github.com-1ecc6299db9ec823/objc-0.2.5/src/message/apple/mod.rs:27
  11 quad objc::message::send_message::h7a64059573e148ed .cargo/registry/src/github.com-1ecc6299db9ec823/metal-0.14.0/src/commandbuffer.rs:178
  12 quad metal::commandbuffer::CommandBufferRef::new_render_command_encoder::h62a5131ea0db9e09 .cargo/registry/src/github.com-1ecc6299db9ec823/metal-0.14.0/src/commandbuffer.rs:114
  13 quad gfx_backend_metal::command::CommandSink::switch_render::h02be97d627fc9c2a gfx/src/backend/metal/src/command.rs:994
  14 quad _$LT$gfx_backend_metal..command..CommandBuffer$u20$as$u20$gfx_hal..command..raw..RawCommandBuffer$LT$gfx_backend_metal..Backend$GT$$GT$::next_subpass::h3aec03514040d4fc gfx/src/backend/metal/src/command.rs:3289
  15 quad _$LT$gfx_backend_metal..command..CommandBuffer$u20$as$u20$gfx_hal..command..raw..RawCommandBuffer$LT$gfx_backend_metal..Backend$GT$$GT$::begin_render_pass::h05bc6be6986f6711 gfx/src/backend/metal/src/command.rs:3254
  16 quad _$LT$gfx_hal..command..render_pass..RenderPassInlineEncoder$LT$$u27$a$C$$u20$B$GT$$GT$::new::hdc83989535af8984 gfx/src/hal/src/command/render_pass.rs:248
  17 quad gfx_hal::command::graphics::_$LT$impl$u20$gfx_hal..command..CommandBuffer$LT$B$C$$u20$C$C$$u20$S$GT$$GT$::begin_render_pass_inline::ha106a8589ba703e0 gfx/src/hal/src/command/graphics.rs:336
  18 quad quad::main::hc24602afc8afad43 gfx/examples/quad/main.rs:688

The CmdEncoder is the more serious leak however:

   0 libsystem_malloc.dylib malloc_zone_calloc
   1 libsystem_malloc.dylib calloc
   2 libobjc.A.dylib class_createInstance
   3 libobjc.A.dylib +[NSObject allocWithZone:]
   4 AMDMTLBronzeDriver -[BronzeMtlCmdBuffer renderCommandEncoderWithDescriptor:]
   5 quad _$LT$$LP$A$C$$RP$$u20$as$u20$objc..message..MessageArguments$GT$::invoke::h0bb243a869d081db .cargo/registry/src/github.com-1ecc6299db9ec823/objc-0.2.5/src/message/mod.rs:128
   6 quad objc::message::platform::send_unverified::he0232bdecfbbbed3 .cargo/registry/src/github.com-1ecc6299db9ec823/objc-0.2.5/src/message/apple/mod.rs:27
   7 quad objc::message::send_message::h7a64059573e148ed .cargo/registry/src/github.com-1ecc6299db9ec823/metal-0.14.0/src/commandbuffer.rs:178
   8 quad metal::commandbuffer::CommandBufferRef::new_render_command_encoder::h62a5131ea0db9e09 .cargo/registry/src/github.com-1ecc6299db9ec823/metal-0.14.0/src/commandbuffer.rs:114
   9 quad gfx_backend_metal::command::CommandSink::switch_render::h02be97d627fc9c2a gfx/src/backend/metal/src/command.rs:994
  10 quad _$LT$gfx_backend_metal..command..CommandBuffer$u20$as$u20$gfx_hal..command..raw..RawCommandBuffer$LT$gfx_backend_metal..Backend$GT$$GT$::next_subpass::h3aec03514040d4fc gfx/src/backend/metal/src/command.rs:3289
  11 quad _$LT$gfx_backend_metal..command..CommandBuffer$u20$as$u20$gfx_hal..command..raw..RawCommandBuffer$LT$gfx_backend_metal..Backend$GT$$GT$::begin_render_pass::h05bc6be6986f6711 gfx/src/backend/metal/src/command.rs:3254
  12 quad _$LT$gfx_hal..command..render_pass..RenderPassInlineEncoder$LT$$u27$a$C$$u20$B$GT$$GT$::new::hdc83989535af8984 gfx/src/hal/src/command/render_pass.rs:248
  13 quad gfx_hal::command::graphics::_$LT$impl$u20$gfx_hal..command..CommandBuffer$LT$B$C$$u20$C$C$$u20$S$GT$$GT$::begin_render_pass_inline::ha106a8589ba703e0 gfx/src/hal/src/command/graphics.rs:336
  14 quad quad::main::hc24602afc8afad43 gfx/examples/quad/main.rs:688
  15 quad std::rt::lang_start::_$u7b$$u7b$closure$u7d$$u7d$::hbf60ed081b7e6cb6 /rustc/9fda7c2237db910e41d6a712e9a2139b352e558b/src/libstd/rt.rs:74
  16 quad std::rt::lang_start_internal::_$u7b$$u7b$closure$u7d$$u7d$::h258fcf3a64cb870d /rustc/9fda7c2237db910e41d6a712e9a2139b352e558b/src/libstd/panicking.rs:59
  17 quad std::panicking::try::do_call::hb3270f6ae846849d /rustc/9fda7c2237db910e41d6a712e9a2139b352e558b/src/libstd/panicking.rs:310
  18 quad __rust_maybe_catch_panic /rustc/9fda7c2237db910e41d6a712e9a2139b352e558b/src/libpanic_unwind/lib.rs:102
  19 quad std::panicking::try::h0e891ee308bc7843 /rustc/9fda7c2237db910e41d6a712e9a2139b352e558b/src/libstd/rt.rs:289
  20 quad std::panic::catch_unwind::hea29067efe6f5e20 /rustc/9fda7c2237db910e41d6a712e9a2139b352e558b/src/libstd/rt.rs:398
  21 quad std::rt::lang_start_internal::ha07e743882e341f3 /rustc/9fda7c2237db910e41d6a712e9a2139b352e558b/src/libstd/rt.rs:58
  22 quad std::rt::lang_start::h5b0fb7b0473c0a80 /rustc/9fda7c2237db910e41d6a712e9a2139b352e558b/src/libstd/rt.rs:74
  23 quad main
  24 libdyld.dylib start

[kvark]

Looks like MTLRenderCommandEncoder objects aren't properly retained (for some reason). Trying to figure out if we handle the ownership semantics wrong for them.

[JohnColanduoni]

Wrapping the whole render loop of quad in objc::rc::autoreleasepool appears to eliminate the leak that happens when the process is focused, though it doesn't fix the leak described in https://github.com/gfx-rs/gfx/issues/2659. So it looks like it's Metal using autorelease somewhere internally that is causing these to leak, not the strong references we take ourselves.

[kvark]

@JohnColanduoni thanks for helping with investigation! Is your conclusion that this is a Metal runtime issue then? Would be nice to have at least some workaround.

[JohnColanduoni]

No I think Metal just expects you to allocate an NSAutoreleasePool for each run around the render loop. We can probably handle this by adding more objc::rc::autoreleasepool blocks in the Metal backend, as long as we ensure we take strong references to anything we keep outside of them (which I think we're already doing).

[kvark]

@JohnColanduoni ok, I see. I thought we already wrap all the things. It would be great to understand which parts are missing in the wrap (man, those AP pools are annoying).

[kvark]

So I was looking at this again. MTLRenderCommandEncoder is definitely the number 1 offender at the moment. And the callstacks pointing to it are all within our autoreleasepool closure, so I'm not sure how to apply the advice of wrapping more things here...