Fresco: Use Bitmap#prepareToDraw to upload Bitmaps to GPU in advance

[ChrisCraik]

Description

Hey, I work on UI Toolkit Graphics in Android. Wanted to pass on an optimization you can enable for displaying Bitmaps.

Android displays Bitmaps as OpenGL textures, and the first time a Bitmap is displayed in a frame, it’s uploaded to the GPU. That can take several milliseconds, but it’s necessary to display the image with the GPU.

In Android N, we added behavior to Bitmap#prepareToDraw() to send an async message to RenderThread to pre-upload. RenderThread uploads these to the GPU when it expects to be idle between frames, instead of while drawing the first frame that uses the Bitmap. If you can do this after decoding, you can make it much less likely that frames are dropped due to uploading.

Reproduction

In Systrace, you can see the problem when any Bitmap is displayed by looking for sections labeled ‘Upload \

Solution

Call Bitmap#prepareToDraw as early as possible when you think a Bitmap will be displayed soon. The most aggressive way to do this is any time you decode an image (especially any immutable image).

The drawback is that if you call it on a bitmap before you modify it, or if it doesn’t get drawn, you’ll be wasting the time spent uploading. If you’re reasonably sure the Bitmap in question will be displayed, it’s still a great way to prefetch the upload work on RenderThread.

Additional Information

Behavior added in Android N:
https://android.googlesource.com/platform/frameworks/base/+/4387190d8ec9fe4e953fcfeb093a644b82cf85ed

For reference, Glide implemented this here: https://github.com/bumptech/glide/commit/dce95506f138bd877aaed1183958f5e186a2bf

It's safe to call on older versions, so you don't need a version check. The method existed prior to N, but was a noop for normal Bitmaps.

[kirwan]

Thanks @ChrisCraik for raising this. We're taking a look.

[lambdapioneer]

Hey @ChrisCraik, I've landed the changes in https://github.com/facebook/fresco/commit/b9f6fdbc9b02971c7ef99bd4ceec7a48f8c196c7 and related. They will go into the next release.

I've verified in systrace that the bitmaps now upload asynchronously on Android N:

It's currently behind an experiment flag for now, but I'll make it default to true once verified with our apps. The fellow reader can look at https://github.com/facebook/fresco/commit/337e140f635aeb7fc2792acd043ec13979ad711a for how to enable this already in your application.

[kirwan]

Now released in v1.4.0.

[erikandre]

Hi @ChrisCraik

In our internal testing we found no performance improvements when using prepareToDraw.
You mentioned that the call needed to be done as early as possible, is there any rule of thumb for how much time is needed between the call and when the image is drawn on screen?
I was also wondering if there are any limitations on when this might improve performance.

[ChrisCraik]

This will only improve performance if you can trigger it at least a full frame in advance - the platform attempts to wait until after a frame has drawn, so that the upload doesn't delay a RenderThread draw. Ideally trigger it after your decoding thread finishes any post-processing.

Are you testing it on an N+ device, with a non-HW bitmap? It's a no-op prior to N, and if you're using a Bitmap.Config.HARDWARE bitmap, upload is already done at creation time.

You might try tracing to see when you're requesting, vs when it's passed to a canvas. It seems to be working in the systrace above, when it was first committed.