Bevy: Android Support

[cart]

It should be possible to run Bevy Apps on Android

[PrototypeNM1]

I'm working on this issue now, can I get it assigned to me to help head off accidental parallel implementations?

[cart]

absolutely! thanks for picking it up. this will be huge :heart:

[cart]

haha just noticed @karroffel already beat me to it

[endragor]

@PrototypeNM1 could you please share your progress and what's left to be done? I have some time on weekends that I could dedicate to adding Android support to Bevy. Last weekend I added Android support to cpal, which was one of the major blockers. There are still some issues on android-ndk-rs side. I could look into that, but it would be nice if you broke this issue into subitems and clarify what is done and what isn't.

[PrototypeNM1]

To my knowledge there are 4 main blockers for Android support.

1. Lack of Android support for cpal (wip) (thanks for tacking this!)
   Once fixed, may reveal issues in it's transitive dependents Rodio or winit.
2. cargo apk is broken for how Bevy structures Cargo.toml (PR)
   Alt. removing glob imports from Bevy's Cargo.toml fixes this.
3. Mobile needs input support, touch and IMU
4. bevy-glsl-to-spirv requires a rewrite or replacement.

I'm working on 4. glsl-to-spirv as written depends on a bundled glslangValidator executable. This can't work on Android without hackery. Luckily glslangValidator has a recently added a C API.

@endragor since you started investigating cpal/oboe issues, I recommend figuring out why cargo apk had issues with including multiple shared libraries, doing so is helpful both for the Rust Android ecosystem and Bevy.

[MichaelHills]

@PrototypeNM1 FYI over at https://github.com/bevyengine/bevy/issues/87 I put up https://github.com/bevyengine/bevy/pull/324 where currently uses shaderc-rs instead of glsl-to-spirv for iOS. Feel free to give it a try.

I too have now run into the missing touch support... I might hack something in just so I can play with it, not exactly sure what the correct API is.

[enfipy]

Hey all! I was able to run Bevy 3d example on my Android phone.

Proof and repo:

Current issues are next:

1. Shaders still not working in runtime (so I did it in compile-time). As they precompiled - we can't use macros so that shadows are weird on the screenshot.
2. Bgra was not worked with Vulkan on my phone so I replaced them with Rgba. Probably we can just replace them with Rgba everywhere in Bevy but this should be tested on other platforms.
3. I couldn't make assets work at all on Android. It's pretty weird but with the newest version of android-ndk-rs - there are no assets when I trying to get them from asset_manager.
4. Audio, 3d models, textures, etc were not tested due to problems with assets.

This is the current status. Any feedback and advice are highly appreciated.

[cart]

Fantastic! This is a big step :smile:

1. Shaders still not working in runtime (so I did it in compile-time). As they precompiled - we can't use macros so that shadows are weird on the screenshot.

Yeah as we discussed elsewhere I think we'll want to solve this by adding pre-compiled shader support with the various permutations available. We'll want a system like that anyway to improve startup times in release builds. Alternatively we can try adding dynamic compilation support by (1) waiting for Naga or (2) somehow making shaderc compile on android

1. Bgra was not worked with Vulkan on my phone so I replaced them with Rgba. Probably we can just replace them with Rgba everywhere in Bevy but this should be tested on other platforms.

Yeah thats a bit annoying, but it seems like we can work around it by selecting formats according to the platform. Alternatively, it might be worth checking with the wgpu folks to see if its something they can fix on their end.

1. I couldn't make assets work at all on Android. It's pretty weird but with the newest version of android-ndk-rs - there are no assets when I trying to get them from asset_manager.

Hmm maybe we need to add them to some sort of asset manifest? I haven't worked with android for awhile, but i vaguely remember something like that.

[PrototypeNM1]

@enfipy Awesome! Referencing your changes for TextureFormat I got runtime shader generation working on Android.

[enfipy]

After about 10 hours of unsuccessful tries to make @PrototypeNM1 example with bevy-glsl-to-spirv work on my Mac (and Windows) - I added shaderc-rs support for Android (but it's still shitty) and was able to make shaders work in runtime.

Status update:

• Shaders are working on runtime but the build still needs to be enhanced (or I very hope Naga will land soon). (Screenshot_1)
• I added a temporary solution for assets loading on Android. I plan to add a better one to Bevy as @cart will finish asset refactoring.
• PNG images rendered successfully (Screenshot_2).
• Text and UI rendered successfully (Screenshot_1).

Things need to be resolved:

• [x] Audio. This PR looks good and hopefully will enable high-performance audio on Android.
• [x] TextureFormat. It looks like some issues still persist with Bgra/Rgba as I couldn't build a Monkey example and faced a similar issue that was before this (but I don't think it's very serious).
• [ ] Shaderc. As I specified above - it's not finished and tested on other platforms yet and hard to build.
• [ ] Request Permissions. From Android version 23 and higher, Android requires applications to request permissions at runtime. I want to add jni call to this in a couple of days.
• [x] Touch Support. I hope this will be available soon with this #696.

This is all I remembered.

Screenshots:

Screenshot_1:

Screenshot_2:

[cart]

Very nice progress! I just merged the asset system changes. I'm making a few more changes to AssetIo to re-add wasm compatibility. It should also make integrating the android AssetIo backend slightly easier (as I'm boxing AssetIo)

[trezm]

I'd like to just say, I'm not working on this issue, but am wildly excited to see its progress. Huge thanks to everyone working on it and know that the community is ecstatic to see this moving forward!!

[PrototypeNM1]

@enfipy @cart Given Enfipy has more time to work on and organize this issue, I think we should transfer this issue to them.

[cart]

I think this is a big enough issue / problem space that I don't think anyone needs to own it.

[PrototypeNM1]

Added support for runtime spirv generation using a rewrite of glsl-to-spirv.

We're still waiting for a fix in cargo-apk's dependencies to land. The alternative of removing glob imports for bevy's workspaces is still a viable alternative if we wanted this working asap.

[mmacedoeu]

Would like to bring attention to proper implementation of winit events Event::Suspended and Event::Resumed at https://github.com/bevyengine/bevy/blob/master/crates/bevy_winit/src/lib.rs#L170 so it can proper release and reload resources

[francesca64]

Would there be any interest in using cargo-mobile to help with building, generating Android Studio projects, running on device, etc?

[cart]

Medium-to-long term I might be interested in adopting higher level abstractions. Short term I'd rather keep it simple, encourage people to become familiar with the "native" mobile tooling, and maintain control over "official" templates. We will likely create an official bevy_template repo in the near future.

A minor note on the current cargo-mobile bevy template: bevy has its own answer to #[mobile_entry_point] called #[bevy_main]. I'd prefer it if you used that instead (to support future non-mobile scenarios). Although thats only available on the master branch right now.