Chakracore: Running ChakraCore on Android?

[gashtio]

I was investigating the viability of porting the current source to Android (and possibly iOS) for a project I'm working on. I tested the performance and usability on desktop and the performance seems really good (on par with our current V8/JavaScriptCore solution), but we need to support Android+consoles as well.

Looking at the roadmap I don't see these platforms on the horizon, but seeing that Edge made it to a preview on Android/iOS maybe there are some plans to make it fully cross-platform and move away from WebKit/Blink.

There are some __ANDROID__ defines scattered here and there in the code, and also an ARM configuration (that I don't know if it's functional) so it's not entirely unsupported.

I wanted to share my ~2 day experience and see if there's anyone else who tried doing this port with any success, or has any pointers what is going wrong or if it's even feasible to do in the current state.

_Preparation:_

• I checked the current issues and found #3270 and #3179 relevant, although there wasn't much information there. PR #2292 suggested that at some point the Android port was working.

_Compilation:_

• First, I tried to compile the Android port on Windows, but that had some unexplainable troubles with the PATH env var and others, so I switched on a Linux machine and the compilation was fairly straightforward.
• I was unable to get the project compile with any other Android NDK other than R15c (too many other errors with previous NDKs).
• There was a redefinition error for int8_t in pal_mstypes.h which was fixed by adding && !defined(__ANDROID__) to the define
• There were some missing includes for the JIT build, so I focused on the NoJIT one, which compiled easily.

tools/android_toolchain.sh <path_to_ndk_r15c>
./build.sh --target=android --arch=arm --no-icu --no-jit --static --debug -j

After that I had libChakraCoreStatic.a.

We're building our Android project with VS on Windows, so I wanted to be sure that there's no NDK incompatibility of some sort, but VS supports NDKs up to R13. R14+ NDKs cause _clang_ to emit warnings so I had to write a small tool that replaced the _clang_ binary with one that transformed the command line args' backslashes to forward slashes and forwarded that to the real _clang_ executable.

With everything built with the same compiler and NDK it was time to run it.

_Running:_

• I tried to run it in a rather complex application that I had, but got runtime errors almost immediately. Initialization went fine, but script execution was broken.
• The ARGUMENTS macro used DECLARE_ARGS_VARARRAY but the stack layout doesn't look like what it's expected to be and the app crashes. I spent a couple of hours in the ARM docs and the disassembly trying to figure out where the var args are supposed to be, without much success. The WIN32 version of the macro couldn't be used, because on gcc/clang va_list is a built-in and cannot be cast, so I simply made a stack of 32 Js::Vars and filled it with va_start/va_arg/va_end for testing (which meant that I can have more than 32 JS args, but I wanted to see where this will get me).
• After the va_list "fix" I got another crash which didn't make any sense and this is where I stopped. The call stack went from Js::InterpreterStackFrame::OP_CallCommon<Js::OpLayoutT_CallI<Js::LayoutSizePolicy<(Js::LayoutSize)0> > >(Js::InterpreterStackFrame * this, const Js::OpLayoutT_CallI<Js::LayoutSizePolicy<SmallLayout> > * playout, Js::RecyclableObject * function, unsigned int flags, const Js::AuxArray<unsigned int> * spreadIndices) into Js::UriHelper::ValidateURIProps and the loop there from 0-127 had a value way out of range and triggered an assert. I tried removing the debug check there, but the crash just moved to UriHelper::Decode, which leads me to believe some jump address is incorrect, maybe the execution relied on the stack satisfying some certain layout.

That's as far as I got, if there is/will be any development or advice how to get the Android (and also iOS) port running, I'd love to hear it.

[jackhorton]

@obastemur may know more about this?

[MikeHolman]

I believe we currently only support ARM builds for Windows, and we only support Linux on x64. The issue you face is most likely to do with calling convention. Our assembly thunks that handle calls use the Windows calling convention, and need to be converted to work with Linux/Android calling convention. I suspect getting this working would be a quite sizable work item.

[obastemur]

You saw __ANDROID__ and __IOS__ stuff because mainly I've been trying to keep cross platform implementation in peace with mobile targets as much as possible.

We don't have an Android CI nor official support hence the build on master / release branch is keep breaking. Couple of things are missing on this repo yet IMHO not that tricky to implement. Yet again, we do not officially support any of those targets hence I'm not able to prioritize any of the fixes or missing implementations.

EDIT: target solution doesn't use same engine among platforms

[gashtio]

@MikeHolman As far as I saw, you're relying on the calling convention for the functions with variable arguments, which should be working with the hack I made (I also tried it on Windows and it worked). It's basically this:

#define DECLARE_ARGS_VARARRAY(va, ...)                              \
    va_list _vl;                                                    \
    va_start(_vl, callInfo);                                        \
    Assert(callInfo.Count < 32);                                    \
    Js::Var _vlBuf[32];                                             \
    for (unsigned int _i = 0; _i < callInfo.Count; ++_i)            \
    {                                                               \
        _vlBuf[_i] = va_arg(_vl, Js::Var);                          \
    }                                                               \
    va_end(_vl);                                                    \
    Js::Var* va = _vlBuf

The PAL functions for ReturnAddress and AddressOfReturnAddress seem to be fine. Is there anything else that requires specific stack layout?

@obastemur It seemed to me that it wouldn't be that tricky, that's why I tried building it in the first place :) If you have any pointers where are the missing parts, or if there's anything very platform-specific (e.g. the calling convention) that might be troublesome I can try again getting it running. It seems to me there are more specifics than I first expected and guessing what is going wrong isn't the most effective.

[obastemur]

@gashtio are you planning to send a PR?

[gashtio]

@obastemur If I get it working, for sure, but I haven't had the time to play around more with that.

I don't know the internals of the VM and if you're relying on calling convention in other places (except unpacking the variable arguments). If you can point out some of those places I can try and port them for Android ARM and send a PR.

If there's nothing more obviously broken that you can think of, then there might be some more subtle difference that I haven't identified. In case of the latter, feel free to close this issue and I may work on this some more when I find the time in the future and someone else hadn't already done it.

At the moment our company has put this experiment on hold since without the ARM support that would require us to support 3 JS VMs instead of 2, while we were hoping we can replace all of them with ChakraCore :).

[obastemur]

@gashtio I was referring to compile time fixes you made. :)

[gashtio]

There's really only the int8_t redefinition fix, and that fixes the NoJIT configuration anyway. I'd rather submit something more complete, considering that this isn't even an officially supported platform :)

[gashtio]

Okay, I took some more time for this and got it working in my tests (Android ARM only).

@obastemur Created PR #4452 for Android support.

Initially I had some new troubles with a SIGBUS in some mutexes, which was fixed by aligning the storage for the mutex. The mutexes were also reworked a bit in the master branch (compared to release/1.8) and that's where I branched from, so that's where I submitted the PR for (not sure if you have a policy to go through the release branch first).

Then, I started debugging the error I had before and it turns out it was because the call to arm_CallFunction in JavaScriptFunction.cpp didn't really work for Android and the code was jumping to incorrect addresses. I didn't really have the time to add proper assembly for Android builds so I just hardcoded calls for functions with up to 8 arguments.
The DECLARE_ARGS_VARARRAY still does some avoidable copies, and I'm not perfectly happy with it, but it gets the job done.