Termux-packages: jack is broken, bad digest algorithm default

[joakim-noah]

I just tried to compile some Java source using jack, but it always fails with some variation of this error:

com.android.sched.util.config.ConfigurationError: Property 'jack.library.digest.algo': The value must be
{MD5,SHA-1,SHA-224,SHA-256,SHA-384,SHA-512} (case sensitive) but is 'SHA'

Looking at the source, I think it's related to that setting being initialized to a default value of SHA, and then some other recent change not liking it. I'm not sure why they wouldn't have run into this upstream, if SHA really isn't allowed anymore.

Can you look into rebuilding jack to avoid this error?

[fornwall]

Hm, do you get this problem when running a simple jack Main.java compilation? The jack.library.digest.also property should be set by the jack invocation script.

I'm hitting another problem:

Property 'jack.library.output.dir' (in Options): The value must be {MD5,SHA-1,SHA-224,SHA-256,SHA-384,SHA-512} (case sensitive) but is 'SHA' (required because 'jack.library' (defined in Options) is set to true and 'jack.library.output.container' (defined in Options) is set to 'dir')

As for why these problems happen a guess would be that it's either from the dex build process loosing/messing up some information, or from running on the ART runtime (hitting a difference between a normal JRE and Android).

[joakim-noah]

Yeah, that's the same issue: I get it on any java compilation, including a simple "Hello world" java file. I was really confused when I saw that error message, as it makes no sense that an output directory must be set to a digest algorithm, but after trying a couple more things I realized it's just a botched error message. For example, try jack --help-properties, which is recommended before that error message when you try to compile any source, and it will spit out all properties and fail when it hits jack.library.digest.algo, with my first error message and a stack trace.

My guess is that something about our setup tries to set the algorithm to the default value of SHA and errors out because it's not allowed, but somehow this doesn't hit the jack compiler normally? ART's not the problem, as I've been running it with jack for the last year. Something likely changed in jack with the latest update to 24.0 or maybe your new dex build process is the issue.

I guess you get the jack and jill jar files from google? Maybe you can decompile that particular jack bytecode file and change it from SHA to a supported digest? That should work around the problem for now.

[fornwall]

I guess you get the jack and jill jar files from google? Maybe you can decompile that particular jack bytecode file and change it from SHA to a supported digest? That should work around the problem for now.

Yes, the jack and jar files are taken from the SDK build tools.

Another possibility could be to download them (from https://android.googlesource.com/toolchain/jack/+/ub-jack, the tgz link) and patch the java source code. Then build the jars and dex them.

[joakim-noah]

I looked into the jack.jar file you provide: there's a config file in there called jack-default-lib/jack.properties, but modifying config.jack.library.defaul.algo to MD5 does nothing. I tried manually editing the classes.dex from jack.jar so that it used MD5, but that caused all kinds of other errors.

Can you provide the original un-dexed jack.jar that you get from the SDK? I'll see if I can decompile, fix, and use that.

[fornwall]

The un-dexed jack.jar is in ./build-tools/24.0.1/jack.jar relative to the base installation of the Android SDK (provided that version 24.0.1 of the build tools is installed). A copy of it is at http://fornwall.net/jack.jar.

One can also clone and build jack from source (a newer version) with:

git clone https://android.googlesource.com/toolchain/jack/
cd jack
git checkout ub-jack
ant

This will build the jar file at ./jack/dist/jack.jar.

[joakim-noah]

I don't use the SDK, thanks for the jar file. I tried modifying the two files that reference jack.library.default.algo, jack-default-lib/jack.properties and com/android/jack/library/JackLibraryFactory.class, but I still get the same error about the algo being SHA either way. I suppose it's a step forward that I don't get a bunch of other errors after hand-modifying the .class file now.

Can you modify that default value in the jack source and build it, to see if it makes a difference? I don't have a JDK that I can use.

[joakim-noah]

Hey Fredrik, any chance you can fix this? As I mentioned in #63, I'd like to build Android apps on my tablet, and I have been able to do so for the last year because of your great work, but I cannot build java as long as jack is broken.

[ashish0304]

I've managed to build jack on my ubuntu laptop. It is working in termux. Please guide me signing the apk in termux.

[fornwall]

@joakim-noah Sorry for the delay here, and thanks for your great help with the issue!

This should hopefully be fixed now - run apt update && apt upgrade to install the updated jack package! Some notes:

1. We build from source now (currently the latest commit in the jack-ub branch) - this allows staying up to date and add patching, which has now been done for SHA -> MD5.
2. I removed the --output-dex . part from the jack script wrapper, so one needs to add this to the invocation if the classes.dex output is desired, as in jack --output-dex . Main.java.

Does it work fine now?

[joakim-noah]

Nice work, jack is back!

I do have one small issue: I recently updated my Samsung tablet to Marshmallow and I now get this message when compiling with jack, which wasn't there with Lollipop:

MdfppReflectionUtils isMdfEnforced encountered an exception. Cause: java.lang.UnsatisfiedLinkError: No implementation found for boolean com.samsung.android.security.CCManager.isMdfEnforced() (tried Java_com_samsung_android_security_CCManager_isMdfEnforced and Java_com_samsung_android_security_CCManager_isMdfEnforced__) Message: null

It doesn't seem to affect the dex output in any way, but spits out this error info twice for some reason.

Also, there's a problem with aapt, opened another issue for that, #490.

[fornwall]

@joakim-noah Hm, seems to be something Samsung specific. Could you see if you can find the system library containing a Java_com_samsung_android_security_CCManager_isMdfEnforced or Java_com_samsung_android_security_CCManager_isMdfEnforced__ symbol?

Related to building apps with jack: I've just created an apksigner package, to allow easy signing of apks:

apksigner [-p password] keystore input-apk output-apk

This takes a keystore (creating it if it does not exist) and an input APK to create a signed and zipaligned output APK.

[joakim-noah]

It shows up in /system/lib/libandroid_runtime.so:

$ readelf -sW /system/lib/libandroid_runtime.so|grep Java_com_samsung_android_security_CCManager_isMdfEnforced
  2957: 000b8dfd     4 FUNC    GLOBAL DEFAULT   13 Java_com_samsung_android_security_CCManager_isMdfEnforced

Does that need to be linked or added to the path somehow?

Thanks for the apksigner: I have been hashing and signing my apps manually, a tedious process. I was planning on scripting that, thanks for a full-fledged tool instead. I will try it out and let you know how it goes.

[joakim-noah]

I finally tried apksigner and one issue I ran into is that it requires a keystore, not a private key and cert, like I had been using with openssl to sign my apps. Apparently, you need keytool to create a keystore, but zip-signer had a way to use a private key and cert also. I don't know if you took that out, but it's not clear how to use the current apksigner, unless you create a keystore separately by running keytool on a different computer.

Maybe you can bring back the zip-signer functionality to specify the key and cert instead?

[fornwall]

@joakim-noah Sorry for the delay here, lost track of this issue.

About the apksigner tool using only a keystore and not a private key and certificate as separate files it's for simplicity of usage, which is important as many have problem with the complexity of signing apk:s.

1. Create a new keystore - in the invocation apksigner -p password keystore-file input-apk output-apk, the keystore file will be created with the specified password if it does not exist.
2. If you have an existing private key and certificate, merge them on a computer with keytool once.

With the above said, do you still need the ability to specify private key and cert files? If so I could add options for that to the command-line tool.

[joakim-noah]

Thanks for the update. I tried creating a keystore using the invocation you gave and it worked, but I could not install the resulting apk. I primarily need this tool to hash the files in the apk and create the necessary hashed manifests, MANIFEST.MF and CERT.SF, which it is now doing. I can always sign it myself, by using the last OpenSSL command laid out from my apk-building process in #63.

However, it would make it easier for me and others if the private key/certificate option was also given, as you will need to generate the key/certificate to properly sign the apk anyway.

As for this jack issue, I've only kept it open because of that Samsung warning I'm still getting, which a google search shows others are getting too. Feel free to spin off the apksigner command-line option to its own issue and close this issue, based on what you think about the jack warning alone.

[Quasic]

Thanks for working on this. I've tried apt update/apt upgrade several times since the signer was introduced and I am including a transcript of apparently the same error. I only get it with this demo library, when using --output-jack but I haven't tried other libraries. Using --output-dex seems to work.

Terminal transcript.txt

[joakim-noah]

Looks like there is still some place where the hash is not configured properly, as I can reproduce @Quasic's issue if I use --output-jack also. I don't normally do that, which is why I hadn't run into this additional bug so far.

[Quasic]

I started making a patch file in a fork, but I'm not sure how to test it on my phone. When I run "bash ./build-packages.sh jack" it gives an error that NDK doesn't point to the right directory, and I don't think it is even defined. I guess I need some dev package? I already have all ndk-* packages installed.

[vishalbiswas]

@Quasic did you try to run bash ./build-package.sh on your phone? It won't work. You can manually download, apply the patch, configure it and compile it from source to test it.

[Quasic]

Oh, I see. That script's for Ubuntu. Thanks for the reply.

[fornwall]

As jack is being deprecated and support for it is going to be dropped by Google, we're dropping support for the jack package in Termux, so I'm closing this issue.

The best available tool for java on Termux is currently the ecj java compiler to compile .java source code to .class files, and then using the dx file to transform from .class to dex format.

[fornwall]

@joakim-noah Are you still interested in seeing private key/certificate support in the apksigner utility, or are you fine with combining them into a java keystore before using apksigner? If you're still interested I can have a look at fixing it (as you say the underlying support is available, it's just not expose by the command-line tool).

[joakim-noah]

Yes, that'd be great if apksigner supported private keys. Right now, I use it for the manifest and then have a script re-sign the apk using openssl and my private key.

[SDRausty]

@joakim-noah How do you, "Right now, I use it for the manifest and then have a script re-sign the apk using openssl and my private key"? Can you post more details please, including generating the private key in Termux?

[joakim-noah]

See my old comment on building an app from scratch, which shows the OpenSSL command I use to sign the app and links to info on generating your own key. I simply have a script that unzips the app generated by apksigner, signs it using my key, then zips it up again.

[joakim-noah]

It turns out that the signature put out by apksigner is fine: the issue is that if you have a self-signed app that is installed and try to replace it with a new version signed by a different certificate, it will silently fail. Uninstalling the app before instaling the new version works around the problem.

I figured this out when experimenting with generating new self-signed certificates and it wouldn't let me install the resulting apps, but my old certificate worked fine. I was poring over the certificate details trying to figure out what was different, when I tried uninstalling the old version and then the new version worked fine. I tried the same with apksigner and it worked too.

@sdrausty, I've written up how I sign my apps in Termux here, including the info on generating the certificate and signing and repacking the apk.

[SDRausty]

@joakim-noah Thanks for the instructions.