Apktool: StringIndexOutOfBoundsException (getHTML)

[eladkarako]

"D:\Software\Java\8\x64\jdk1.8.0_151\bin\java.exe" -d64 -jar "D:/DOS/android/bin/apktool.jar"
--force --output "project_calcalist_5049" decode  "D:/DOS/android/calcalist_5049.apk"

Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF8 -Duser.language=en -Xmn512m -Xms512m -Xmx2048
m -Xverify:none -XX:+UseParallelGC -XX:ParallelGCThreads=2 -Dfile.encoding=UTF8 -Duser.language
=en
I: Using Apktool 2.4.1-b05f19-SNAPSHOT on calcalist_5049.apk
I: Loading resource table...
Exception in thread "main" java.lang.StringIndexOutOfBoundsException: String index out of range
: 24
        at java.lang.String.substring(String.java:1963)
        at brut.androlib.res.decoder.StringBlock.getHTML(StringBlock.java:164)
        at brut.androlib.res.decoder.ARSCDecoder.readValue(ARSCDecoder.java:360)
        at brut.androlib.res.decoder.ARSCDecoder.readEntryData(ARSCDecoder.java:276)
        at brut.androlib.res.decoder.ARSCDecoder.readTableType(ARSCDecoder.java:252)
        at brut.androlib.res.decoder.ARSCDecoder.readTableTypeSpec(ARSCDecoder.java:175)
        at brut.androlib.res.decoder.ARSCDecoder.readTablePackage(ARSCDecoder.java:131)
        at brut.androlib.res.decoder.ARSCDecoder.readTableHeader(ARSCDecoder.java:82)
        at brut.androlib.res.decoder.ARSCDecoder.decode(ARSCDecoder.java:48)
        at brut.androlib.res.AndrolibResources.getResPackagesFromApk(AndrolibResources.java:786)
        at brut.androlib.res.AndrolibResources.loadMainPkg(AndrolibResources.java:67)
        at brut.androlib.res.AndrolibResources.getResTable(AndrolibResources.java:59)
        at brut.androlib.Androlib.getResTable(Androlib.java:66)
        at brut.androlib.ApkDecoder.setTargetSdkVersion(ApkDecoder.java:236)
        at brut.androlib.ApkDecoder.decode(ApkDecoder.java:118)
        at brut.apktool.Main.cmdDecode(Main.java:170)
        at brut.apktool.Main.main(Main.java:76)

I'm trying to process this APK:
https://apkpure.com/%D7%9B%D7%9C%D7%9B%D7%9C%D7%99%D7%A1%D7%98/com.opentech.calcalist/download/160-APK?from=versions%2Fversion

I'm using an APKTool version 2.4.1-197d46-SNAPSHOT from source.
(can download from https://github.com/eladkarako/icompile/tree/master/unofficial-apktool)

[Ticklefish]

This APK is badly constructed unfortunately. That error isn't something apktool can overcome, I think.

I tried decompiling the file without processing "resources.arsc" but got an error saying:

Exception in thread "main" brut.androlib.AndrolibException: brut.directory.DirectoryException: Error copying file: resources.arsc

This might have been done deliberately to stop people from modding the APK.

[eladkarako]

interesting..
I too thought that this is some kind of method to prevent decompiling,
but now I'm not so sure,
I've dumped some of resources successfully with AAPT (for example aapt dump badging calcalist_5049.apk),

and if AAPT can successfully dump the resources, and APKTool uses AAPT in the background,
I think there is a bug in the way APKTool accepts the output of AAPT,
..in this case anyway...

[iBotPeaches]

Some weird strings crashing the decoder. I replicate.

"အသံပြန်ဖွင့်ပါ"

Marking as bug.

[eladkarako]

I've tried make Java "see the light" by using some global command-line options too:
set "JAVA_TOOL_OPTIONS=-Dfile.encoding=UTF8 -Duser.language=en"
that explicit the input files encoding, but without luck.

I.D.K. if you've pushed a fix yet,
but it still happens even in a recent (November 30, 2020) build d63088 from master.

verbose ? [N]
assets ? [Y]
res ? [Y]
smalli source ? [Y]
----------------------------------------------------
[INFO] java:             D:\Software\Java\8\x64\jdk1.8.0_151\bin\java.exe
[INFO] java arc.:        -d64
[INFO] java opts:        -Dfile.encoding=UTF8 -Duser.language=en -Xmn512m -Xms512m -Xmx2048m -Xverify:none -XX:+UseParallelGC -XX:ParallelGCThreads=2 -Dfile.encoding=UTF8 -Duser.language=en

[INFO] global language:  [LC_ALL=en_US.UTF-8] and [LANG=en_US.UTF-8]
[INFO] ApkTool.jar       D:/DOS/android/bin/apktool.jar
[INFO] extract-target:   D:/DOS/android/כלכליסט_v5.0.49_apkpure.com.apk
[INFO] additional arg.s:
----------------------------------------------------

"D:\Software\Java\8\x64\jdk1.8.0_151\bin\java.exe" -d64 -jar "D:/DOS/android/bin/apktool.jar"  --force --output "project_כלכליסט_v5.0.49_apkpure.com" decode  "D:/DOS/android/כלכליסט_v5.0.49_apkpure.com.apk"

Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF8 -Duser.language=en -Xmn512m -Xms512m -Xmx2048m -Xverify:none -XX:+UseParallelGC -XX:ParallelGCThreads=2 -Dfile.encoding=UTF8 -Duser.language=en
I: Using Apktool 2.4.2-d63088-SNAPSHOT on כלכליסט_v5.0.49_apkpure.com.apk
m.apk
I: Loading resource table...
Exception in thread "main" java.lang.StringIndexOutOfBoundsException: String index out of range: 24
        at java.lang.String.substring(String.java:1963)
        at brut.androlib.res.decoder.StringBlock.getHTML(StringBlock.java:160)
        at brut.androlib.res.decoder.ARSCDecoder.readValue(ARSCDecoder.java:360)
        at brut.androlib.res.decoder.ARSCDecoder.readEntryData(ARSCDecoder.java:276)
        at brut.androlib.res.decoder.ARSCDecoder.readTableType(ARSCDecoder.java:252)
        at brut.androlib.res.decoder.ARSCDecoder.readTableTypeSpec(ARSCDecoder.java:175)
        at brut.androlib.res.decoder.ARSCDecoder.readTablePackage(ARSCDecoder.java:131)
        at brut.androlib.res.decoder.ARSCDecoder.readTableHeader(ARSCDecoder.java:82)
        at brut.androlib.res.decoder.ARSCDecoder.decode(ARSCDecoder.java:48)
        at brut.androlib.res.AndrolibResources.getResPackagesFromApk(AndrolibResources.java:790)

        at brut.androlib.res.AndrolibResources.loadMainPkg(AndrolibResources.java:67)
        at brut.androlib.res.AndrolibResources.getResTable(AndrolibResources.java:59)
        at brut.androlib.Androlib.getResTable(Androlib.java:66)
        at brut.androlib.ApkDecoder.setTargetSdkVersion(ApkDecoder.java:236)
        at brut.androlib.ApkDecoder.decode(ApkDecoder.java:118)
        at brut.apktool.Main.cmdDecode(Main.java:179)
        at brut.apktool.Main.main(Main.java:82)

Perhaps modifying the following code to your usage can help with normalizing things for you with native functions?
https://github.com/eladkarako/Overkill-UTF-8-Support-For-Windows-And-Java-With-Full-Code-And-Usage-Example/blob/520ea39f4dda215db6879281b2f95a5ad0276eeb/_compile.cmd#L43

https://github.com/eladkarako/Overkill-UTF-8-Support-For-Windows-And-Java-With-Full-Code-And-Usage-Example/blob/master/input2stdout.java#L15

(along with
https://github.com/eladkarako/Overkill-UTF-8-Support-For-Windows-And-Java-With-Full-Code-And-Usage-Example/blob/master/input2stdout.cmd#L18-L24)

[iBotPeaches]

Perhaps modifying the following code to your usage can help with normalizing things for you with native functions?
https://github.com/eladkarako/Overkill-UTF-8-Support-For-Windows-And-Java-With-Full-Code-And-Usage-Example/blob/520ea39f4dda215db6879281b2f95a5ad0276eeb/_compile.cmd#L43

I don't think the issue is having to do with output, but rather the actual low level decoding of a UTF8/UTF16 string. We are reading the StringBlock with a custom parser to handle the needed changes - https://github.com/iBotPeaches/Apktool/blob/master/brut.apktool/apktool-lib/src/main/java/brut/androlib/res/decoder/StringBlock.java

So this is a lot of if/loops and more. Its buggy and needs to be patched with the idea of multibyte characters, surrogate characters and more. Unicode, et all.

[eladkarako]

hmm, yes, I agree that's quite a low-level Unicode parser alright.

Can't really help with this, I've only used Google's escaper and re2j in the past,
and I've avoided for the life-of-me the pit that is Unicode parsing..