Sentry-dart: Android Proguard stacktraces are not readable even after uploading the symbol files

[ganeshrvel]

_Platform:_

• [ ] Dart
• [x] Flutter Android or iOS
• [ ] Flutter Web

_IDE:_

• [ ] VSCode
• [x] IntelliJ/AS
• [ ] XCode
• [ ] Other, which one?

_split-debug-info and obfuscate (Flutter Android or iOS) or CanvasKit (Flutter Web):_

• [x] Enabled
• [ ] Disabled

_Platform installed with:_

• [x] pub.dev
• [ ] GitHub

Output of the command flutter doctor -v below:
[✓] Flutter (Channel unknown, 1.22.0, on Mac OS X 10.15.6 19G73, locale en-IN)

[✓] Android toolchain - develop for Android devices (Android SDK version 29.0.3)
[✓] Xcode - develop for iOS and macOS (Xcode 11.6)
[✓] Connected device (2 available)

The output goes here...

The version of the SDK (See pubspec.lock):
4.x.x

---

I have the following issue:

We are unable to map the sentry stacktrace logs to the manually uploaded symbol files. We have the latest version of sentry_flutter package and even though we have manually uploaded the native symbol files of the Android build to sentry, using the below command, we are still unable to see the decoded stacktraces.

sentry-cli upload-dif -o org-name -p project-name /Users/username/flutter-app/native-symbols/

We use these two flags to build the release app bundle: --obfuscate --split-debug-info=path

Attaching the screenshots of a dashboard event

Screenshot of Uploaded debug information files

What are we missing here? What else do we need to do for decoding the ProGuard Obfuscated android stacktraces?

Steps to reproduce:

• Step

Actual result:

• Actual

Expected result:

• Result

[marandaneto]

@ganeshrvel thanks for raising this, a few questions though.

Android Proguard is not related to symbol files, so I don't quite understand where is the issue.

Which stack traces are unreadable? the ones captured on Dart or Java/Kotlin code?

sentry-cli upload-dif -o org-name -p project-name /Users/username/flutter-app/native-symbols/

You've replaced the org-name and project-name, right? they are just place-holders

--obfuscate --split-debug-info=path

where it's path should be the /Users/username/flutter-app/native-symbols/ you've set above, have you done that? path should be the path where it outputs the files, and then you call sentry-cli passing that path.

Another test would be to skip --obfuscate and try only --split-debug-info={path}, let us know the result as well, thanks.

[ayush221b]

Hey @marandaneto I work with Ganesh, and to answer your questions:

1. We have also tried to upload the mapping.txt file generated by Proguard manually, but every time we try it says that 0 new mapping files were uploaded, so I am guessing that mapping.txt file was already uploaded by the Sentry Gradle Plugin, is that right? If it was already uploaded, can I see the uploaded file in the Sentry Dashboard like I see the symbol files which are uploaded.

2. The stack traces captured on Dart/Flutter are not readable.

3. Yes, we replaced the org and project names, that is how we were able to upload the files.

4. Yes, we have the same path, where we store the symbol files, and we upload the symbol files from

5. We would prefer to obfuscate the code though, what would be different if we didn't use the obfuscate flag?

• Also, is there a need for us to associate the uploaded symbol files with a release, if yes, how do we do it manually?

Thanks for getting back so fast, awaiting your response.

[marandaneto]

hey @ayush221b thanks for helping us out to dig this issue.

Hey @marandaneto I work with Ganesh, and to answer your questions:

1. We have also tried to upload the mapping.txt file generated by Proguard manually, but every time we try it says that 0 new mapping files were uploaded, so I am guessing that mapping.txt file was already uploaded by the Sentry Gradle Plugin, is that right? If it was already uploaded, can I see the uploaded file in the Sentry Dashboard like I see the symbol files which are uploaded.

yes, it could be, if the code didn't change at all, that would be the reason as we hash the file before trying to upload it.
mapping is only useful for Java/Kotlin stack traces, so it's not really related to this issue.

1. The stack traces captured on Dart/Flutter are not readable.

ok, so you indeed need to upload the debug symbols.

1. Yes, we replaced the org and project names, that is how we were able to upload the files.
2. Yes, we have the same path, where we store the symbol files, and we upload the symbol files from

great, just wanted to confirm.

1. We would prefer to obfuscate the code though, what would be different if we didn't use the obfuscate flag?

it was meant for testing only, there's a gotcha though, when using the obfuscate flag, it'll require a fix on the Android SDK.
https://github.com/getsentry/sentry-java/releases/tag/4.0.0-alpha.2 (Issue: https://github.com/getsentry/sentry-java/pull/1092)
which is not stable yet, it's supposed to be GA late this month, this is only a thing when combining split-debug-info AND obfuscate, but it's not really related to the issue right now, your problem is only with the split-debug-info

• Also, is there a need for us to associate the uploaded symbol files with a release, if yes, how do we do it manually?

only source maps (eg Flutter web) should be associated with a release, not Flutter Mobile, as the files already have the debug id in them which is loaded at runtime, so it's possible to find the correct files to symbolicate the stack traces.

Thanks for getting back so fast, awaiting your response.

could you confirm that the steps you've done are:

// compile (should be a release build)
flutter build apk --split-debug-info=$SYMBOLS --obfuscate
// upload symbols
sentry-cli upload-dif --org $SENTRY_ORG --project $SENTRY_PROJECT $SYMBOLS
// install, path of the release apk, that's important
adb install build/app/outputs/flutter-apk/app-release.apk
// execute and capture an event, check if it's symbolicated

also, are you using sentry.io or on-premise? if sentry.io, could you share the link of the issue above? I'd like to dig a bit.
you could send to [email protected] thanks

[marandaneto]

@ganeshrvel as you are compiling an appbundle, how are you installing the appbundle to test it out? using the bundletool CLI?

could you follow these steps and try it again?

https://docs.sentry.io/platforms/android/usage/advanced-usage/#integrating-the-ndk

the main/only point is:

set android.bundle.enableUncompressedNativeLibs=false to your gradle.properties and repeat the steps, compile, upload symbols install and test it, thanks.

[kuhnroyal]

There was an issue that the obfuscated Dart code/sourcemaps can not yet be parsed by Sentry, something was wrong with the format. Can't find it atm.

Edit:
Found it: https://github.com/flutter/flutter/issues/59321

[marandaneto]

this was with the old package, sentry v3, but it works ootb for Android on sentry_flutter

[ganeshrvel]

@marandaneto We got it working

I am leaving the steps here so that it will help others as well

We use both obfuscate and split-debug-info flags for generating the apk/bundle

# for apk
flutter build apk --obfuscate --split-debug-info=$SYMBOLS

# for bundle
flutter build appbundle --obfuscate --split-debug-info=$SYMBOLS

And of course we had to manually upload the symbols

sentry-cli login #only the first time
sentry-cli upload-dif -o org -p project $SYMBOLS

AndroidManifest file:

    <application
            ......
            android:extractNativeLibs="true" //<------ this line
    >

gradle.properties:

....
android.bundle.enableUncompressedNativeLibs=false

./build.gradle:

buildscript {
    dependencies {
         .....
        // https://github.com/getsentry/sentry-android-gradle-plugin/releases
        classpath 'io.sentry:sentry-android-gradle-plugin:1.7.36'
    }
}

./app/build.gradle:

apply plugin: 'io.sentry.android.gradle'

.....

sentry {
    autoProguardConfig true
    autoUpload false
    uploadNativeSymbols false
    includeNativeSources true
}

[marandaneto]

cool, the extractNativeLibs thingie won't be required in the future if https://github.com/getsentry/sentry-native/pull/431 gets approved.

autoProguardConfig should be false, as there will be no effect.
includeNativeSources should only be true if your own App has native code (C/C++), otherwise, it could be false.

for Dart/Flutter, in the future, this will available thru #132 but it's currently blocked.

[mathrocco]

@marandaneto is there any other channel to search for help? I'm having the same issue here, symbolicated issues with partially unreadable stacktraces. I've read almost every document on sentry's website, but I couldn't make it work completely.

At first, the stacktrace was unsymbolicated, but somehow I could make it symbolicated now. But I don't understand why the exception isn't completely readable. Here are some screenshots:

My question is if it's somehow related to the source context problem, addressed by #132, or if it should work with current supported dart and sentry features.

Thanks for the help, in advance.

[marandaneto]

https://github.com/getsentry/sentry-dart/issues/132 is something else.

what is the umsymbolicated part? if you mean _Fa, that's a tooling limitation on Flutter's side.

https://flutter.dev/docs/deployment/obfuscate#caveat
https://docs.sentry.io/platforms/flutter/troubleshooting/#known-limitations

the screenshot and its stack trace look good though, is it only about the title?

[mathrocco]

@marandaneto yes, I meant both the title and those addresses in the second column. Comparing to a stacktrace from a build without --split-debug-info, it seemed odd to me (although it has the same information in it). It wasn't clear for me that the obfuscation caveat has this effect on the console.

Thanks for the quick reply, it helped me a lot!

[marandaneto]

oh got it, they are just the image_addr that have been used for symbolication, no issues in there.