Sentry-java: Proguard support

[bretthoerner]

Just taking public notes here. The work will actually span multiple repos.

My understanding is we need (roughly):

• A way for users to upload the Proguard mapping files when they do a release (part of sentry-cli, and a Gradle plugin that uses it)
• Support for Proguard processing in Sentry, which was started here but has been blocked while some reprocessing PRs were finished

I'm not sure what else is necessary yet. Proguard is nice in that the mapping files are generated locally when you do a build, so I don't think we need to do any of the difficult stuff we do for iOS as far as chasing down symbol files on someone else's server...

@mitsuhiko is the real expert on Sentry (re-)processing and mobile so maybe he can chime in here.

[AndreasBackx]

I have an idea in mind on how we could upload proguard mapping files. Currently Fabric's Crashlytics (which is what we're using for Android) has an Android Studio plugin which seems to automatically upload your mapping file:

For Fabric to properly de-obfuscate your crash reports, you need to remove this line from your configuration file, or we won’t be able to automatically upload your mapping file:

Source

An Android Studio plugin could be created that has the same behaviour. The plugin should be relatively easy to make. We would only need to figure out how to trigger some code to run when a signed APK is made and allow for the developer to input some values. But then I'm also wondering what we should do if someone generates 2 signed APKs and only uploads the first one? This might be an issue for the core.

I haven't worked with sentry-cli yet, but another option would be to be able to offer a plugin to the user that triggers that a script containing sentry-cli stuff. I feel that Sentry should be as easy as Crashlytics when it comes down to ease-of-use and integration with existing tools.

You're also right that nothing difficult needs to be done in order to obtain the mapping file.

[mitsuhiko]

My guess is we extend sentry-cli and then invoke that as part of the build process in android studio. Missing proguard files (with reprocessing enabled) can be dealt with in a delayed fashion if users want that.

[bretthoerner]

I think an Android Studio plugin is a good long term goal, but I think for now just offering sentry-cli support that they can trigger as part of their build (as you've both said) is enough for a first version.

@AndreasBackx I'm not sure what you mean about generating 2 APKs and uploading one. Is generating multiple APKs from an Android project common? Or are you referring to something else?

[AndreasBackx]

@bretthoerner I'm referring to an edge case that should perhaps be considered. I personally haven't really looked into whether mappings change that much between builds and whether this should even be considered. But it was a question worth asking imo.

@mitsuhiko sounds reasonable. I would perhaps be up for making the Android Studio plugin. Perhaps not in this repository depending on the opinions about that here. I'd be my first IntelliJ/Android Studio plugin and I guess it would be a relatively easy plugin to get started there.

[Hazer]

@bretthoerner @AndreasBackx There are many cases of multiple APKs and their mappings really change. I can generate multiple APKs from the same project, and its quite common. There are cases for free/pro version, different processor architecture versions or simply slightly different staging/production code. Nice reference of how complex it can be.

This system, usually use different versionCodes for architecture and different package names for things like free/pro-staging/production, so I don't think it will be an issue if some version misses, as it is easy to track which one.

@AndreasBackx I may be wrong, but I believe using a Gradle Plugin instead of an Android Studio plugin could be easier to get started and can even be better, as it can be used in CI servers and have higher integration with the build system, making it easier to upload the right mappings, it is also easy to make settings like "fail build if uploading failed", "uploading retry count" and so on. It can be just a wrapper around sentry-cli.

The Android Studio plugin, for me, fits in the same realm as a Fastlane plugin for example, nice for the long term, will be harder to reason about (multiple apks, UI, installation) than Gradle itself, kind of get out of scope and the sentry-cli may be sufficient in most cases.

Bottom line, it all comes down to the sentry-cli.

Would be nice to hear @mitsuhiko thoughts on this.

[mitsuhiko]

My preference would be to load this all into sentry-cli and wrap it in plugins. This is also what @HazAT's last changes to the fastlane plugin were. It means we just have one place we need to change and the plugins and integrations themselves are light.

[AndreasBackx]

@Hazer I'll check out the reference later tonight when I'm home. Thank you.

Regarding the AS/Gradle plugin, a Gradle plugin would definitely be better. The AS plugin could simply use that Gradle plugin even. I think Fabric adds the Gradle plugin during the AS plugin's setup, can't check right now.

[bretthoerner]

Proguard support was merged (with a Gradle plugin). Both to be released shortly, would love if anyone on this thread gave it a whirl.

[bretthoerner]

See docs here: https://docs.sentry.io/clients/java/modules/android/

Need to publish the Gradle plugin and do a sentry-java release still. In the next day I hope.

[AndreasBackx]

@bretthoerner great to hear this! I'll be moving from Crashlytics to Sentry for Android and then all of our projects will be using Sentry. Are there any gotchas that I perhaps need to look out for?

[bretthoerner]

No gotchas that I know of. I just pushed sentry-android 1.2.0 and the Gradle plugin 1.0.0-beta. They will take a while to sync to mirrors. The documentation should cover usage, please let us know if you run into issues.

[andreidiaconu]

Hi guys! Landed here looking for solutions for a crash I'm getting during the build process on our continuous integration machine. I'll probably also submit a bug and look into the issue myself, but until I do that, here is the stacktrace

Caused by: java.lang.NullPointerException: Cannot invoke method getFile() on null object
  at io.sentry.android.gradle.SentryPlugin.getSentryCli(SentryPlugin.groovy:61)
  at org.gradle.internal.metaobject.BeanDynamicObject$GroovyObjectAdapter.invokeOpaqueMethod(BeanDynamicObject.java:579)
  at org.gradle.internal.metaobject.BeanDynamicObject$MetaClassAdapter.invokeMethod(BeanDynamicObject.java:506)
  at org.gradle.internal.metaobject.BeanDynamicObject.tryInvokeMethod(BeanDynamicObject.java:191)
  at org.gradle.internal.metaobject.ConfigureDelegate.invokeMethod(ConfigureDelegate.java:80)
  at io.sentry.android.gradle.SentryPlugin$_apply_closure1_closure2.doCall(SentryPlugin.groovy:157)
  at org.gradle.api.internal.ClosureBackedAction.execute(ClosureBackedAction.java:71)
  at org.gradle.util.ConfigureUtil.configureTarget(ConfigureUtil.java:160)
  at org.gradle.util.ConfigureUtil.configure(ConfigureUtil.java:106)
  at org.gradle.util.ConfigureUtil$1.execute(ConfigureUtil.java:123)
  at org.gradle.api.internal.DefaultDomainObjectCollection.all(DefaultDomainObjectCollection.java:136)
  at org.gradle.api.internal.DefaultDomainObjectCollection.all(DefaultDomainObjectCollection.java:154)
  at org.gradle.api.DomainObjectCollection$all.call(Unknown Source)
  at io.sentry.android.gradle.SentryPlugin$_apply_closure1.doCall(SentryPlugin.groovy:136)
  at org.gradle.listener.ClosureBackedMethodInvocationDispatch.dispatch(ClosureBackedMethodInvocationDispatch.java:40)
  at org.gradle.listener.ClosureBackedMethodInvocationDispatch.dispatch(ClosureBackedMethodInvocationDispatch.java:25)
  at org.gradle.internal.event.AbstractBroadcastDispatch.dispatch(AbstractBroadcastDispatch.java:42)
  at org.gradle.internal.event.BroadcastDispatch$SingletonDispatch.dispatch(BroadcastDispatch.java:230)
  at org.gradle.internal.event.BroadcastDispatch$SingletonDispatch.dispatch(BroadcastDispatch.java:149)
  at org.gradle.internal.event.AbstractBroadcastDispatch.dispatch(AbstractBroadcastDispatch.java:58)
  at org.gradle.internal.event.BroadcastDispatch$CompositeDispatch.dispatch(BroadcastDispatch.java:324)
  at org.gradle.internal.event.BroadcastDispatch$CompositeDispatch.dispatch(BroadcastDispatch.java:234)
  at org.gradle.internal.event.ListenerBroadcast.dispatch(ListenerBroadcast.java:140)
  at org.gradle.internal.event.ListenerBroadcast.dispatch(ListenerBroadcast.java:37)
  at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:93)
  at com.sun.proxy.$Proxy23.afterEvaluate(Unknown Source)
  at org.gradle.configuration.project.LifecycleProjectEvaluator.notifyAfterEvaluate(LifecycleProjectEvaluator.java:75)
  ... 76 more

I should say that the plugin works as expected on my machine. Great job, btw!

We are using circleCI and the build step it fails at is ./gradlew dependencies

[mitsuhiko]

That is interesting. Looks like the resource loader is unavailable (at least that's why getResource might fail). We will have a look at this.

As a temporary workaround you can download sentry-cli manually and put the path to it into your sentry.properties:

cli.executable=/path/to/sentry-cli

[andreidiaconu]

Thanks for the quick reply and workaround. After looking at the code I think it is also worth mentioning that my local machine is a mac os and circleCI is a linux machine

[andreidiaconu]

For anyone landing here with similar issues, adding (cd .. && npm install sentry-cli-binary) before dependencies (dependencies->pre) did the trick as a temporary solution

[bretthoerner]

Found the underlying issue btw: https://github.com/getsentry/sentry-java/pull/455

[Shashank58]

Hi, I recently switched from firebase crash to sentry and I'm having some problems when I try to generate a signed apk(note this has proguard enabled). I followed your guide https://docs.sentry.io/clients/java/modules/android/#proguard but I keep getting an invalid token 401 like so

:app:persistSentryProguardUuidsForRelease
> compressing mappings
> uploading mappings

error: http error: Invalid token (401)

:app:persistSentryProguardUuidsForRelease FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':app:persistSentryProguardUuidsForRelease'.
> Process 'command '/var/folders/zz/53_11t613cb35r9hkzrx_dr00000gn/T/.sentry-cli1816203041936387711.exe'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

BUILD FAILED

Total time: 9.147 secs

My sentry.properties, just like its suggested in the docs is

defaults.org=my_org
defaults.project=my_project
auth.token=my_auth_token

I generated my auth token from https://sentry.io/api/ and have the same on sentry.properties file. Can someone please confirm if that is the right auth.token that i should be putting in?

[mitsuhiko]

Yes, that's where you get the auth token from. I would recommend running sentry-cli separately from the shell eg: SENTRY_PROPERTIES=/path/to/sentry.properties sentry-cli info to see if it works and why not (--log-level=debug can help to diagnose this).

[Shashank58]

When I run the above command I get this

[INFO] sentry_cli::api request GET https://sentry.io/api/0/
[INFO] sentry_cli::api > GET /api/0/ HTTP/1.1
[INFO] sentry_cli::api > Host: sentry.io
[INFO] sentry_cli::api > Accept: */*
[INFO] sentry_cli::api > User-Agent: sentry-cli/1.16.0
[INFO] sentry_cli::api < HTTP/1.1 200 OK
[INFO] sentry_cli::api < Server: nginx
[INFO] sentry_cli::api < Date: Thu, 06 Jul 2017 08:03:55 GMT
[INFO] sentry_cli::api < Content-Type: application/json
[INFO] sentry_cli::api < Content-Length: 44
[INFO] sentry_cli::api < Connection: keep-alive
[INFO] sentry_cli::api < X-XSS-Protection: 1; mode=block
[INFO] sentry_cli::api < Content-Language: en
[INFO] sentry_cli::api < X-Content-Type-Options: nosniff
[INFO] sentry_cli::api < Vary: Accept-Language, Cookie
[INFO] sentry_cli::api < Allow: GET, HEAD, OPTIONS
[INFO] sentry_cli::api < X-Frame-Options: deny
[INFO] sentry_cli::api < X-Served-By: web-3d5bfb0f
[INFO] sentry_cli::api < Strict-Transport-Security: max-age=31536000; includeSubDomains; preload
[INFO] sentry_cli::api response: 200
[INFO] sentry_cli::api body: {"version": "0", "user": null, "auth": null}
Sentry Server: https://sentry.io/
Default Organization: -
Default Project: -

[mitsuhiko]

That looks like no auth token is actually sent. Did you point the SENTRY_PROPERTIES environment variable to the right properties file?

[Shashank58]

Oops, had the wrong path, this is the result with corrected path

[INFO] sentry_cli::api request GET https://sentry.io/api/0/
[INFO] sentry_cli::api using token authentication
[INFO] sentry_cli::api > GET /api/0/ HTTP/1.1
[INFO] sentry_cli::api > Host: sentry.io
[INFO] sentry_cli::api > Accept: */*
[INFO] sentry_cli::api > User-Agent: sentry-cli/1.16.0
[INFO] sentry_cli::api > Authorization: Bearer 2d3a7a47***
[INFO] sentry_cli::api < HTTP/1.1 401 UNAUTHORIZED
[INFO] sentry_cli::api < Server: nginx
[INFO] sentry_cli::api < Date: Thu, 06 Jul 2017 08:07:41 GMT
[INFO] sentry_cli::api < Content-Type: application/json
[INFO] sentry_cli::api < Content-Length: 27
[INFO] sentry_cli::api < Connection: keep-alive
[INFO] sentry_cli::api < X-XSS-Protection: 1; mode=block
[INFO] sentry_cli::api < X-Content-Type-Options: nosniff
[INFO] sentry_cli::api < Content-Language: en
[INFO] sentry_cli::api < Vary: Accept-Language, Cookie
[INFO] sentry_cli::api < Allow: GET, HEAD, OPTIONS
[INFO] sentry_cli::api < X-Frame-Options: deny
[INFO] sentry_cli::api < WWW-Authenticate: xBasic realm="api"
[INFO] sentry_cli::api < X-Served-By: web-c9683d4c
[INFO] sentry_cli::api < Strict-Transport-Security: max-age=31536000; includeSubDomains; preload
[INFO] sentry_cli::api response: 401
[INFO] sentry_cli::api body: {"detail": "Invalid token"}
Sentry Server: https://sentry.io/
Default Organization: <my_org_name>
Default Project: <my_project_name>

Authentication Info:
  Method: Auth Token
  (failure on authentication: http error: Invalid token (401))

[mitsuhiko]

Sounds like your token is not valid. Are you sure you did not accidentally make a copy paste error?

[Shashank58]

Is there anyway I can log which auth is being sent from sentry-cli? I pretty sure I copy-pasted the proper auth

[mitsuhiko]

@Shashank58 you can see the truncated auth token in the log above.

[Shashank58]

Yup, that's the same auth (The 'Bearer' not included though).

[mitsuhiko]

The only suggestipn I have is to recreate the token. Maybe it corrupted somehow but that's a stretch. In any case that would be a server error and I suggest opening a support ticket.

[huck1eberry]

Hello everyone.
Sorry if I missed something, but it is not clear for me how sent mapping files are linked to related app versions (with specific applicationId and version code). Does it happen automatically via generated/merged manifest data (passed as command argument in case of the manual upload)? Or do we have to make some additional configuration declarations to specify the target version?
I'am not familiar with Sentry' report-managing view, but is there any possibility to have multiple mapping files for different app versions (with "deobfuscated" crashlogs for current and previous app versions) like we have in the Google Play console.

[bretthoerner]

Hey @huck1eberry,

If you use the Sentry Android Gradle Plugin as documented here: https://docs.sentry.io/clients/java/modules/android/

It will generate a sentry-debug-meta.properties file that will be included in your APK. It contains a UUID that maps to the current version in your Android manifest, and that UUID + your Proguard files are uploaded to Sentry and linked together. At runtime the Android SDK will read that properties file when an exception occurs and send the UUID along with the error so that the server backend knows which Proguard files to use.

All Proguard mappings are kept Sentry-side so multiple versions can be running concurrently in the wild and will be deobfuscated for you.

Does that answer the question?

[huck1eberry]

Sounds great! Thank you for your prompt reply.
Is there a way to update a mapping file for a particular version? If we have to rebuild this version of the app.

[bretthoerner]

If the UUID is the same the last upload should overwrite and "win."

[adarzulfu]

Hi, I have problem with proguard mapping implementation. I try to use the plugin for auto updating mapping file. I create sentry.properties under project level root.

Not: it is for an android library project

• What went wrong:
  A problem occurred configuring project ':xxx'.
  > Ambiguous method overloading for method java.io.File#.
  Cannot resolve which method to invoke for [null, class java.lang.String] due to overlapping prototypes between:
  [class java.lang.String, class java.lang.String]
  [class java.io.File, class java.lang.String]

[bruno-garcia]

@adarzulfu This issue is closed. Please open a new issue. If possible, please provide a repro to make it faster for us to investigate.