Android: Scanning NFC tags always produces "Open Link?" confirmation pop-up (is it a bug?)

[donkawechico]

Home Assistant Android version:
2.3.0-306-full
Android version:
10
Phone model:
Pixel 2
Home Assistant version:
0.114.4
Last working Home Assistant release (if known):
N/A
Description of problem:
Not sure if it's a bug or not, but any time I scan an NFC tag, I get a confirmation dialog titled "Open Link?". I have the option to "Cancel" or "Open link". When I tap "Open link", an intent appears that's clearly part of Home Assistant app, and the event fires successfully.

My expectation (or rather, my "hope") is that if my phone is unlocked, I should be able to scan a tag, and have the event fire without any interaction from me.

I'm guessing this is a security measure on Android that can't be disabled, but I thought I would post the issue in the hopes that it's something that can be changed.

Traceback (if applicable):

Screenshot of problem:

Additional information:
I've verified that the "Open by default" settings show that opening links is set to open in the app, and that "www.home-assistant.io is listed as a supported link (see screenshot)

[JBassett]

That open link ui is not from this application. I also have a Pixel 2 and I'm unable to reproduce. Do you have any other NFC applications installed?

[dshokouhi]

I believe this is an android thing. On my phone Tasker, HA and another NFC app get triggered however for these tags specifically I chose HA to be the default and I no longer get that prompt. This prompt specifically I have not seen. Maybe there is another NFC app triggering this pop up?

[JayNewstrom]

I also notice this behavior, I'm running Android 10.0 on a Pixel 4a. I also think this is an Android level dialog, since when I click yes, it takes me to the home assistant app.

[donkawechico]

I also have a Pixel 2 and I'm unable to reproduce.

Are you Verizon by any chance? Maybe their flavor of Android adds this as some kind of security measure.

Do you have any other NFC applications installed?

Not that I'm aware of. I went ahead and uninstalled a few potentials like tasker, AutoTools, AutoApps, AutoVoice, etc. Still get the popup.

I also notice this behavior, I'm running Android 10.0 on a Pixel 4a.

Are you Verizon?

[JayNewstrom]

Are you Verizon?

I am on verizon, but it's an unlocked phone that I bought directly from google. I can pop the sim card and restart later tonight or sometime tomorrow to see if that could have anything to do with it though.

[donkawechico]

If it helps, here's what I get from adb logcat the moment I scan the tag and the popup appears. (I don't open link though)

09-03 11:29:40.824  3250  3250 D NativeNfcTag: Connect to a tech with a different handle
09-03 11:29:40.906  3250  9064 D NativeNfcTag: Starting background presence check
09-03 11:29:40.949   817   817 D [email protected]: AUDIO_STREAMING: 1
09-03 11:29:40.949   801  6582 D audio_hw_primary: enable_snd_device: snd_device(2: speaker)
09-03 11:29:40.950   801  6582 D audio_route: Apply path: speaker
09-03 11:29:40.951   801  6582 D ACDB-LOADER: ACDB -> send_audio_cal, acdb_id = 15, path = 0, app id = 0x11130, sample rate = 48000
09-03 11:29:40.951   801  6582 D ACDB-LOADER: ACDB -> ACDB_CMD_GET_AUDPROC_GAIN_DEP_STEP_TABLE, vol index 0
09-03 11:29:40.951   801  6582 D ACDB-LOADER: ACDB -> GET_AFE_TOPOLOGY_ID for adcd_id 15, Topology Id 112fc
09-03 11:29:40.951   801  6582 D         : Failed to fetch the lookup information of the device 0000000F 
09-03 11:29:40.951   801  6582 D ACDB-LOADER: Error: ACDB AFE returned = -19
09-03 11:29:40.951   801  6582 D audio_hw_primary: enable_audio_route: usecase(1) apply and update mixer path: low-latency-playback speaker
09-03 11:29:40.951   801  6582 D audio_route: Apply path: low-latency-playback speaker
09-03 11:29:40.985   801  6582 D audio_hw_primary: out_write: retry previous failed cal level set
09-03 11:29:40.985   817   817 D [email protected]: AUDIO_STREAMING: 0
09-03 11:29:41.793   826   937 I CHRE    : @ 2290.159: [ImuCal] [NanoSensorCal:GYRO_RPS] Offset | Temperature [C]: 0.004153, -0.007389, 0.001480 | 29.74
09-03 11:29:41.793   826   937 I CHRE    : @ 2290.159: [ImuCal] [NanoSensorCal:GYRO_RPS] Temp Sensitivity: 0.000247, -0.000070, -0.000148
09-03 11:29:41.793   826   937 I CHRE    : @ 2290.159: [ImuCal] [NanoSensorCal:GYRO_RPS] Temp Intercept: -0.003216, -0.005279, 0.005893
09-03 11:29:42.279  8408  8433 I Finsky  : [130] lba.run(36): Stats for Executor: BlockingExecutor ldc@4758b8b[Running, pool size = 2, active threads = 0, queued tasks = 0, completed tasks = 12]
09-03 11:29:42.281  8408  8433 I Finsky  : [130] lba.run(36): Stats for Executor: LightweightExecutor ldc@f12f468[Running, pool size = 4, active threads = 0, queued tasks = 0, completed tasks = 72]
09-03 11:29:42.368  8408  8433 I Finsky  : [130] lba.run(36): Stats for Executor: bgExecutor ldc@1f9f081[Running, pool size = 4, active threads = 0, queued tasks = 0, completed tasks = 67]

[donkawechico]

I can pop the sim card and restart later tonight or sometime tomorrow

That would be really awesome. Thank you.

[JayNewstrom]

I also wonder if this could be related to Android Pay. I'm using Android pay with a few cards.

[donkawechico]

I also wonder if this could be related to Android Pay. I'm using Android pay with a few cards.

I wondered the same, so I went ahead and uninstalled it. Still seeing the popup :-

I also just tried installing Current Activity, which tells you what activity is currently running. It doesn't show any activity for the popup.

On the surface, that sounds like evidence that this is an OS level thing, but maybe a dialog can be something other than an "activity"?

[JayNewstrom]

This has to be related to Verizon/my SIM card. It's as easy as taking the SIM card out and restarting the device, scan the NFC tag, and the dialog no longer appears (launches directly into the home assistant app, and runs the tag scan).

Immediately after putting the SIM card back in, scanning the NFC tag, the dialog appears.

[JayNewstrom]

It's worth noting this only happens with https urls (home assistant related or not). If I write another NFC tag with a custom scheme (homeassistanttestapp://tag/abc-def-etc) then it goes straight to the application, without interception. I wonder if it would be worth adding a configuration to do the universal NFC links or a custom scheme.

[donkawechico]

Good detective work! Thank you so much for investigating and finding that potential workaround.

It's worth noting this only happens with https urls

I assume you mean any http protocol (https and http)? Or are you saying https produces popup, http does not.

I wonder if it would be worth adding a configuration to do the universal NFC links or a custom scheme.

That would be _great_ for me. The pop-up makes NFC tags not much better (maybe even worse) than a single-tap home screen widget.

[JayNewstrom]

@JBassett / @dshokouhi would the be a feature you'd accept?

[dshokouhi]

So the reason why we went with https://www.home-assistant.io for the URL is so that the same tag can be used for iOS and Android. I don't think we can change it as its also what HA core is expecting.

https://companion.home-assistant.io/docs/integrations/universal-links

[donkawechico]

I'm a dev, but know next to nothing about NFC flows, schemas, or how HA consumes NFC tags, so apologies for the ill-informed suggesti-question.

But can it be something like this:

1. NFC read/write activity has an optional checkbox for writing a different tag scheme: (e.g. homeassistanttestapp://tag/abc-def-etc)
2. Write the alternate tag onto NFC tag
3. When tag scanned, an HA activity is launched which translates the alternate scheme... waves hands around to indicate dev magic... into the home-assistant.io URL for use in core?

[JBassett]

I'm a dev, but know next to nothing about NFC flows, schemas, or how HA consumes NFC tags, so apologies for the ill-informed suggesti-question.

But can it be something like this:

1. NFC read/write activity has an optional checkbox for writing a different tag scheme: (e.g. homeassistanttestapp://tag/abc-def-etc)
2. Write the alternate tag onto NFC tag
3. When tag scanned, an HA activity is launched which translates the alternate scheme... waves hands around to indicate dev magic... into the home-assistant.io URL for use in core?

The problem with this is we are trying to come up with a standard url so any NFC device can scan the tags that are written by the app. If we do this the iOS app for example won't be able to open the link.

[donkawechico]

Ah, I see.

And I assume it wouldn't suffice to say that this could be the choice / responsibility of the user? For example, my gf and I only use Android. So we'd be fine with choosing the iOS-incompatible scheme for our personal tags.

I totally understand feature creep, and believe you folks know best how to balance user needs, maintenance costs, and feature direction. I just wonder if there's some middle-ground here for folks with Pixel + Verizon, which I assume is a decent chunk of folks.

[JayNewstrom]

I think the ideal way to solve this is to figure out what application is adding this behavior and uninstall/disable it.

[donkawechico]

I think the ideal way to solve this is to figure out what application is adding this behavior and uninstall/disable it.

Oh. I guess I misunderstood. Thought your sim test meant this was something specific to Verizon hardware. Also, the "Current Activity" app didn't detect any running activity with the popup, so it feels OS-y to me.

But maybe we could compare app lists and see what's in common just to see?

[JayNewstrom]

I did a factory reset, and my same results are happening from https://github.com/home-assistant/android/issues/876#issuecomment-689084616

This has to be something Verizon is doing, but I've read through the logs as much as I could, with no obvious signs.

It doesn't have anything to do with any apps installed from the user perspective. Again, this has to be something Verizon is configuring/adding to the device while connected to their network.

[donkawechico]

It also seems like it may be specific to Pixels. My gf has Samsung + Verizon and she does not experience the popup.

[MattAKAFred]

Not much to add, but just wanted to share this also affects Pixel 3 + Android 11 on Verizon.

[fcassirer]

I have the same issue on a Pixel 2 XL, I have no Verizon cruft on my phone, I bought it directly from Google. This just feels like an OS level dialog, it even has a very googly look & feel. As I'm sure someone has noted, having to click the link on an open dialog defeats the purpose of having NFC and just tapping and going. I could have just created a shortcut for the HA action on my phone ... I haven't been successful in finding any useful references to this dialog other than here ...

[dshokouhi]

@fcassirer its not about verizon cruft but more about the verizon sim card that seems to be at play here, take out your sim and try it like they did up above. I have a pixel on Google Fi and I do not see this issue, neither does my wife who has a pixel 3 xl. It does not surprise me that a carrier will make modifications outside the norm here.

[donkawechico]

I haven't been successful in finding any useful references to this dialog other than here ...

I had the same experience :( Even did reverse image lookups to see if anyone posted screenshots anywhere. Nuthin'

[fcassirer]

@dshokouhi i can try they but my suspicion is that even if it were to make a difference it would likely be due to something with cellular vs WiFi rather than Verizon. I can't imagine there is any software layer installed by Verizon based on the Sim card. perhaps some security check used for Google pay over cellular (Just thinking out loud here as Verizon Sim based. specific behavior seems unlikely to me) Hopefully at some point we will stumble upon an explanation ...

[donkawechico]

Well, I found a new clue (maybe). Doesn't fix the issue, but I was able to make the pop-up invisible by finding the System app called "Nfc Service" (Go to Apps->"Show System" and search the list) and disabling its ability to draw over other apps.

Weirdly, if I "Force Stop" the service, nothing changes... the popup still appears (or is invisible) and blocks the request.

But it might be a clue as to the true source of the issue.

(Also, I finally tried removing SIM card and can confirm that it removes the popup behavior and lets the request through as expected)

[donkawechico]

Ahhh. Just got Android 11 released to my Pixel 2 this morning and now the popup shows a new sentence:

"Your phone received a link through NFC"

I was able to use that to find the (older version of the) source code responsible:

https://android.googlesource.com/platform/packages/apps/Nfc/+/refs/heads/master/src/com/android/nfc/NfcDispatcher.java#768

That line of code is the source of the "Open link?" title text. So the modal is definitely from google's Nfc package.

[dshokouhi]

I bet if you keep looking you'll see that theres some carrier specific override somewhere. It wouldn't be the first time I heard of this in AOSP, they did it for Sprint when 4g WiMax was a thing lol. It was used to replace an icon because they wanted it.

[donkawechico]

I'm not really seeing anything carrier-specific in here (admittedly not sure what to look for), but here's the commit that introduced it

And here's the message included in the commit/PR

Confirmation dialog for opening NFC-scanned urls

This adds an confirmation dialog displayed before opening an url from a
scanned NFC tag. The feature is off by default for now and will be
selectively enabled via a build overlay, but we plan to turn this on
unconditionally in the future.

[dshokouhi]

Might not be located there but we can see that by default its still not enabled https://cs.android.com/android/platform/superproject/+/master:packages/apps/Nfc/res/values/config.xml;l=4

Maybe see if this search site will let you find where it is so you can submit a AOSP fix to get it fixed upstream :)

[donkawechico]

Reading the code path, I can see that there's a config option called "enable_nfc_url_open_dialog" that's set to false by default. And indeed, if it's false, we exit early, so _something_ must be setting that to true. Maybe that's Verizon?

Another "out" available is this line:

if (dispatch.isWebIntent() && dispatch.hasIntentReceiver()) {
   // launch the popup
}

[donkawechico]

Might not be located there but we can see that by default its still not enabled

Ha. Jinx :)

[donkawechico]

@dshokouhi So, if adding the option to write homeassistant: schema tags is off the table in Companion, what about letting the app _read_ those types of tags when encountered? Or will it do that already?

Like, could I currently write that style schema to a tag in such a way that companion will react and process it? Or does it currently require the http style to be able to process it.

[fcassirer]

isn't the issue with using the non http protocol that it is platform specific? We have both iOS and Android phones and want to use the same NFC tag. I believe I Have seen a post where someone said the alternate protocol worked on Android but the HA devs said they wanted it to be platform neutral. Fwiw, I find we have to hit an ok dialog on iOS too ... Again, makes the whole use case of the tap and go pointless. Good job on the deep dives

[donkawechico]

isn't the issue with using the non http protocol that it is platform specific?

Yes. Companion doesn't want to support writing and reading NFC tags with android-specific schemas.

What I'm asking is whether allowing the android companion to _read_ (but not write) android schemas is a possibility, or if that's still too big a violation of the principle they're trying to stick to.

Good job on the deep dives

Thanks :) Was definitely satisfying to find the source. But disappointing to see there's not a whole lot to be done

[fcassirer]

If you want to write a tag with the alternate protocol, wouldn't that just launch to the app without the app needing to be aware? The payload might need to be crafted creatively to deliver the tag uuid. Again, pretty sure someone got that to work, but it still makes it Android only. Tags are cheap so having an Android and iOS one side by side might not be the worst thing even if it is a bit silly.

[donkawechico]

Again, pretty sure someone got that to work

Are you referring to this comment? I interpreted that as just "it launched the app", not "it launched the app and successfully sent the event". But maybe I'm wrong. @JayNewstrom ?

[fcassirer]

I think I have it working for both Android and IOS, no pop-ups. So on Android I downloaded an app called NFC Tag Writer and it has an option to read a tag, save it in a local data store and then write it back. I wrote the tag written by HA back to another NFC tag and then used the saved link in 'My datasets' to write a clone of the tag. I then added an application via the checkbox in the datastore 'Add launch application' to launch to the tag. This appears to have added 52 bytes of metadata which shows HA as the launch app. When I now tap that tag, it works without the popup. It also works on IOS.

I suspect we just need to see if the HA tag writer can add this additional metadata when writing the tag.

FYI: This is an NFC AAR record.

[redwngsrul]

I think I have it working for both Android and IOS, no pop-ups. So on Android I downloaded an app called NFC Tag Writer and it has an option to read a tag, save it in a local data store and then write it back. I wrote the tag written by HA back to another NFC tag and then used the saved link in 'My datasets' to write a clone of the tag. I then added an application via the checkbox in the datastore 'Add launch application' to launch to the tag. This appears to have added 52 bytes of metadata which shows HA as the launch app. When I now tap that tag, it works without the popup. It also works on IOS.

I suspect we just need to see if the HA tag writer can add this additional metadata when writing the tag.

FYI: This is an NFC AAR record.

I can confirm this solution worked for me too. YAY! Nice work @fcassirer

[JayNewstrom]

Looks like it's a pretty easy change.

https://developer.android.com/guide/topics/connectivity/nfc/nfc.html#aar

https://github.com/home-assistant/android/blob/master/app/src/main/java/io/homeassistant/companion/android/nfc/NFCUtil.kt#L19

I can submit a PR later if no one beats me to it.

[fcassirer]

That would be super convenient. I poked around a bit (not extensively) and haven't seen any similar capability available in the NFC spec that would be honored on IOS, you still have to acknowledge the launch/link pop-up.

[donkawechico]

FYI: This is an NFC AAR record

Makes sense! This line in google NFC code shows that if it's AAR, it exits the popup code early.

Awesome!

[JayNewstrom]

We'll also need to update the iOS app to write this metadata for the tag.

[JayNewstrom]

Android PR is merged. I'm working on the iOS PR now.

[JBassett]

Fixed by https://github.com/home-assistant/android/pull/969

[donkawechico]

@JayNewstrom Can confirm this works for me on Android now!! Big thanks to you, @fcassirer, and @dshokouhi for your help!