Firefox-ios: Remove use of google-mapping.json

[say-yawn]

With the introduction of disconnect-mapping.json, google-mapping.json was deprecated and no longer being used to generate tracking protection lists.

Acceptance Criteria

• [ ] google-mapping.json is removed

[garvankeeley]

There is no disconnect-mapping.json in shavar that I see https://github.com/mozilla-services/shavar-prod-lists, is not released yet?

[say-yawn]

Oh! It was at shavar-list-creation https://github.com/mozilla-services/shavar-list-creation/blob/master/disconnect_mapping.json

[garvankeeley]

Why is this file in a different repo? Isn't it to be used in combination with the disconnect list in shavar-prod-lists?

[englehardt]

Why is this file in a different repo? Isn't it to be used in combination with the disconnect list in shavar-prod-lists?

I suspect there is some legacy reason as to why it's in that repo. It used to be a python file of constats imported by the list creation script.

It could definitely be moved to the shavar-prod-lists repo. Is there a reason you need it moved?

[say-yawn]

https://github.com/mozilla-services/shavar-prod-lists/pull/110 PR re-organized all the domains in Disconnect Category to the respective categories and as such disconnect_mapping.json is no longer needed. @garvankeeley can we close this issue now?

[garvankeeley]

I don't think we should close it, there has been no change to the parser, it still uses google-mapping.json: https://github.com/mozilla-mobile/firefox-ios/blob/master/content-blocker-lib-ios/ContentBlockerGen/Sources/ContentBlockerGenLib/ContentBlockerGenLib.swift#L24

[say-yawn]

Do you have a timeline on when you expect to remove it?

[say-yawn]

Updated the requirements to just remove the google_mapping.json (Disconnect category no longer exists).

[garvankeeley]

I'll try get this bug triaged in our next meeting

[say-yawn]

Hi @garvankeeley, just wondering if you have any updates in regards to this? Thank you!

[garvankeeley]

I added it to the sprint backlog, it should get picked up in the next two weeks.

[garvankeeley]

Updated the requirements to just remove the google_mapping.json (Disconnect category no longer exists).

Hi @skim1102, can you clarify the following. I see the changes as being
1) remove use of google_mapping.json. (One concern I have about this is that the file is not removed from the repo https://github.com/mozilla-services/shavar-prod-lists/blob/master/google_mapping.json)
2) there is no more 'Disconnect' category in the disconnect-blacklist.json, therefore I must remove this from our parser or it will crash as it expects that category to exist.
3) possibly I need to add handling for disconnect-mapping.json? I don't know anything about that file, and we aren't handling it currently.
4) Update our dependency to the current tip of master (I assume). But there is a missing file on master normalized-lists/base-fingerprinting-track.json it was committed to a branch here: https://github.com/mozilla-services/shavar-prod-lists/commit/777688fdf5420c2e00c6cde2689a9d5a21796d4d

[say-yawn]

1. remove use of `google_mapping.json`. (One concern I have about this is that the file is not removed from the repo https://github.com/mozilla-services/shavar-prod-lists/blob/master/google_mapping.json)

We will remove this once you can tell me that you are no longer using the file. It will be documented on https://github.com/mozilla-services/shavar-list-creation/issues/118

2. there is no more 'Disconnect' category in the `disconnect-blacklist.json`, therefore I must remove this from our parser or it will crash as it expects that category to exist.

Yes, this will be fine since all the domains that was in the Disconnect category was mapped into the existing categories.

3. possibly I need to add handling for `disconnect-mapping.json`? I don't know anything about that file, and we aren't handling it currently.

No, disconnect-mapping.json has been removed from being used in the config (https://github.com/mozilla-services/shavar-list-creation-config/pull/60) and in the list creation script (https://github.com/mozilla-services/shavar-list-creation/pull/121) and as such you will not need to use this file.

4. Update our dependency to the current tip of master (I assume). But there is a missing file on master `normalized-lists/base-fingerprinting-track.json ` it was committed to a branch here: [mozilla-services/shavar-prod-lists@777688f](https://github.com/mozilla-services/shavar-prod-lists/commit/777688fdf5420c2e00c6cde2689a9d5a21796d4d)

Dependency should be pointing to the latest desktop release version of normalized-lists/base-fingerprinting-track.json current latest release is 75, with 76 becoming the new release on May 5th. Not sure when the new release date is but please ping me if the normalized-lists/base-fingerprinting-track.json is not available when this is being worked on.

Hope this helps!

[garvankeeley]

We will remove this once you can tell me that you are no longer using the file. It will be documented on mozilla-services/shavar-list-creation#118

ok, no need to wait, our dependencies are pinned to a SHA

Dependency should be pointing to the latest desktop release version of normalized-lists/base-fingerprinting-track.json current latest release is 75, with 76 becoming the new release on May 5th. Not sure when the new release date is but please ping me if the normalized-lists/base-fingerprinting-track.json is not available when this is being worked on.

@skim1102 normalized-lists is not on the 75 or 76 branch that I can find, so we can't use those.
Filed https://github.com/mozilla-services/shavar-prod-lists/issues/174

[garvankeeley]

Unblocking as my questions are answered, google-mapping.json is in fact redundant on the branch iOS is on, and I can wait on the linked shavar PR until a newer shavar branch is updated and switch iOS to point to that, and land it together with the google-mapping change.

[SimonBasca]

Verified fix on 26.0 (18064).