Shadowsocks-android: Forward DNS requests for all domains in proxy list

[iKirby]

There are [proxy_list] rules in both bypass-china.acl and bypass-lan-china.acl. But only DNS requests for domains in GFWList are forwarded to proxy server in these two modes.

Google is trying to bring some of its services to China, like Google Translate app. It seems that the app connects to clients1.google.cn and translate.google.cn regardless what network you're using. So using local DNS may get IPs of Google servers located in China, which may be slow (or unable to connect) for some proxy servers. (#1488)

Forwarding all DNS requests for domains in [proxy_list] should solve this issue.

btw, Google has not brought all features of Translate app to China (for example, camera translation is not available in China) so we have to connect through proxy to use all the features.

[Mygod]

Fixed in beta branch.

[iKirby]

@madeye I see that 4.4.0 is released with the fix for this issue, but it doesn't work.

I generated a list from bypass-lan-china.acl using the code used to generate the list from shadowsocks, then tested it with overture 1.3.6rc2.
In the log you can see that the rules won't match translate.google.cn and even www.google.com, but due to DNS pollution and IP network matching, overture used alternative DNS to get the correct answer for www.google.com.

time="2017-12-13T17:41:38+08:00" level=debug msg="Question from 127.0.0.1: ;translate.google.cn.\tIN\t A"
time="2017-12-13T17:41:38+08:00" level=debug msg="Domain match fail, try to use primary DNS"
time="2017-12-13T17:41:38+08:00" level=debug msg="Answer from DNSPod: translate.google.cn.\t49\tIN\tA\t203.208.39.248"
time="2017-12-13T17:41:38+08:00" level=debug msg="Answer from DNSPod: translate.google.cn.\t49\tIN\tA\t203.208.39.239"
time="2017-12-13T17:41:38+08:00" level=debug msg="Answer from DNSPod: translate.google.cn.\t49\tIN\tA\t203.208.39.255"
time="2017-12-13T17:41:38+08:00" level=debug msg="Answer from DNSPod: translate.google.cn.\t49\tIN\tA\t203.208.39.247"
time="2017-12-13T17:41:38+08:00" level=debug msg="Try to match response ip address with IP network"
time="2017-12-13T17:41:38+08:00" level=debug msg="Matched: IP network 203.208.39.248 203.208.32.0/19"
time="2017-12-13T17:41:38+08:00" level=debug msg="Finally use primary DNS"
...
time="2017-12-13T17:41:59+08:00" level=debug msg="Question from 127.0.0.1: ;www.google.com.\tIN\t A"
time="2017-12-13T17:41:59+08:00" level=debug msg="Domain match fail, try to use primary DNS"
time="2017-12-13T17:41:59+08:00" level=debug msg="Answer from DNSPod: www.google.com.\t58\tIN\tA\t69.171.237.16"
time="2017-12-13T17:41:59+08:00" level=debug msg="Try to match response ip address with IP network"
time="2017-12-13T17:41:59+08:00" level=debug msg="IP network match fail, finally use alternative DNS"
time="2017-12-13T17:41:59+08:00" level=debug msg="Answer from GoogleDNS: www.google.com.\t185\tIN\tA\t172.217.4.132"

[madeye]

What's your list?

[iKirby]

@madeye
list.txt

It was generated using the getListFile function from BaseService.kt .

[madeye]

There is no google.com and translate.google.cn in it. You may need to add them to your custom acl.

[iKirby]

@madeye I'm not using my custom acl. The acl I used is the one that comes with shadowsocks app, bypass-lan-china.acl.

Overture may not be able to recognize lines like google(.(?!-)[a-zA-Z0-9-]{1,63}(?<!-)){1,2}, which caused the domain matching problem.

[madeye]

Try b88f8366

[iKirby]

@madeye Sorry for the late reply. I made a new build but unfortunately it didn't work. The rules still won't match DNS query for translate.google.cn.

[madeye]

It looks a regexp issue. Try fca0e87.

[iKirby]

@madeye The commit finally fixed the issue. Thanks!

[iKirby]

EDIT: I'm sorry I just noticed that 4.4.3 has been released, and 4.4.3 doesn't have this issue.

@madeye Just noticed that this happens again in 4.4.2.

Tested using dig in Termux with Shadowsocks 4.4.2, route is bypass LAN & mainland china.

$ dig translate.google.cn @127.0.0.1 -p 5450

; <<>> DiG 9.11.1-P3 <<>> translate.google.cn @127.0.0.1 -p 5450
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 63107
;; flags: qr rd ra; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;translate.google.cn.           IN      A

;; ANSWER SECTION:
translate.google.cn.    3600    IN      A       203.208.48.88
translate.google.cn.    3600    IN      A       203.208.48.87
translate.google.cn.    3600    IN      A       203.208.48.95
translate.google.cn.    3600    IN      A       203.208.48.79

;; Query time: 35 msec
;; SERVER: 127.0.0.1#5450(127.0.0.1)
;; WHEN: Wed Jan 03 14:57:05 CST 2018
;; MSG SIZE  rcvd: 188

$ dig twitter.com @127.0.0.1 -p 5450

; <<>> DiG 9.11.1-P3 <<>> twitter.com @127.0.0.1 -p 5450
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 37306
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;twitter.com.                   IN      A

;; ANSWER SECTION:
twitter.com.            3600    IN      A       104.244.42.1
twitter.com.            3600    IN      A       104.244.42.129

;; Query time: 454 msec
;; SERVER: 127.0.0.1#5450(127.0.0.1)
;; WHEN: Wed Jan 03 14:57:28 CST 2018
;; MSG SIZE  rcvd: 94