Packages: <ddns-scripts_cloudflare.com-v4>: cloudflare format it's json request

[Lyoko-Jeremie]

Maintainer: @lantis1008 (find it by checking history of the package Makefile)
Environment: (put here arch, model, OpenWrt version)
OpenWrt 19.07.1 r10911-c155900f66 / LuCI openwrt-19.07 branch git-20.048.32085-85a6f07

ddns-scripts_cloudflare.com-v4 : 2.7.8-12

curl --version :
curl 7.66.0 (i486-openwrt-linux-gnu) libcurl/7.66.0 mbedTLS/2.16.3
Release-Date: 2019-09-11
Protocols: file ftp ftps http https
Features: IPv6 Largefile SSL

Description:

i used the ddns-scripts_cloudflare for more than 6m , it work well before today.
but now i get a error from it , follow is the log :

(i replace all the ip/id/email to a ramdon, it not important .)

 132435  note : PID '31932' started at 2020-04-29 13:24
 132435       : ddns version  : 2.7.8-12
 132435       : uci configuration:
ddns.myddns_ipv4.cacert='IGNORE'
ddns.myddns_ipv4.check_interval='5'
ddns.myddns_ipv4.domain='[email protected]'
ddns.myddns_ipv4.enabled='1'
ddns.myddns_ipv4.force_interval='10'
ddns.myddns_ipv4.interface='wan10'
ddns.myddns_ipv4.ip_interface='pppoe-wan10'
ddns.myddns_ipv4.ip_source='interface'
ddns.myddns_ipv4.lookup_host='eth10.aaa.com'
ddns.myddns_ipv4.password='***PW***'
ddns.myddns_ipv4.service_name='cloudflare.com-v4'
ddns.myddns_ipv4.use_https='1'
ddns.myddns_ipv4.username='[email protected]'
ddns.myddns_ipv4=service
 132435       : verbose mode  : 0 - run normal, NO console output
 132435       : check interval: 300 seconds
 132435       : force interval: 36000 seconds
 132435       : retry interval: 60 seconds
 132435       : retry counter : 0 times
 132435       : No old process
 132435       : last update: never
 132435       : Detect registered/public IP
 132435       : #> /usr/bin/nslookup eth10.aaa.com  >/var/run/ddns/myddns_ipv4.dat 2>/var/run/ddns/myddns_ipv4.err
 132435       : Registered IP '11.111.111.11' detected
 132435  info : Starting main loop at 2020-04-29 13:24
 132435       : Detect local IP on 'interface'
 132435       : #> ip -o addr show dev pppoe-wan10 scope global >/var/run/ddns/myddns_ipv4.dat 2>/var/run/ddns/myddns_ipv4.err
 132435       : Local IP '22.22.222.222' detected on interface 'pppoe-wan10'
 132435       : Update needed - L: '22.22.222.222' <> R: '11.111.111.11'
 132435       : parsing script '/usr/lib/ddns/update_cloudflare_com_v4.sh'
 132435       : #> /usr/bin/curl -RsS -o /var/run/ddns/myddns_ipv4.dat --stderr /var/run/ddns/myddns_ipv4.err --insecure --noproxy '*' --header 'X-Auth-Email: [email protected]'  --header 'X-Auth-Key: ***PW***'  --header 'Content-Type: application/json'  --request GET 'https://api.cloudflare.com/client/v4/zones?name=aaa.com'
 132438       : #> /usr/bin/curl -RsS -o /var/run/ddns/myddns_ipv4.dat --stderr /var/run/ddns/myddns_ipv4.err --insecure --noproxy '*' --header 'X-Auth-Email: [email protected]'  --header 'X-Auth-Key: ***PW***'  --header 'Content-Type: application/json'  --request GET 'https://api.cloudflare.com/client/v4/zones/e2342342345466e8f1343245c9d6/dns_records?name=eth10.aaa.com&type=A'
 132442  WARN : CloudFlare reported an error:
 132442       : {
  "result": [
    {
      "id": "eb234234321423543523453466558",
      "zone_id": "e2342342345466e8f1343245c9d6",
      "zone_name": "aaa.com",
      "name": "eth10.aaa.com",
      "type": "A",
      "content": "11.111.111.11",
      "proxiable": true,
      "proxied": false,
      "ttl": 120,
      "locked": false,
      "meta": {
        "auto_added": false,
        "managed_by_apps": false,
        "managed_by_argo_tunnel": false,
        "source": "primary"
      },
      "created_on": "2020-04-25T22:21:08.698187Z",
      "modified_on": "2020-04-25T22:21:08.698187Z"
    }
  ],
  "success": true,
  "errors": [],
  "messages": [],
  "result_info": {
    "page": 1,
    "per_page": 20,
    "count": 1,
    "total_count": 1,
    "total_pages": 1
  }
}
 132442 ERROR : IP update not accepted by DDNS Provider
 132442       : Waiting 300 seconds (Check Interval)
 135228       : ************ ************** ************** **************

after i run the same curl command in shell, and check the error string in the src file.
i find that some blank space inserted on the json string.

example:

https://github.com/openwrt/packages/blob/9c95fedca3d22b01c28dda7c72e70dfec9ca0f2e/net/ddns-scripts/files/update_cloudflare_com_v4.sh#L88
the origin line is:

    grep -q '"success":true' $DATFILE || {

and i fixed it to similar as the https://github.com/openwrt/packages/commit/0080bb4bba61171dd2b0654ce36359e5191db01a

    grep -q '"success":\s*true' $DATFILE || {

and then it tell me :

 135232  WARN : Could not detect 'record id' for host.domain.tld: 'eth10.aaa.com'
 135232 ERROR : No update send to DDNS Provider
 135232       : Waiting 300 seconds (Check Interval)
 135436  note : PID '31640' terminated by 'SIGTERM' at 2020-04-29 13:54

so i add the \s* to all the reg on grep command that process this json :
https://github.com/openwrt/packages/blob/9c95fedca3d22b01c28dda7c72e70dfec9ca0f2e/net/ddns-scripts/files/update_cloudflare_com_v4.sh#L136

__ZONEID=$(grep -o '"id":\s*"[^"]*' $DATFILE | grep -o '[^"]*$' | head -1)

https://github.com/openwrt/packages/blob/9c95fedca3d22b01c28dda7c72e70dfec9ca0f2e/net/ddns-scripts/files/update_cloudflare_com_v4.sh#L146

__RECID=$(grep -o '"id":\s*"[^"]*' $DATFILE | grep -o '[^"]*$' | head -1)

https://github.com/openwrt/packages/blob/9c95fedca3d22b01c28dda7c72e70dfec9ca0f2e/net/ddns-scripts/files/update_cloudflare_com_v4.sh#L153

__DATA=$(grep -o '"content":\s*"[^"]*' $DATFILE | grep -o '[^"]*$' | head -1)

https://github.com/openwrt/packages/blob/9c95fedca3d22b01c28dda7c72e70dfec9ca0f2e/net/ddns-scripts/files/update_cloudflare_com_v4.sh#L181

__PROXIED=$(grep -o '"proxied":\s*[^",]*' $DATFILE | grep -o '[^:]*$')

and then it work well like before.

[lantis1008]

Thank you, i will send a patch. I was not seeing the same behaviour the other day, but i am now. I must have cached previous good results.

[lantis1008]

Nevermind, this has already been taken care of by https://github.com/openwrt/packages/pull/11989. Yet to be merged.
Thanks @cherrot

[Lyoko-Jeremie]

good job !
thanks to you all ~~
@lantis1008 @cherrot

[Lyoko-Jeremie]

and BTW, i seems that the patch only allow it have 1 space. Could there be more than one ?

[lantis1008]

The patch covers space or no space.
It would be unexpected for there to be multiple spaces, but not outside the realms of possibility. I think very unlikely, but cross that bridge later.

[cherrot]

I could replace \? with \s* :)

Consider the JSON response is always generated by a library, in either pretty format or a compact format. So we could suppose that stricting to 1 space is kind of enouph? 😄

[Lyoko-Jeremie]

i think that they may change the lib or only change the lib params to change the JSON format .
so i think allow multi space will work well.

and nowsdays, the first request response a non-format json, i think it may changed to format in some day future. So, why not change all the reg to allow multi space in json string ? 🤣

[neheb]

https://github.com/openwrt/packages/pull/11989 was merged. Closing.