Cdnjs: Map files are not cached at CDN

[miyurusankalpa]

Map files are served from the backend and is not cached at Cloudflare.

Test URLs:
https://cdnjs.cloudflare.com/ajax/libs/jquery/3.2.1/jquery.min.map
https://cdnjs.cloudflare.com/ajax/libs/twitter-bootstrap/3.3.7/css/bootstrap.min.css.map

RedBot:
https://redbot.org/?uri=https%3A%2F%2Fcdnjs.cloudflare.com%2Fajax%2Flibs%2Fjquery%2F3.2.1%2Fjquery.min.map

---

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

[PeterDaveHello]

@terinjokes is it possible to cache map file here? Thanks.

cc @ryankirkman

[MattIPv4]

@PeterDaveHello How would I go about checking if these files are still not cached?

[miyurusankalpa]

@MattIPv4 you can look for the cloudflare cache status header. if it is not presented, the resource is served from the backend.

For me, I can also tell it by comparing load times with browser cache disabled.

[MattIPv4]

Cloudflare will be looking into this from Monday onwards. I will post updates I get here.

[simon-says]

@MattIPv4 in order to do this we want to make a change to have Cloudflare attempt to cache all URLs served from under cdnjs.cloudflare.com - e.g. requests matching the pattern:

http(s)://cdnjs.cloudflare.com/*

Can you confirm all responses from the origin are considered static and therefore are safe for us to attempt caching on? The cache key we use for this is:

{scheme}://${host_header}${uri_iqs}

Which means for any request for https://cdnjs.cloudflare.com/example.map?blah=123 including all variations of the query string we'll store a single cached asset under the key https://cdnjs.cloudflare.com/example.map. This same logic will apply to all requests under http(s)://cdnjs.cloudflare.com/*

[MattIPv4]

Hi @simon-says, I think this will be an issue for the package.json files available under each library as these are not static and get updated with each new version.

Every file within a library version is static however.

Eg. https://cdnjs.cloudflare.com/ajax/libs/jquery/3.4.1/* is static but https://cdnjs.cloudflare.com/ajax/libs/jquery/package.json is not, as the version number is updated with each release and other values such as the autoupdate config may be changed.

cc @PeterDaveHello / @ryankirkman can you confirm I'm correct with the above?

[PeterDaveHello]

@MattIPv4 you're right.

@simon-says I wonder if we can ignore/not serve "ajax/libs/*/package.json"?

[MattIPv4]

@PeterDaveHello I'm concerned that some users may now make use of these are we serve them and so making a change to no longer serving them may pose an issue. Ignoring them from the cache map would be the best solution I think.

_The only reason they are served now is that we added .json to the file extension whitelist for libraries that used JSON data files and a side effect was that the package.json files also become available._

[PeterDaveHello]

@MattIPv4 yes, I can understand your concern, maybe it'll be great to set a short cache TTL for them ;)

[simon-says]

So are you wanting to cache ajax/libs/*/package.json but with a shorter edge cache TTL? How short?

And then outside of that pattern, the current caching config we apply to .js files is safe to apply to http(s)://cdnjs.cloudflare.com/*?

[MattIPv4]

Hi Simon,

As far as I know, the config for .js can be applied to everything else :)
As for the package.json files, I'll leave that to @PeterDaveHello as I'm not sure what the best TTL would be.

[PeterDaveHello]

@simon-says Yes, what do you think about 600 seconds for them? I'm also fine with 300 seconds or not caching them.

[MattIPv4]

@simon-says would it be sensible to work on https://github.com/cdnjs/cdnjs/issues/8655 at the same time when you deploy the change to the cache config to resolve this issue? :)

[SukkaW]

Is the issue being continued?

curl https://cdnjs.cloudflare.com/ajax/libs/jquery/3.2.1/jquery.min.map -v

< HTTP/1.1 200 OK
< Content-Type: application/octet-stream
< Content-Length: 131666
< Last-Modified: Tue, 14 Jul 2020 19:58:41 GMT
< Vary: Accept-Encoding
< Timing-Allow-Origin: *
< Access-Control-Allow-Origin: *
< Cache-Control: public, max-age=30672000
< Served-In-Seconds: 0.001
< Accept-Ranges: bytes
< CF-Cache-Status: DYNAMIC
< Server: cloudflare

[Redacted]

cache-control header is set, while Cloudflare still shows CF-Cache-Status: DYNAMIC

[MattIPv4]

@tc80 @xtuc We should get this and some of the other open CDN config issues here sorted once we've got KV into prod fully.

[runlow]

Not sure if related (expected both to give a 200 status code):

$ curl -s --head https://cdnjs.cloudflare.com/ajax/libs/twitter-bootstrap/4.5.0/css/bootstrap.min.css.map | head -n1
HTTP/2 200
$ curl -s --head https://cdnjs.cloudflare.com/ajax/libs/twitter-bootstrap/4.5.2/css/bootstrap.min.css.map | head -n1
HTTP/2 404

[tc80]

@runlow in your example, twitter-bootstrap/4.5.2/css/bootstrap.min.css.map simply does not exist in cdnjs (returning 404), which is interesting! We don't autogenerate .map currently, but we will try to in the future.

The issue here is that the .map files that are in cdnjs (returning 200) are not being cached properly by Cloudflare.

To view cache headers:

curl https://cdnjs.cloudflare.com/ajax/libs/twitter-bootstrap/4.5.0/css/bootstrap.min.css.map -I |& grep -i cache

cache-control: public, max-age=30672000
cf-cache-status: DYNAMIC