Got: Enhance the `cache` option

[szmarczak]

The cache mechanism:

• [x] should be a promise
• [x] should be replaceable with any other module*
• [ ] should have something hook-like to allow storing different properties e.g. ip addresses
  
  
  
  • [ ] should accept different caching logic
  
  
  • [ ] should make a new request when the cached response cannot be updated (#963)
  
  
  • [ ] beforeCache hook
  
  

* Through the request option. Accepting ClientRequest and IncomingMessage as the return type. Return undefined to fallback. You can override the request option also by returning a ClientRequest or an IncomingMessage instance in a beforeRequest hook.

[sithmel]

About #746:
What about the approach of adding an hook that can override the headers (including the caching ones), before they are evaluated by cacheable-request ?
This way you can set different caching headers, tweaking the cache key (etag or even the url) etc.

[szmarczak]

@sithmel beforeCache hook? Sounds good to me.

[sithmel]

Happy to implement it as soon as this is done and merged

[sithmel]

@szmarczak I guess I should wait for https://github.com/sindresorhus/got/pull/1051 to be merged ... do you agree ?

[szmarczak]

@sithmel You don't have to. Feel free to send a PR anytime, I'll merge with master later. No worries! :)

[MrSwitch]

Can i add another suggest feature which is kind of related to being a promise #1078

[rifler]

i've just read ky docs https://www.npmjs.com/package/ky#hooksbeforerequest

The hook can return a Request to replace the outgoing request, or return a Response to completely avoid making an HTTP request. This can be used to mock a request, check an internal cache, etc

sounds quite similar for https://github.com/sindresorhus/got/issues/746 needs

[szmarczak]

@rifler It's already done. It hasn't been documented yet.

[szmarczak]

@rifler 779062a1b99a9a1c13737b9fd613e185d97b158b

[adityapatadia]

It would be great if you can also write how can we give custom request. It's still not clear from Readme.

[szmarczak]

@adityapatadia http.request returns a ClientRequest instance for example. I will write a more detailed tutorial soon.

[adityapatadia]

@adityapatadia http.request returns a ClientRequest instance for example. I will write a more detailed tutorial soon.

I understand the return type but I am not sure how can I return custom request function.

Do you mean something like this?

beforeRequest: [
            async options => {
                return new CacheableRequest(http.request);
            }
        ]

[szmarczak]

No. I mean ClientRequest, not CacheableRequest:

const got = require('got');
const https = require('https');

got('https://example.com', {
    hooks: {
        beforeRequest: [
            () => {
                // `https.request` returns a ClientRequest instance
                return https.request('https://httpbin.org/anything');
            }
        ]
    }
}).json();

[szmarczak]

You can pass an IncomingMessage instance as well. To generate one you can use for example https://github.com/lukechilds/responselike

[adityapatadia]

Understood. Thanks for help.

[szmarczak]

No problem, if you got any other questions, feel free to ask :)

[hk7math]

May I know if #746 is possible yet? When remote host returns headers with {"cache-control": "no-cache"}, is there a way to override it without adding beforeRequest and afterResponse hooks to implement custom caching? Thank you very much!

[adityapatadia]

No problem, if you got any other questions, feel free to ask :)

Sorry to get back after long time but I tried this and it throws many errors related to agent and timeout.

beforeRequest: [async options => {
        return new Promise((resolve, reject) => {
          let cr;
          if(options.url.protocol == "https:"){
            cr = new cacheableRequest(https.request);
          } else {
            cr = new cacheableRequest(http.request);
          }
          cr(options).on("request", resolve);
        });
      }],

Basically there is no easy way to just edit some part of cacheableRequest module and then plug it in this library.

[szmarczak]

@adityapatadia You're creating a new CacheableRequest instance on every got call. You can do this instead:

const runCachedHttps = new CacheableRequest(https.request);
const runCachedHttp = new CacheableRequest(http.request);

...

beforeRequest: [async options => {
        return new Promise((resolve, reject) => {
          const cr = options.url.protocol == "https:" ? runCachedHttps : runCachedHttp;
          const instance = cr(options);
          instance.on("request", resolve);
          instance.on("error", reject);
        });
      }],

...