Nock: Strip sensitive data from nock's “back” function before saving to disk

[kevinburkeshyp]

We are using nock's back function to store and later replay external HTTP requests in our tests. We commit the stored fixtures to our git source so that our tests can be run in environments where they don't need to make network requests.

Nock does not store request headers, but it does store the URI of the request. Unfortunately, some of those HTTP requests contain sensitive data in the query string that we would rather not commit to source code.

Is there a way to remove certain parameters from the nockback stored request path, and have nock match a request even though some parameters are missing from the stored path? (some kind of ignoreParams=['userid', 'secret-token']) would be nice.

[pgte]

Nock doesn't interpret the query string (yet).
I suggest that you use path-filtering to solve this.

Any issue with that please reopen this.

[kevinburkeshyp]

thanks for the quick response!

I guess I am a little confused about how to use path-filtering with nock
back - there doesn't seem to be any post-capture hook into the response. I
could load the file from disk and strip it there, but that seems cumbersome.

-kevin

On Apr 17, 2015, at 01:45, Pedro Teixeira [email protected] wrote:

Nock doesn't interpret the query string (yet).
I suggest that you use path-filtering
https://github.com/pgte/nock#path-filtering to solve this.

Any issue with that please reopen this.

—
Reply to this email directly or view it on GitHub
https://github.com/pgte/nock/issues/300#issuecomment-93945222.

[pgte]

Ha, sorry, you mean you don't want to record some data, hmmm...

Any ideas @kelaban ?

[kelaban]

@pgte, we can expose some more hooks such as afterRecord which can pass the recorded nock back before writing it to disk. We can go the simple route and let the caller be responsible for doing whatever they want to that output object or abstract away some of the details

the hook would probably called here

[pgte]

@kelaban want to propose an API?

[kelaban]

The easiest would be something like this.

function afterRecord(nockedObjects) {
  nockedObjects.forEach(nocked) {
    nocked.path = desensitizeQueryStr(nocked.path);
 }

  return nockedObjects;
}

nockBack('someFixture.json', {afterRecord: afterRecord}, function (nockDone) {

    doHTTPStuff(nockDone);

});

[lafave]

Exposing afterRecord would be great. Currently the only way that I know of to whitelist hosts w/ nockBack e.g. "don't record localhost" is to open the recording, parse it, delete the localhost recordings, and write it back to disk.

The same method could be applied to removing sensitive data, but I'm not sure that there's a way to get the requests to match when using the modified recording:

"path": "http://example.com:80/resources/1?secret-token=1234" <-- this should not be committed to version control.

"path": "http://example.com:80/resources/1?secret-token=REDACTED" <-- this could be committed to version control, but the paths would then not match when using the recording.

Are there any examples of path + request body filtering with nockBack?

[kelaban]

Look here under nock back usage and options. See the pre/post processing options you have. The scope object you get passed in the before/after callbacks is one of those

[kevinburkeshyp]

Hi @kelaban, I am pretty sure I tried this with the latest nock version and didn't see anything like those properties attached to the before function call. I didn't set up a reduced test case, however.

[kelaban]

Please test it (assuming you're loading a fixture which has already been recorded). Looking at the code it iterates through each scope and calls your handler which each scope object

[kevinburkeshyp]

On Sun, Apr 19, 2015 at 5:50 PM, Keith Laban [email protected]
wrote:

assuming you're loading a fixture which has already been recorded

This might be the key assumption. So, if the fixtures are already on disk,
we have access to the scope and can strip params etc from them but the keys
are already visible on disk in the test fixture. We commit these to source
so Circle, travis etc can run without making third party network requests.

If they are not on disk, we load them from over the network and save them
to disk. During this step, we don't have access to the scope or any of the
parameters you linked to (this is what I found in my test). So the requests
get written to disk with all of the sensitive parameters we don't want.

We could edit the request files by hand to strip the keys (or write a tool
that does this after the files have been serialized, as Adam mentioned),
but this is cumbersome and error-prone.

kevin

[holic]

:+1:

[matt-oakes]

This would be extremely helpful to me. I'm testing my server, so the tests are making a request to the express server using the request library and then that makes requests to various services. When using nockback it will also record and playback the HTTP request made to the server I would like to test, which obviously means none of my code is ever actually run.

The method described by @kelaban would be enough to filter out the calls to localhost for example.

[amejiarosario]

In my fixture I have something like this:

"path": "/api/v3/account?timestamp=1513990042776&recvWindow=60000&signature=037025c1fbe97b03112cb31d57c49b9711fa1e6efbe4f7f26257869252b7a9",

The timestamp changes everytime, How do I make it ignore the queries with afterRecord? If I remove them, then would it still match the path even if is partial?

I did something like this:

function afterRecord(nockedObjects) {
  nockedObjects.forEach(nocked => {
    nocked.path = nocked.path.split('?')[0];
  });

  return nockedObjects;
}

and it worked for removing the queries but then I get Error: Nock: No match for request. How do I solve that? It looks like filteringPath could help but not sure how to use it with nockBack

Tried this:

function beforeFunction(scope) {
  scope.filteringPath = function (path) {
    console.log(`filtering the path... ${path}`);
    return path.split('?')[0];
  };
}

and didn't work either, still getting the Error: Nock: No match for request

[lock[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue and add a reference to this one if it’s related. Thank you!