Karate: Hide sensitive information from console and reports

[MercierAdrien]

As discussed here, there is no possibility to hide something from the console output if you want to keep the log level to DEBUG.

Background:
    * url "https://www.google.fr"

    Scenario: hide password
        Given path "login"
        And form field username = 'john'
        And form field password = 'secret'
        When method post
        Then status 200

Gives the following console output :

10:38:34.710 request:
1 > POST https://www.google.fr/login
1 > Accept-Encoding: gzip,deflate
1 > Connection: Keep-Alive
1 > Content-Length: 29
1 > Content-Type: application/x-www-form-urlencoded; charset=UTF-8
1 > Host: www.google.fr
1 > User-Agent: Apache-HttpClient/4.5.5 (Java/1.8.0_191)
username=john&password=secret

Using * configure report = { showLog: false } allows to exclude the whole request from the report, but it still appears in the console output.

Could it be possible that this command also prevents the console output? Or maybe a command indicating that the request that follows (or the whole scenario?) should no be printed on the console?

[ptrthomas]

moving to project roadmap board: https://github.com/intuit/karate/projects/3#card-22529225

[ptrthomas]

for others landing here and from #449 - please refer this solution on Stack Overflow (see comment): https://stackoverflow.com/a/56745485/143475

here's some sample code:

jsonPaths.forEach(path -> {
    Path fullPath = Paths.get(path);
    try {
        String content = new String(Files.readAllBytes(fullPath), charset);
        content = content.replaceAll("(api-key: )(\\w*)", "api-key: masked");
        Files.write(fullPath, content.getBytes(charset));
    } catch (IOException e) {
        e.printStackTrace();
    }
});

[aanutter]

Masking credentials in the console can done through the logback.xml:
<pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %replace(%msg){'(?&lt;=api-key:).*', ' xxxx'}%n</pattern>

[ptrthomas]

an update for anyone landing here, just figured it is very easy to hide all logs in the console by commenting out the STDOUT log appender as follows, refer https://stackoverflow.com/a/56937882/143475

<root level="warn">
    <!-- <appender-ref ref="STDOUT" /> -->
    <appender-ref ref="FILE" />
</root>

EDIT: just learnt there is another option using a ThresholdFilter:

    <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
        <filter class="ch.qos.logback.classic.filter.ThresholdFilter">
            <level>WARN</level>
        </filter>
        <encoder>
            <pattern>[%-5level] %d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %logger{36} @%L - %msg%n</pattern>
        </encoder>
    </appender>
    <logger name="com.intuit.karate" level="DEBUG"/>

this is great for teams who don't want too much console output especially when running CI jobs

and if you comment out the FILE appender you won't have anything in the karate.log file either - which may solve some teams with super paranoid security requirements.

so we are now left with values in the JSON files and HTML report which may need an elegant solution in the long term.

[bwaters2]

I helped @aanutter come up with a solution for this a few months back. I think a nice solution would be to pass in a list of values to mask with karate.configure and then mask then in the RequestLoggingInterceptor check for these values before logging headers.

I've already implemented this for headers here: commit with solution

For the request body it would be trickier because it is a String and not key-value pairs. The simplest solution would probably be to skip logging the entire request body when configured to do so.
That could be done in one line here with a change like:
if( request instanceof HttpEntityEnclosingRequest && config.getShouldLogRequestBodyBoolean()){then do logging stuff}

If you like this solution I can open up a polished PR in a week or two.

[ptrthomas]

@bwaters2 yeah this is an ask I'm getting from elsewhere as well. actually I was beginning to type out a couple of solutions but I just got this idea, thanks for the prompt

I propose that since this is such a horribly complex area, we don't attempt to solve it via config - but instead introduce an interceptor that can give you control before and after a request is made. it actually will solve some other random asks like "we need to retry requests" that I got from elsewhere. I'm leaning towards just adding a method to the ExecutionHook - also there are some other interesting use cases such as "rate-limiting" and "header-manipulation-on-steroids" that we can possibly solve in one shot.

what needs some thought is separation of the real data and the logged data, but should be quite doable. I'd like some time for this thought to "bake"

EDIT: why the interceptor makes sense IMO is because the user can finely control which request patterns need masking instead of us introducing a performance penalty across all requests

[ptrthomas]

for all watching this - we implemented a new configure logModifier approach. here is the doc: https://github.com/intuit/karate/tree/develop#log-masking

please provide feedback !

[soerenhenkel]

This sounds very promising. We have urgent need for this feature. Is there any chance for a 0.9.5.RC5 (or even 0.9.5) release, so that we can give it a try with a non-local build?

[ptrthomas]

@soerenhenkel okay, we can do an RC5 over the weekend. it will help if you try it out and see if you need API changes. the reason I'm holding is I'm wondering if this should be moved into the "hooks" that is explained here: https://github.com/intuit/karate/issues/970#issuecomment-557443551

[soerenhenkel]

@ptrthomas Thanks a lot. I'm looking forward to run tests with the log-masking API after RC5 is available, and will definitly provide feedback.
Today I had a look at the hooks things that you mentioned, and did some experiments with RC4. I was indeed able to manipulate the generated results, nevertheless I have the feeling that a seemless integration of log-masking into the hooks API might not be trivial. For example, I was surprised to find that the logged messages are not available afterStep() ("method post" in this case), but only afterScenario(). Also, I like the fact that the new HttpLogModifier does not require any knowldege about the internal structure of Scenario, ScenarioResult, Step, StepResult etc. I wonder how the same would be achieved with the hooks API?
(But don't get me wrong, I think that the hooks API is also a very useful achievement. I just feel that both APIs could coexist, because they address different needs.)

[ptrthomas]

@soerenhenkel good point on the simpler API. so we'll go with that.

[soerenhenkel]

@ptrthomas Thanks again for your support. I just tried the log masking feature in RC5. It does all we need, and I find the API clear and inuitive to use. No complaints, I like it :)

[ptrthomas]

@soerenhenkel great to hear ! thanks

[ptrthomas]

released 0.9.5