wrangler tail output modes

[toinbis]

💡 Feature request

Overview and problem statement

Current ouput of wranlger tail is useful for machine-parsing, but is (very close to) being unusable for simple logging needs for the development process. Introducing output modes would make it convenient to have logging in cases where we need to actually have the worker running on final domain instead of localhost.

Basic example

$ wrangler tail --output OUTPUTMODE

If 'simple' logging mode would just print console.log statements as it does in wrangler dev mode:

$ wrangler tail  --output simple
<... initialization messages ...>
[timestamp] This is first console.log output.
[timestamp] This is second console.log output.
[timestamp] Some simple exception message.

Thanks for considering!

[jahands]

What's wrong with piping to jq/rq/etc?

[jahands]

Here's simple log output using jq:

wrangler tail | jq -r '.logs[] | .message[] as $m | "[\(.timestamp/1000|todate)[\(.level)] \($m)"'

This converts this:

"logs": [
    {
        "level": "log",
        "message": [
            "persistenceCounter: 318 | reqCounter: 24",
            "another log message"
        ],
        "timestamp": 1591469995112
    }
]

to this:

[2020-06-06T18:59:55Z][log] persistenceCounter: 318 | reqCounter: 24
[2020-06-06T18:59:55Z][log] another log message

edit: simplified jq query

[toinbis]

Thanks a lot for taking time to respond and especially crafting the command itself @jahands !

The only thing what's wrong with it - it's not apparently obvious. For one - you need to know jq as such exist. Adding your suggested snipped to docs would do the job I suppose!

As for the ticket itself - I still think its a minor shortcomming when it comes to developer experience, so I tend to leave it open for wrangler team to at least consider it.

[jahands]

You're welcome! This was my first time using jq queries, I'm impressed with what can be done with it 🙂 I agree that adding jq examples to the docs would be a good idea.

I think the issue with having built in output modes other than JSON is that while you just wanted timestamps and log messages, someone else might want the connecting IP added, or perhaps certain headers that relate to their app, or maybe only error log messages.

If there is interest from the maintainers I could submit a PR with a page on parsing tail with jq

[jahands]

Here's another example adding the IP address to the output. I also split it into multiple lines to make it clearer how it works:

wrangler tail | jq -r \
       '# loop through all logs while still having the full object accessible
    .logs[] as $o |
        # loop through messages in each log
    $o.message[] as $m |
        # set variables for clarity (you can put these in the format string instead)
    .event.request.headers."cf-connecting-ip" as $ip |
    ($o.timestamp / 1000 | todate) as $datetime |
        # Format the output
    "[\($datetime)][\($o.level)][\($ip)] \($m)"'

edit: simplified a bit

[jahands]

@toinbis This jq example is now in the official docs at https://developers.cloudflare.com/workers/tooling/wrangler/commands/#tail

[EverlastingBugstopper]

Using jq for this is great but I agree with @toinbis that we could probably provide a built-in prettifier. Won't be super high priority for us since you can parse things with jq how you want, but we'll leave the issue open since I think it's a super valid feature request.

[jahands]

@EverlastingBugstopper what do you think built in pretty tail output should look like? Should it be configurable?

[EverlastingBugstopper]

🤷 we'd have to make some design decisions for sure. not sure what it'd look like, though we'd probably want it to look pretty similar to what wrangler dev looks like

[toinbis]

Great discussion, perfect outcomes! Thanks @jahands , thanks @EverlastingBugstopper !

[simplenotezy]

How can I modify tail so I only get error messages? I am trying to figure out why I get an error 1011 from workers, but due to the amount of requests incoming to my worker, it's pretty difficult to filter error messages only

[simplenotezy]

Also, if I run: wrangler tail --port 8888 | jq -r '.logs[] | .message[] as $m | "[\(.timestamp/1000|todate)[\(.level)] \($m)"'

I get:

🦚  Setting up log streaming from Worker script "jwr-nuxt". Using ports 8888 and 8081.
⠁   This may take a few seconds...
Closing tail session...
Error: Could not extract tunnel url from cloudflared

[ispivey]

I think it's worth looking at how some other tools solve the problem of output modes. A few examples off the top of my head include:

• gcloud logging read supports a --format flag. That flag is used across many gcloud commands. The default format is something human-readable that's specific to the command. Other options include config, csv, default, diff, disable, flattened, get, json, list, multi, none, object, table, text, value, yaml. docs.
• kubectl supports a -o or --output flag across all its commands (this isn't logging specific). Supported options include wide, yaml, json, jsonpath={template}. docs.

It would be nice to eventually support the same output-formatting flags across many commands, although we should definitely start small. Just something to keep in mind when designing a solution.

For background, a huge reason we started out sending JSON to wrangler is to support additional formats – so this is something we'd love to see.

Finally, while designing this, we discussed supporting tab-separated ouptut as well, like:

NAME       URL                      REQUEST_ID                TIME_UTC                 LOG
my-worker  https://workers.works/cats  56e749284e44ed8f-SJC  2020-03-03 21:39:23.843  This is log message 1
my-worker  https://workers.works/cats  56e749284e44ed8f-SJC  2020-03-03 21:39:24.607  This is log message 2

[ispivey]

@simplenotezy would you mind opening a separate issue? I'd love to keep this one dedicated to designing a second human-readable logging format for wrangler tail, which was the original feature request. I'll take a look at the new issue once you open it.

[simplenotezy]

Sure thing @ispivey

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[jasikpark]

Does anyone know of a way to simply browse the logs easily? Or at least pretty print? Can jq handle that?

Including exceptions...