Firebase-tools: Disable terminal logs for firebase serve command

[GorvGoyl]

Whenever I run firebase serve and loads the firebase site locally, my VSCode terminal fills with lengthy and unnecessary logs like this.

127.0.0.1 - - [29/Nov/2019:21:10:23 +0000] "GET /security HTTP/1.1" 200 10208 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36"
127.0.0.1 - - [29/Nov/2019:21:10:24 +0000] "GET /assets/css/Features-Boxed.css HTTP/1.1" 200 1075 "http://localhost:5000/security" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36"
127.0.0.1 - - [29/Nov/2019:21:10:24 +0000] "GET /assets/css/style.css HTTP/1.1" 200 12392 "http://localhost:5000/security" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36"
127.0.0.1 - - [29/Nov/2019:21:10:24 +0000] "GET /__/firebase/init.js HTTP/1.1" 200 - "http://localhost:5000/security" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36"
127.0.0.1 - - [29/Nov/2019:21:10:24 +0000] "GET /assets/img/svg/home_white.svg HTTP/1.1" 200 1305 "http://localhost:5000/security" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36"
127.0.0.1 - - [29/Nov/2019:21:10:24 +0000] "GET /__/firebase/7.5.0/firebase-app.js HTTP/1.1" 200 12066 "http://localhost:5000/security" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36"
127.0.0.1 - - [29/Nov/2019:21:10:25 +0000] "GET /__/firebase/7.5.0/firebase-analytics.js HTTP/1.1" 200 21623 "http://localhost:5000/security" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36"
127.0.0.1 - - [29/Nov/2019:21:10:25 +0000] "GET /__/firebase/7.5.0/firebase-auth.js HTTP/1.1" 200 163643 "http://localhost:5000/security" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36"
127.0.0.1 - - [29/Nov/2019:21:10:25 +0000] "GET /assets/js/main.js HTTP/1.1" 200 5794 "http://localhost:5000/security" "Mozilla/5.0 (Windows 
NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36"
127.0.0.1 - - [29/Nov/2019:21:10:25 +0000] "GET /__/firebase/7.5.0/firebase-app.js.map HTTP/1.1" 200 70975 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36"
127.0.0.1 - - [29/Nov/2019:21:10:25 +0000] "GET /__/firebase/7.5.0/firebase-analytics.js.map HTTP/1.1" 200 113064 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36"
127.0.0.1 - - [29/Nov/2019:21:10:25 +0000] "GET /assets/css/style.css.map HTTP/1.1" 200 32580 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) 
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36"

I usually put some console.log statements here and that get lost in all this mess. How to disable it?

Terminal: VSCode built-in Terminal
Firebase: 7.8.1
Project: Firebase web app
Platform: Windows

[samtstern]

@mbleigh I have had this same thought as well but these logs are from way before my time.

1. Where do they come from? I assume superstatic?
2. Would it be more appropriate for them to be hidden under --debug? Do you think people are relying on this level of verbosity?

[bkendall]

I do believe these come from superstatic, though I'd have to go digging to confirm.

[ultraGentle]

I, too, use VSCode and experience the same issue with console.log getting lost among the terminal verbosity.

Beginner question: is there a way to view the logs in Chrome DevTools instead of the terminal?

[ultraGentle]

A suggestion: a helpful temporary measure would be to add a bunch of whitespace between the firebase serve logs and any console.log printouts from the code.

My current workaround to make the logs easier to read is to add "*******" before all of my logs, so they pop out at a glance against the field of extraneous material.

(This is clearly not ideal, but some similar kind of visual demarcation would be helpful.)

[ultraGentle]

A related issue (for me, at least) is that the firebase serve lines wrap, thereby taking up even more space and often requiring a lot of scrolling to actually find my own logs.

I don't know if line wrap can be turned off from within the firebase CLI -- I think not; it's terminal-specific?

Finally, not sure if it's too off-topic for here, but I noted in #1859 that some iconography would help the user's own logs pop out (you can see an example there).

[DCtheTall]

@samtstern what about having a Boolean flag that defaults to true? I know people (e.g. me) do use these logs to monitor requests to local servers, so maybe turning them off by default isn't ideal?

[samtstern]

@DCtheTall thanks for chiming in! I assume you're not the only one who relies on these long-standing logs.

Maybe a new --quiet flag that's off by default?

[DCtheTall]

@samtstern that makes sense. Would it suppress logs from all of the services? Just hosting/functions?

[samtstern]

@DCtheTall thanks for the offer! Because this is an API change we'd need to first agree on the API here and then push it through our internal "API review process" before we could accept your PR. Although once we do all that, I would be happy to review your contribution.

Those are good questions you raise and --quiet is probably not the right API for that reason. I think most of the other emulators right now don't make too much noise. The issue here just seems to be that the logs from superstatic dominate.

[DCtheTall]

That makes sense. Perhaps a flag that quiets just the hosting logs (drawing a blank on a good name)? It could just control the Boolean field here:

https://github.com/firebase/firebase-tools/blob/15a582cd2d92ad7477e9364a61bb17a98478fe8e/src/serve/hosting.js#L21

[ultraGentle]

The hosting logs aren't the only ones that it would be useful to suppress. Functions also generate logs as follows:

127.0.0.1 - - [16/Dec/2019:19:01:49 +0000] "GET /__/firebase/7.5" ...
127.0.0.1 - - [16/Dec/2019:19:01:49 +0000] "GET /__/firebase/7.5" ...
127.0.0.1 - - [16/Dec/2019:19:01:49 +0000] "GET /__/firebase/7.5" ...
127.0.0.1 - - [16/Dec/2019:19:01:50 +0000] "GET /__/firebase/7.5" ...
i  functions: Beginning execution of "func1"
i  functions: Finished "func1" in ~1s
i  functions: Beginning execution of "func2"
i  functions: Finished "func2" in ~1s

Which can become quite lengthy if there are many functions.

Provided there are no errors in function execution, I usually just want to see my own console.log printouts, without intervening logs from either hosting or functions.

For instance, while developing a web app, I'll have it wired up through Browsersync, so that any change results in hot reloading, and re-generates any logs I have.

But when there are many functions logs, these can make it difficult to compare the new set of logs to the previous, since they appear in between. They can also scroll my old logs right out of view.

I hope that's clearly described enough. Basically, it would just be nice to have a way to optionally suppress all logging except for in-code console.log (and it's ilk), for a debugging experience that can be more easily tailored to the situation.

[ultraGentle]

Or, approaching this from the opposite direction, in addition to choosing which logs to view in the terminal,

When using hosting, maybe the developer's own console.log statements in functions/index.js could be output to the browser devtools console at localhost:5000 as well, and the firebase-tools logs would stay in the terminal.

This would neatly separate out manually inserted logs from default firebase-tools logs, and have the advantage of keeping both front- and back-end logs in the same place.

(That probably doesn't seem like a plus to those on teams big enough to have a separation of front- and back-end roles, but since Firebase makes it so easy for front-end devs to add back-end functionality to their projects, I wouldn't be surprised if there are quite a few users who would benefit!)

It's a slightly different use case, but overlapping enough that I thought it was worth mentioning here.

[ultraGentle]

Happy coincidence alert: Firebase tools 7.11.0 introduces the ability view logs and debug from browser devtools.

The reason that's relevant to this issue is that devtools doesn't show the serve logs or the execution took x seconds logs.

It's not a 100% overlap in functionality, but it may remove the need for this feature being baked into the CLI.

[DCtheTall]

@samstern do you think this can be closed then? Do you think adding the option to quiet the superstatic server is worth the internal API review?

[filipesilva]

I'm trying to use the emulator in a e2e environment and find the hosting emulator to log a lot of messages, making reading the CI logs hard.

For instance, if you have tests and run them using `firebase emulators:exec \"npm test\"", your test logs will get lost in the middle of the hosting server logs.

So for CI workflows (which are listed in https://firebase.google.com/docs/emulator-suite/install_and_configure) I think it's still worth it to quiet the server somehow.

[hariprasadiit]

We can use grep -v as a temporary workaround when you start hosting emulator.

superstatic server logs 127.0.0.1 - - before all lines. we can filter those lines to see only emulator logs

firebase emulators:start --only functions,pubsub,hosting | grep -v "127.0.0.1 - -"

[ultraGentle]

@hariprasadiit That's clever -- thanks! To explore some implications from that idea:

Since everyone visiting this thread wants to filter for different combinations of logs, perhaps each type of log could simply be prefixed with one or more text tags, to allow for targeted filtering via grep.

For instance, I often want to see only my own console.log statements from index.js, so I grep -v -E "127|hosting|functions|Google|-". Technically, this achieves the desired output.

But instead of -v (to invert) and -E (to allow the | multiple expressions), it would have been easier to simply grep "console." — _if_ function logs were text-tagged console.log : your-log-here.

This prefixed tagging is already present in the form of hosting: [hosting message], functions: [functions message]. But one could expand upon it by adding a tag emulator or firebase-tools to distinguish messages that come from the tools vs. come from the actual code.

Additionally, in the case of logs generated from in-app code, I'd propose prefixing these with console.log :, console.warn :, console.error :to allow in-terminal filtering for any/all of those via grep.

I'd imagine other folks' use cases could be similarly addressed by relevant text-tagging.

Finally, perhaps it would be easier for the firebase-tools team to add a bit of identifying text than to actually add behaviors to the emulator.

Anyway, thanks for the grep suggestion!

[samtstern]

I really appreciate the continued discussion here! What I am learning is that no two people want the same logs, or have the same idea for what logs should be exposed in "quiet" mode vs "loud" mode. Some people absolutely want those Hosting logs all the time, others never want them.

@ultraGentle I do think you're right that in the end we need some sort of total logging overhaul but that would be a really huge undertaking throughout the entire code base, not sure I want to dig into that right now.

So I think we should put this on the backburner for the moment, I think most people can grep there way to where they want to be.

[ultraGentle]

Just to clarify, I was suggesting that _instead_ of an overhaul, just adding the text console to the output before user logs would allow people to grep for it.

As it is, the user's own console.log statements are the only ones that _aren't_ preceded by function: or hosting: or whatever, making them awkward to grep for.

[malikasinger1]

That makes sense. Perhaps a flag that quiets just the hosting logs (drawing a blank on a good name)? It could just control the Boolean field here:

https://github.com/firebase/firebase-tools/blob/15a582cd2d92ad7477e9364a61bb17a98478fe8e/src/serve/hosting.js#L21

how the hell I can disable this shitty logs brust

[malikasinger1]

google if you want us to leave google cloud just let us know, dont do these shitty things to disturb our ongoing projects

[abeisgoat]

So this is largely fixed by the new UI for the Emulator Suite when you open the UI in your browser, you can go to the logs tab and simply click on "functions" and it will filter to only logs from your function emulator, no hosting or otherwise. We're still fleshing out the query language, but the core complaint here "I can't find my function logs" is definitely addressed by using the in-browser log viewer.

Hosting logs are also now prefixed with "hosting" so @hariprasadiit workaround is now even simpler firebase emulators:start --only functions,pubsub,hosting | grep -v "hosting:".

In terms of the everything @ultraGentle suggested, that's all possible through the logs UI in the emulator UI. I'm inclined to say this is functionally fixed. If there are queries that people still want to do to their emulator logs, it's probably better to extend the logging query language in the UI then attempt to add total control over these currently verbose terminal logs.

[abraham]

@abeisgoat I don't think that resolves this ticket. The Emulator Suite UI is helpful for filtering logs for manual viewing but there isn't a way to silence the noisy Firebase logs in the command line.

[abeisgoat]

So I think our general philosophy is that in the terminal we prefer verbosity and allow filtering to be done in the UI. If, as per the original request, we added the ability to quiet down the Hosting emulator logs, what would you see after running serve? Would you like it to just be totally silent?

[filipesilva]

@abeisgoat on serve it wouldn't make much of a difference to me what shows up, since the separate process makes it easy to pipe the output somewhere.

On firebase emulators:exec I'd very much like to silence or redirect all post-initialization logging from firebase. That way I can read CI logs and see test errors without many lines from the emulator.

[samtstern]

@filipesilva you have a very valid point here and I think saying that emulators:exec quiets all non-user debug logs by default is a good idea. How does that sound to you?

[filipesilva]

SGTM, if I need the firebase logs I could just run it again with serve.

[sesam]

We've tried to work around this limitation, and it's kind-of working. We do:

1. redirect to take control over STDERR and STDOUT
2. use sed -e ... to prepend log lines with where they came from.
3. filter things. This is where it gets problematic.

We'd want to get STDERR outputs both as they happen (during manual debugging and to break a long pipeline manually if it's obviously borked), and at the end in a separate clearly marked block, as otherwise it's to easy to overlook some STDERR outputs.

We'd also like to have more granularity on outputs from emulators, as we're currently at a loss how we can filter that properly.
Maybe we can start each emulator as a separate process and pipe each of those separately. This will be our next step, as that would enable us moving a lot of firebase trigger outputs elsewhere. As is, the trigger log outputs break in in the middle of other outputs, even in the middle of a log line from our pipeline tests, which makes them very hard to read.