Espeasy: Handle the way log is presented in the web GUI, now it's hard keep up reading/copying.

[mattlward]

NOTE: This is not a support forum! For questions and support go here: https://www.letscontrolit.com/forum/viewforum.php?f=1

Steps to reproduce

How can we trigger this problem?
View logs and try to copy a line or 2 of the active log window.

Does the problem presist after powering off and on? (just resetting isnt enough sometimes)
Yes

Expected behavior

Highlight a line of the log and hit ctrl-c or right click copy.

Actual behavior

Log scrolls and unhighlights the line or lines that were selected.

System configuration

Hardware:
Wemos D1
Software or git version:

  | v2.0-20180213
-- | --

[Budman1758]

While your at it it could be a bit larger too. As in more lines displayed.

[TD-er]

@Budman1758 The Web log already takes up quite a lot of RAM.
Maybe we should not make that buffer bigger, but make the logging happen in the webpage itself (in the browser, using JavaScript)

I was also thinking about changing the log filtering. Maybe also an idea to add such filtering settings to the browser page itself. Then you can dynamically change what is sent to the log.

And there again is something that could benefit from a JSON interface ;)

[papperone]

I woudl rather remove the log from the web-interface if this will save a substancial amount of RAM!
then rather we can use remote tools to analize it (using Syslog server option) or have a flag to set ON/OF possibility to dump the log to a file in SPIFF (but maybe this will imply faster worn out of flash)

[TD-er]

The log buffer is now about 20*80 bytes (or more chars per line)
Using a syslog server is better, but that's harder to use for a lot of users.
If we could stream it to the browser, then you'll have less memory requirements in the ESP.

Saving to SPIFFS is a bad idea.

[TD-er]

I have been thinking about this issue (and logging in general).

I think we should:

• Retrieve web-log through JSON API call, in which you query the log topic and get all logs since the last query (this limits to 1 active viewer via web)
• Add JavaScript viewer which will store up-to N lines of log data (stored in browser memory)
• Add topic selector to log viewer, which will filter what will be sent to this log feed (N.B. topic will be sent through the API call to the ESP)
• Returned data will also indicate how much the buffer on the ESP was filled, to indicate the update frequency to the JS viewer.

This allows us to:

• Filter per topic (one topic per task(plugin) / Controller / protocol + detail level: info / error / debug)
• Add log window on any page with default filtering on the topics related to that page (e.g. MQTT OpenHab data when controller page with that controller is active)

Such topics should be introduced in any log output and be dynamically selectable. Only difference is to the syslog server (and SD?), where these settings should be stored since syslog (and SD) are not interactive.

[uzi18]

Maybe we can log it to spiffs.... and free some RAM.

15.02.2018 9:28 AM "Gijs Noorlander" [e-mail] napisał(a):

I have been thinking about this issue (and logging in general).

I think we should:

• Retrieve web-log through JSON API call, in which you query the log
  topic and get all logs since the last query (this limits to 1 active viewer
  via web)
• Add JavaScript viewer which will store up-to N lines of log data
  (stored in browser memory)
• Add topic selector to log viewer, which will filter what will be
  sent to this log feed (N.B. topic will be sent through the API call to the
  ESP)
• Returned data will also indicate how much the buffer on the ESP was
  filled, to indicate the update frequency to the JS viewer.

This allows us to:

• Filter per topic (one topic per task(plugin) / Controller / protocol
• detail level: info / error / debug)
• Add log window on any page with default filtering on the topics
  related to that page (e.g. MQTT OpenHab data when controller page with that
  controller is active)

Such topics should be introduced in any log output and be dynamically
selectable. Only difference is to the syslog server (and SD?), where these
settings should be stored since syslog (and SD) are not interactive.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/letscontrolit/ESPEasy/issues/867#issuecomment-365856659,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAHOU7CZu_rpvJ4obqbx6U6rnA0NyyU-ks5tU-q3gaJpZM4SEnAd
.

[TD-er]

No, do not log to spiffs, that's surely not a good idea for several reasons.

[uzi18]

If we can bit select what to log there it could be helpful.

[TD-er]

For when it isn't online (no syslog server) you mean?
But do not log to SPIFF to save RAM. It will wear out the flash in no-time and you will hit the 100 flashes limit per day in the matter of minutes if you're not careful.

[uzi18]

Another idea is to provide circular buffer for log and ajax /log.json,
every request we need to store last offset to know how many data to send.
On webpage request every 0,5s-1s data and append to Textview if needed.
It is best idea for now and the future.

[TD-er]

That was exactly my idea to do.
Now the log buffer is split into lines, but most of these lines will not exceed the max. line length, so there is a lot of waste.

[uzi18]

Sorry did not read Your big post. But this solution works great in one of my project.

[TD-er]

I was just thinking, given the idea to split logs into topics, we could also log to MQTT.
Is that something that's already been done anywhere?

[Grovkillen]

Log to mqtt is something I'd love!

[Grovkillen]

The new web log is taking care of this issue.