Julia: Base logging todo

[c42f]

Now that #24490 is merged containing the core functionality, there's still a lot of loose ends to tidy up, but at least they can be merged incrementally. Here's a list [edit - organized and sorted by priority]:

Important user-visible changes for 1.0

Core log message generation

• [x] Dispatch flisp syntax/lowering depwarns to logging system (PR: #25257)
• [ ] Dispatch C-level runtime warnings to logging system, arguably most importantly jl_binding_deprecation_warning (should fix any final parts of #19686 and #24028)
• [ ] Remove --depwarn=error and replace with something finer grained. Arguably blocked until the previous two points are resolved.

User experience / logger backends

• [x] More capable Logging.ConsoleLogger, with markdown-based pretty printing, and other formatting niceties which will allow SimpleLogger to be further simplified. See MicroLogging.InteractiveLogger for a start at this. See also #25106, and https://github.com/JuliaLang/julia/pull/24490#issuecomment-351977978 at least. This is less API changes, and more setting a strong precedent for the data people include in their log events when they use the system.
• [x] Standardize progress API or simply remove all reference to it from the documentation for now (see #8064, and comments starting at https://github.com/JuliaLang/Juleps/pull/30#issuecomment-301242856, also https://github.com/JunoLab/Juno.jl/issues/29)
• [x] Refine conventions for harvesting of backtraces (see https://github.com/JuliaLang/julia/pull/25370#discussion_r160065706 )
• [ ] Simple logger configuration ( #25404 )

Non-API breaking tweaks and bug fixes

• [x] depwarn frame sanitization (see #25130 and https://github.com/JuliaLang/julia/pull/24490#issuecomment-351939859)
• [x] User documentation (#25154)
• [x] Resolve naming conflict with @kmsquire's package ( See also #25182 and PR: https://github.com/JuliaLang/METADATA.jl/pull/12653 )
• [x] Resolve issues with deprecated tests ( See https://github.com/JuliaLang/julia/commit/5d77d1b8ddd4bbe340e087802bf928ac208e4229#r26334167 and PR #25239 / #25229 )

Non critical breaking changes

• [x] Make @show use the logging system so that shown variables can be captured as values rather than text when desired. (Main issue - which log level should this be? debug kinda makes sense, but info is arguably more useful for quick hacking.) Didn't have time to do this before 1.0.
• [x] Improve @test_logs usage for matching multiple log messages - nicer syntax + be slightly more conservative about features.
• [ ] Clean up @test_logs failure Test.Result type - improve the way test sets capture results.

Wildcards - advice needed

• [ ] Log forwarding from remote workers. In particular people using parallel workloads probably care about this. Pkg2 also uses separate processes extensively, and the tests clearly ended up a bit of a mess, with some logs being captured on stderr from a separate process.

[StefanKarpinski]

How much of this is user-visible? We should heavily prioritize user-visible changes since we're trying to do a feature freeze here and get an alpha out. Non-user-visible changes can continue to be made during the alpha period, although ideally those should be minimal, this seems important enough that we can allow it.

[c42f]

I've organized the list here into a rough priority. I think I've got a fairly solid handle on how to do most of these things because I've already done the bulk of the investigation work and a fair chunk of the code. As to priority, I've laid them out as I see them, roughly, but happy to rearrange a bit.

The main thing I feel quite unsure about at this point is the remote workers issue as I really haven't done any investigation here.

[StefanKarpinski]

Remove --depwarn=error and replace with something finer grained.

Note that we can always keep this and add a more fine-grained mechanism on top of it in the future and just redefine the behavior of this option in terms of the more fine-grained mechanism.

[c42f]

That's a good point, it's not strictly required that this be deprecated. My concern is that when implementing this with a logger, --depwarn=error is essentially a logger configuration mechanism and I suspect a command line option will compose badly with whatever logging configuration system we end up with. It can probably be hacked together in some way.

[StefanKarpinski]

I guess we can always deprecate --depwarn and codename the release "Inception".

[samoconnor]

@c42f Does the new logging infrastructure have support for multiple simultaneous log streams?
e.g. could I have a seperate log stream per task? or collect all the logs from a function call into a seperate log stream?

[c42f]

Yes, this is exactly the way it's designed: with_logger(f, logger) calls f with all logs for the current task directed to logger. In addition, any child tasks which are spawned by f (or any deeper part of the call stack) inherit logger as the task local logger.

[c42f]

This list is kinda out of date at this point. The items which haven't been dealt with are either low priority or are tracked in their own separate issues. So I think I'll close this.