Go: proposal: Go 2: return exit code from main (remove log.Fatal)

[smyrman]

Goal

Returnable exit codes should provide an easy and safe way for programmers to exit their programs so that deferred functions are run by default. The new pattern should "force" (a.k.a. help) the programmer to write safer programs, similar to how the explicit error handling "forces" programmers to deal with error in an explicit manner.

The overall goal is therefore that this limited language change would lead to fewer bugs in Go programs related to graceful shutdowns. To that end, some features _may_ also be added, removed or changed in the standard library to recommend better practices. Any changes to the standard library, if accepted, would _have_ to allow the go fix tool (or a similar tool used in the Go 2 code upgrade process), to detect and replace code for retaining the original behavior in valid programs.

Background

I wrote a post a while back about how log.Fatal (and os.Exit) does not fit well with other Go patterns. This post forms the background for this proposal, and can be read to get some context.

In short, when using log.Fatal (or os.Exit), processes exit immediately, not allowing for deferred functions to run. While this may be the desired way for a program to work in some cases, in most cases, I would assume it's a bug. In my opinion, this pattern for exiting a program stands in contrast to how go's error handling works, where explicitness through error return values have been chosen over e.g. the less verbose (and probably less safe) exception throw/catch pattern used in some languages.

func main() {
    defer func() {
        fmt.Println("defer functions may or may not run...")
        fmt.Println("...or they may get interrupted in the middle of execution")
    }()
    os.Exit(1)
}

I believe it would be more in line with Go language design to switch to _returning_ exit codes from the main function in Go 2, allowing for deferred functions to complete. I.e. similar to how this works in _C_, but nor _necessarily_ following the exact same syntax and semantics if there is a _good use case_ for something else.

Note on numeric exit codes

The conventions for exit codes are quite different between POSIX and Windows, or even between different programs. Linux itself seam to accept an int32 where most Unix shells would only retrieve the last 8 bits, or fallback to 255 (out of range). Windows ExitCodes are defined to be int32, but treated as unsigned by the PowerShell. There may or may not be some library support added to help define common error codes for Go programs (potentially with different values for different operating systems), but this is not the main objective of this proposal, so I will not go into any great depth about this.

However, usually languages (including Go) have been able to assume at least three values, and use them on all systems regardless of OS conventions: success (0), failure (1), invalid command-line flags or parameters (2).

Suggested syntax

Unless there is a good reason for something else, I suggest we follow the most obvious syntax, and let main return a normal int.

package main

func main() int {
    defer fmt.Println("defer function run")
    return 0 // success
}

package main

func main() int {
    defer fmt.Println("defer function run")
    return 1 // failure
}

How it should work

Basically, the main function of the main packages should receive a new required return parameter that returns a numeric exit code (in some form), that is returned to the system when a program exits. The semantics of this would be equal to the following code in Go 1 today:

func main() {
    os.Exit(program())
}

func program() int {
    defer fmt.Println("defer function runs")
    return 0
}

As you can see, this is not a _hard_ pattern to implement today, but it's also not an officially _recommended_ or particular common pattern. It also feels a lot like a certain Python convention:

def main():
   ...

if __name__ == "__main__":
    main()

Which is what we are going to change. Thousands of new Go programmers could potentially arrive at the semantics above within the first day of picking up the language and without the hacky feel to it.

Support in standard library (optional)

To make the feature more convenient in use, and to recommend better practices, the following changes are suggested for the standard library.

In particular, the removal of some functions is suggested to make space for the new feature, without accumulating language complexity.

These are only suggestions though, and real implementations may find better patterns or packages for containing such changes, depending on e.g. what else will change in the standard library for Go 2, if anything. Some or all of the suggestion may also be rejected for not bringing enough value, or to minimize changes to the Go standard library for version 2, if that's desirable. Non of the changes are _necessary_ for the exit return status to work, and bring value.

package errors

Package errors receives a new interface ExitCoder, and a function ExitCode(err error) int that can be used to retrieve an exit code from an error value.

// in package errors

// ExitCoder is an optional interface for error implementations that can
// be used to suggest an exit code for a given error.
type ExitCoder interface {
    // ExitCode recommends a numeric exit code to return
    // from the main function.
    ExitCode() int
}

// ExitCode returns 0 if err is nil, the result of ExitCode if ExitCoder is
// implemented by err, or 1.
func ExitCode(err error) int {}

Package flags

Errors returned from flags.FlagSet.Parse(args) error, will implement the errors.ExitCoder interface and always return 2 (error for invalid command line usage).

flag.Parse() is changed to flag.Parse() error, and will not exit on error by default. Preferably the whole flag.ErrorHandling parameters and options are simply removed, depending on how easy it is for tools like go fix to handle this.

Package logs

The log.Fatal and log.Fatalln functions are removed completely, both from the package and log.Logger type(s). A go fix pattern is crated to update any usage of log.Fatal(x)/ log.Fatalln(x) in Go 1 programs to use log.Print(x)/log.Println(x) (or the equivalent call on a log.Logger instance) followed by syscall.Exit(1) for Go 2.

This removal should help recommend good practice. As mentioned in my post I have seen that even _experienced_ Go programmers may use log.Fatal within imported packages, but the truth is that it isn't even safe to use log.Fatal in the main.main function, if that function includes a deferred function necessary for grace-full shutdown.

Note on os.Exit

os.Exit can probably be removed. All reference of it would call the existing function syscall.Exit instead, showing that this is now considered a more _low level_ operation.

Backwards compatibility (go fix)

Summary of what go fix would need to replace in Go 1 code for retaining backwards compatibility when translating the program to Go 2 code, if all the individual library updates are accepted.

main.main signature

// go 1
package main
...
func main() {
    ...
}

// go 2
package main
...
func main() int {
    ...
    return 0
}

This is the only _required_ pattern, if none of the _breaking_ changes to the standard library are accepted. Alternatively, even this pattern is not _strictly_ necessary, as the int return parameter for the main function _could_ be made optional (like in C). I am not to big fan of that approach though, as it adds complexity to the language by giving the programmer an additional and unnecessary choice.

flag package changes

// go 1 (may need to detect that this is file is go 1 code somehow)
flag.Parse()


// go 2 IF error handling support is not removed.
func init() {
    flag.CommandLine = flag.NewFlagSet(os.Args[0], ExitOnError) // if not already set
}
...
flag.Parse()
...

// go 2 IF error handling support removed (and go fix finds go1 flag.CommandLine.errorHandling == ExitOnError)
if err := flag.Parse(); err != nil {
    syscall.Exit(errors.ExitCode(err))
}

// go 2 IF error handling support removed  (and go fix finds go1 flag.CommandLine.errorHandling == PanicOnError)
if err := flag.Parse(); err != nil {
    panic(err)
}

// go 2 recommended pattern when calleld from main in
// new programs (for reference only):
func main () int {
    ...
    if err := flag.Parse(); err != nil {
        return errors.ExitCode(err)
    }
    ...
}

A similar pattern is need to match calls to Parse on flag.FlagSet instances (including an explicit call to flag.CommandLine.Parse), that has the ExitOnError or PanicOnError set as the ErrorHandling method _and_ does not already handle or capture the error; programs could still have code to handle this, even if it's not effective with the actually configured ErrorHandling.

// go 1
flag.NewFlagSet(a, b)

// go 2 IF error handling support is removed.
flag.NewFlagSet(a)

log.Fatal removal

// go 1
log.Fatal("foo")

// go 2
log.Print("foo")
syscall.Exit(1)

os.Exit removal

// go 1
os.Exit(...)

// go 2
syscall.Exit(...)

The log.Fatal pattern would also need to match log.Fatalln, and Fatal/Fatalln calls on log.Logger instances. there also may be good use-cases for calling it directly in some cases. Alternatively, it may be OK to remove os.Exit, if there is an equivalent syscall function that can be called in it's place. This would indicate that the Exit function is intended for more low-level control only.

---

UPDATES: Removed multiple syntax suggestions, leaving only one, and updated suggestion for library changes, moved the Goal section to the top, rewrote Backwards Compatibility

[smyrman]

Updated description to include single syntax example, rather than three different ones. Updated suggested changes to libraries.

[ianlancetaylor]

There are other ways to exit a program without running any deferred functions: call panic from a different goroutine, call syscall.Exit.

The only advantage of returning a value from main is to skip the call to os.Exit. Removing os.Exit entirely means that all programs have to thread their exit status back to the main function, which can be quite awkward, may require a global variable, and is especially painful when going through different goroutines.

If you want to adopt a programming style in which you don't call os.Exit or log.Fatal, that seems quite reasonable. But not everybody will adopt that style. I don't see a reason to change the language here.

[smyrman]

Maybe you are right, maybe a language change is not what's needed.

The _goal_ of the proposal was to provide, especially new programmers, with a simple and _recommended_ way for exiting programs that _handles_ defer functions upon program exit; safe by default. I.e. lower the chance of falling into the current language pit-fall and thus make the language easier to learn. It's not a _goal_ to impose any speicifc programming style, rather just to make the language slightly more explicit.

While I am happy this proposal to be rejected _as is_, maybe before closing of the issue, it would be nice to try to figure out if this _pitfall_ is a big enough problem to care about or not? If so there might be other solutions to it that works better, and don't require a language change.

I myself have been writing Go since 2011, so I don't have a problem understanding how os.Exit (nor log.Fatal) works, but I have seen others that do, also experienced Go programmers.

Just to clarify the current proposal:

Removing os.Exit entirely means that all programs have to thread their exit status back to the main function, which can be quite awkward, may require a global variable, and is especially painful when going through different goroutines.

I am _not_ suggesting to remove syscall.Exit(), which is in practice the same as os.Exit() (apart from some missing race detector decorators). As you say, not everyone would apply a new style, and that's fine.

As the proposal is _not_ removing os.Exit semantics entirely, I don't see how the change introduces _awkwardness_ nor how it would require global variables:

1. People who _want_ the semantics of log.Fatal could still call:

log.Print(msg)
syscall.Exit(1)

This is exactly the same as log.Fatal(msg), except it's more _explicit_, and not something that's easily done by mistake. See e.g. https://dave.cheney.net/2015/11/05/lets-talk-about-logging for @davecheney's opinion.

1. People who _want_ to write programs with safe shutdown (or as safe as possible), would already _today_, need to return an error to main (or use global variables, if that's the way you like to write programs). They then either need a wrapper function or deferred call to os.Exit (in the right order), to ensure safe exit.

I believe the real effect of the proposed change would be more along the lines of:

• Make immediate exits _more explicit_ (through os.Exit and log.Fatal removal; syslog.Exit kept).
• Make exits that _honor_ defer functions and safe shutdown principals less awkward and more native.
• Optionally (with library support): allow _translating_ an error into an exit code, as well as embed an exit code _suggestion_ with an error through the errors.ExitCoder interface.

The latter point is probably the _least refined_ part of this proposal, and a proper proposal document would need some more thought around this. Perhaps could some ideas also be taken from errors.Wrap in https://github.com/pkg/errors, if desirable.

Also, maybe a language change _is_ to drastic.

There are also a lot of other _less refined_ parts of this proposal that wold need to be cleared up. E.g. another very related pit-fall to that of os.Exit is not handling errors from deferred calls. Here is a code example that demonstrates that:

func main() int {
    // assumes errors from flag.Parse to implement errros.ExitCoder and returns 2.
    if err := flag.Parse(); err != nil {
        log.Print(err)
        return errors.ExitCode(err) // exits with 2.
    }
    session, err := db.Open()
    if err != nil {
        session.Close()
    }
    defer db.Close()

    // Assume errors returned from ListenAndServe does not implement the ExitCoder interface
    err := http.ListenAndServer(":8008", nil)
    return errors.ExitCode(err) // closes the db session, exits with 0 or 1
    // oops.. session.Close() errors not handled...
}

I have an idea for a (non-language change) proposal that wold work better for that, but it might introduce other errors and have other shortcomings; most language features probably work nicely if it's only used in exactly the way the inventor intended it;-)