Zig: Hacking-friendly "--sloppy" mode for Zig.

[vi]

As far as I understand, Zig places great emphasis on code maintainability.

Unfortunately, restrictions required for it are at conflict with ease of development, debugging and prototyping.

I suggest to have a special Zig mode that trades code maintainability for short-term ease, ergonimics and speed of hacking. Command-line option may be --sloppy or --devmode or --lax.

Proposed rules for such mode:

• When Zig would have a proper central package repository, the code is syntax-checked without --sloppy as a minimal acceptance test.
• In Zig documentation, books and tutorials, users should be recommended to avoid publishing (or even checking in) --sloppy mode code.
• Normal and sloppy mode choice should not be tangled with LLVM optimisation level, testing, assertion enabledness and other aspects that are typically different between debug in release.
• Additional built-in and library functions may be available. Example: https://github.com/ziglang/zig/issues/3296
• Errors about unreachable code (and similar), as well as other non-critical errors are downgraded to warnings. Such warnings may be hidden. Similar for ignoring errors, missing switch cases and so on.
• A warning may be emitted just for mere enabling --sloppy mode, as an eye sore reminding that it is not supposed to be permanently enabled.
• Mere turning on --sloppy without modifying source code should not affect program behaviour.
• Comptime functions may check for sloppy mode and @compileError if it is not enabled. The reverse should not be idiomatic.
• Sloppy-only functions and language features are not considered part of Zig. They may be removed / changed even after Zig goes 1.0.0.
• Stable, official versions of Zig compiler may omit support of --sloppy mode. Master builds may be required.

[Srekel]

A suggestion for option name: --exploratory

Sorry for bikeshedding, but the topic came up on Discord a while back and someone said that "Zig is currently not great for exploratory code", or something to that effect, so that's where it comes from. :)

Sometimes, you just don't know exactly what you're building, or how you want to do it, so you're "exploring" how to write the code. (Especially true for people new to Zig.) Having to handle for example allocation errors when you "know" they won't happen slows you down more than it helps in this particular stage.

[andrewrk]

Related: #68

[pixelherodev]

The main issue with this idea (in my mind at least) is that it'll almost certainly result in people forgetting to turn the feature off, and then having to clean up a giant mess of code when they go to e.g. submit to a Zig package repository.

I think this idea is definitely workable, but it might need a tad more thought.

Another more minor issue: I don't think downgrading things like unreachable code to warnings is the best idea - that I think should show up as an even lower priority, as otherwise too many warnings could stack up quickly, making it hard to find what you're actually looking for. On the other hand, having it show up as a warning means there's a greater incentive to fix it, to clean up the warnings list even in --sloppy mode.

This definitely needs a lot more thought, though I do think it has merit.

[vi]

I don't think downgrading things like unreachable code to warnings is the best idea - that I think should show up as an even lower priority, as otherwise too many warnings could stack up quickly, making it hard to find what you're actually looking for.

Obviously, in the sloppy mode warning classes should also be ignorable.

For non-sloppy-mode usage, warnings should be ignorable on case-by-case basis, when unreachable code is by design due usage of to generic (comptime) tricks.

[vi]

people forgetting to turn the feature off, and then having to clean up a giant mess of code when they go to e.g. submit to a Zig package repository.

That's inevitable. It is also inevitable that sloppy-only libs that only live on Github, not on the official repository will apear.

But I assume there are many projects that don't come to the phase where it is reasonable to publish them (i.e. experiments). Pre-publishing cleanup (during which README, documentation, tests, etc. may be written) is simply not needed for them. Requiring clean code for everything (including one-off throw-away code) may be suboptimal.

[Srekel]

Jonathan Blow just tweeted a thread (how timely! :) ) related to this: https://twitter.com/Jonathan_Blow/status/1194289482469502976

More of my perspective: I tend to always work in "full" warnings/warnings as error-mode in C/C++, so I appreciate releasing things where the code doesn't have hidden bugs or whatever.

But I agree with Blow here, and it probably needs to be more granular than on a project/build-wide setting? Like let's say you're working on a game and you're working on a subsystem that you want to fiddle with a bit before deciding on the final implementation, it makes sense to be able to mark that as sloppy, in some sense. I'm not sure if that can be easily solved in practice though... :)

And like @vi said, sometimes you work on something that you never have time to finish as a "proper" project, but that may still be valuable to people. Here's one of mine, https://github.com/Srekel/the-worldinator, it's not done yet and not sure if it ever will be, but I imagine someone might learn something from it.

[vi]

More of my perspective: I tend to always work in "full" warnings/warnings as error-mode in C/C++, so I appreciate releasing things where the code doesn't have hidden bugs or whatever.

hidden bugs or whatever

Some warnings are not about hidden bugs, but about maintainability. Notably unused code warnings (including unused imports and unused mutability). Those warnings are most often triggered by something being unfinished (although sometimes they point to a bug as well, but rarer).

For example, when I start doing things in Rust, I typically add #[allow(unused)] as soon as I see first of such warnings. Eventually this annotation goes away, as module stabilizes. It may reappear when heavy refactoring is undergoing. Something similar can happen in Zig projects. Start in exploratory mode, then go to normal mode. When reogranisation/refactoring happenning, switch the project back to exploratory mode for a while. For this it may be worth supporting specifying --sloppy both on command line (quick debugging sessions spanning minutes/hours) and inside source code (project reorg spanning days/a week).

"proper" project, but that may still be valuable to people

Nobody forbids publishing unuploadable-to-official-repository code to a github.

[dbandstra]

Also related: https://github.com/ziglang/zig/issues/335 (proposal to add more errors for unused/unreachable things - these should be reduced to warnings in sloppy mode)

[marler8997]

Instead of a command-line option, I would suggest some type of configuration inside the source code. If code is written in this sloppy mode, then it's not really a command-line "option" that you can enable or disable, it's a feature of the code itself. The code is either written in "sloppy" mode or it's not, so the code should declare what mode it's written in.

Allowing each source file to declare this also allows you to compile multiple modules with sloppy mode enabled/disabled on a per-module basis without having to invoke zig multiple times to enable/disable it for each one.

[marler8997]

After reading more about this it actually may make more sense to make it a command-line option. I thought this was a permanent type of mode for zig source code, but it looks like it's just a temporary switch to aid in faster refactoring/development.

[vi]

@marler8997 , I think it should primairly command-line switch, but also triggerable from build.zig (making it permanent-ish for a project).

Production-oriented version of Zig compiler may reject both.

[pixelherodev]

Production-oriented version of Zig compiler may reject both.

I think that's a good idea.

[Srekel]

Related: #224

[marcthe12]

This very useful for one off things or proof of concept. I suggest to make it as annoying so it is used sparing. I suggest avoid adding features but instead downgrading several errors to warning if they are related to safety or code maintenance. So if want to test a new method to do something as a proof of concept, I could test spend more time to test if the idea works.

[nektro]

the only material change to the language this proposal proposes is Errors about unreachable code, as well as other non-critical errors are downgraded to warnings which would be a very very bad idea. Zig already has plenty of ways to make it so userland code can ignore errors if it so chooses.

one large barrier making Zig less "hacking-friendly" imo is needing to always having an allocator handy, but I don't think there are any plans to change that. and it's not that big of a mental change to need to add one extra property to your structs.

if someone doesn't like the ethos that Zig is going for, I would hypothesize that they are more than welcome to use another language.

[pixelherodev]

The more I think about this proposal, the less I like it.

Additional built-in and library functions may be available. Example: #3296

The specific issue linked could be solved in userspace. More generally, the idea of adding functionality in such a mode seems quite problematic.

Errors about unreachable code (and similar), as well as other non-critical errors are downgraded to warnings. Such warnings may be hidden. Similar for ignoring errors, missing switch cases and so on.

This seems like a bad idea. There's a reason these are errors. My general rule for this is that if something indicates a problem with the code, it should be an error and not a warning.

A warning may be emitted just for mere enabling --sloppy mode, as an eye sore reminding that it is not supposed to be permanently enabled.

People are used to ignoring warnings in other languages (especially C). Adding one for this won't accomplish anything.

Mere turning on --sloppy without modifying source code should not affect program behaviour.
Comptime functions may check for sloppy mode and @compileError if it is not enabled. The reverse should not be idiomatic.

These two points are contradictory. If it's possible to check, at comptime, whether sloppy mode is enabled, some functionality will be written to depend on it - and quite likely to change behavior based on it. Being able to check for sloppy mode at all seems like a bad idea, given the stated intent.

Fundamentally, this proposal means creating a compilation mode which ignores the Zen of Zig. It explicitly is intended to favor writing code over reading code, it replaces compile errors with both runtime crashes and bugs, and it prioritizes short-term developer convenience.

Another critical point: we intend to have a language server embedded in stage2 at some point. This should make refactoring painless anyways. I believe any other use cases for a sloppy mode can be addressed similarly - instead of making it possible to disable the language's safety features, we should work on making it so that you won't want to.

Overall, this seems to me to provide some short-term benefits at the expense of long-term code quality, and to make matters worse it requires complicating the compiler. Accordingly, I've changed my vote for to a vote against this proposal.

[pixelherodev]

It is inevitable that sloppy-only libs that only live on Github, not on the official repository will apear.

This is only inevitable if we make it an option.

But I assume there are many projects that don't come to the phase where it is reasonable to publish them (i.e. experiments). Pre-publishing cleanup (during which README, documentation, tests, etc. may be written) is simply not needed for them. Requiring clean code for everything (including one-off throw-away code) may be suboptimal.

Requiring clean code for everything may be suboptimal.

This is my fundamental issue with this proposal. This proposal will result in bad code being written and published - and, realistically, used as-is.

Some warnings are not about hidden bugs, but about maintainability. Notably unused code warnings (including unused imports and unused mutability). Those warnings are most often triggered by something being unfinished (although sometimes they point to a bug as well, but rarer).

This is why we don't have those warnings in Zig.

I think that if there are places where Zig is currently inconvenient to write, we should focus on fixing that without simply giving up and encouraging bad practices.

[dbandstra]

I think that this line of complaint is avoiding the original point, that this is a opt-in flag, which will be added in a way (to be determined) that makes it clear that it's only for use in the thick of development. We're not proposing to "add warnings". And if people are able to post or paste online code that only builds in sloppy mode, who cares? They can also publish code that doesn't compile at all. Or code that only compiles on an old version of Zig.

I don't think people are going to take a third party library seriously if it only compiles in sloppy mode. You as the application developer will most likely be using those libraries in source form, and you are the one setting sloppy mode or not.

Tabs vs space is a hangup for some people, this is a hangup for me. It's extremely annoying to get unreachable code or unused variable errors when you are trying to debug something and commenting out 10 different regions of code in turn. (And I don't agree with having to rely on IDE features to smooth this out.)

And I don't think invoking "Zen of Zig" is valid in itself. If it says that Zig is supposed to favor readability over writeability, you can interpret that to any degree you want. Why are we going to have a live reloading incremental compiler? That's pure writeability.

Disclaimer: I might be arguing for a smaller subset of the sloppy feature than was originally proposed here.

[pixelherodev]

They can also publish code that doesn't compile at all. Or code that
only compiles on an old version of Zig.

Code that doesn't compile won't be given attention. "Old versions of zig"
becomes irrelevant post-1.0, sloppy code doesn't.

>

I don't think people are going to take a third party library seriously
if it only compiles in sloppy mode. You as the application developer will
most likely be using those libraries in source form, and you are the one
setting sloppy mode or not.

Sloppy libraries will probably result in the application developer
turning on sloppy code too.

Tabs vs space is a hangup for some people, this is a hangup for me.
It's extremely annoying to get unreachable code or unused variable errors
when you are trying to debug something and commenting out 10 different
regions of code. (And I don't agree with having to rely on IDE features
to smooth this out.)

Refactoring wouldn't be an IDE feature, it'd be built into the compiler.

And I don't think invoking "Zen of Zig" is valid in itself. If it says
that Zig is supposed to favor readability over writeability, you can
interpret that to any degree you want. Why are we going to have a live
reloading incremental compiler? That's pure writeability.

Incremental compilation has no affect on readability, and is not favoring
writability over readability as a resuly. Sloppy code is less readable as
a direct cost of the improved writability, which does.

Disclaimer: I might be arguing for a smaller subset of the sloppy
feature as originally proposed here.

A smaller subset I'd likely be fine with, I'm against the proposal as
outlined in the OP.

[dbandstra]

Ok, rereading the issue there is a lot more going on here than I had in mind. To me, sloppy mode should just suppress lint-like errors like unreachable code, unused variables, using var where you could have used const, declarations being required to be in a certain order, etc. That's it. I definitely don't support the idea of letting code check if sloppy mode is enabled, adding sloppy-only library functions, and stuff like that.

[vi]

we intend to have a language server embedded in stage2 at some point. This should make refactoring painless anyways

This pushes the language into Java-esque IDE-only mode and detracts from text editor-based flow.

[vi]

Sloppy libraries will probably result in the application developer turning on sloppy code too.

I don't thing that would happen in masse if typical Zig user values good code (and I expect average Zig user mindset to be different compared to baseline).

[nektro]

This pushes the language into Java-esque IDE-only mode and detracts from text editor-based flow.

No it doesn't require IDE's at all. It only sets the minimum above notepad(++). Vim, VS Code, and any text editors that support basic extension would support a zig extension that is capable of automatically running zls and zig fmt

[rohlem]

Zig already has plenty of ways to make it so userland code can ignore errors if it so chooses.

It is proposed to introduce a compile error if you create a var that you never modify (#224) or never read from (#2304).
I don't know of any way for userland (the code being compiled) to "ignore" this compile error (short of employing a custom preprocessor).
If for this reason alone, I am in support of a --sloppy flag.
EDIT: I've since found out we already have an error for unused block labels. That one seems even less sensical, but at least block labels aren't quite as common (at least in my code) as variables, so it evens out.

[nektro]

using const where available is objectively better for performance and security over using var/let. how would pushing users to write better code be a bad thing? and why can't they use another language if this is a hard point for them?

[dbandstra]

Because this

It's extremely annoying to get unreachable code or unused variable errors when you are trying to debug something and commenting out 10 different regions of code in turn.

I don't think "use another language" is a very constructive response. This is an ergonomic detail, not some essential piece of the language.

[SpexGuy]

Some reasons to use var even though the value isn't modified:

• you are using a debugger and want to poke other values in for testing
• the value is a struct and you happen to only call functions on it that accept * const T, but those consts are implementation details, not interface details, and you are semantically modifying the contents in a way that may require the interface to change to *T in the future.
• it is semantically important that a value is runtime known.

This third bullet is surprisingly common. Here's one case:

const slice: []const u8 = ...;
var substr_index = 0;
const substr = slice[0..substr_index];
const ptr = @as([]const u8, substr).ptr;

In this example, changing substr_index from var to const makes it comptime known, which changes the third line from a runtime substring to a comptime sub-array. When substr_index is runtime-known, the type of substr is []const u8 and it has a valid pointer. Therefore the value of ptr is well defined. But if substr_index is comptime-known, its type changes to *[0]const u8, which is zero-sized and does not have a valid pointer. Therefore ptr becomes undefined. Pretty much any use of ptr after this point will trigger UB, but only if substr_index is const.

And here's another:

var i: u32 = 4;
const array = &[_]u32{ i };
array[0] = 4;

Here, i is never modified. However, i being runtime-known means that the array literal expression [_]u32{ i } is not comptime known. This makes it a stack local variable, meaning that @TypeOf(array) in the above code is *[1]u32, and therefore the line after is allowed to assign to it. But if i is changed to const, it's now comptime known. Which means that the array literal is comptime known and goes in the constants section. This makes @TypeOf(array) turn out to be *[1]const u32, which causes the line after to fail to compile.

---

That said, my personal opinion here is that we should un-accept #224 instead of accepting this. I'm strongly against the idea of creating two versions of the language.

[vi]

I'm strongly against the idea of creating two versions of the language.

In Rust world it seems to work ("nightly" vs "stable") - production users typically use stable. "nightly" for hacking + "stable" on CI is often a good combo, providing access to additional tools in development environment while keeping things in check for production environment.

Why wouldn't that work for Zig?

[SpexGuy]

IMO it's different because Zig aims to be a language that doesn't change often. As I understand it, Rust's nightly contains features which are planned for addition to the language but may not be available yet or may change, so beware. This seems like the primary reason that CIs tend not to allow nightly - it's likely to suddenly stop working at some point. But with Zig, the language doesn't change. So we have two separate dialects that are just as stable as each other, but one is more permissive than the other. Making a codebase compliant with the strict subset of Zig will be thought about the same way we think about converting html into xhtml. It's great if you are super strict about it but not worth it in almost all cases.

[rohlem]

Nobody asked for it, but here's a fun idea for __making absolutely sure__ --sloppy mode is not used in production: Give it a timestamp argument and auto-deactivate it after some time (f.e. one hour).

zig build --sloppy="2020-11-07-00:10"
Error: Your sloppy time is up! Current time: 2020-11-07-01:12

(Yes, you can write a script, but you can also just patch out errors from the compiler, and it's easy enough to adjust by hand.)
(And I'm only half-serious. But also only half-joking.)

[vi]

Trying to write the code that contains no unused things that are caught by Zig's "errors" is like trying to build a house while keeping construction site tidy and clean at all times.

Construction sites are typically messy and not pleasant to be in (at least in clean clothing without a hard hat). Cleaning up and removing all the garbage (in programming language case: unused things and dead code) typically happens near the end of construction process.

[vi]

Give it a timestamp argument and auto-deactivate it after some time (f.e. one hour).

Maybe a better idea is to allow only one Zig source file/module to be sloppy at a time: --sloppy=src/workinprogress.zig.

[SpexGuy]

Trying to write the code that contains no unused things that are caught by Zig's "errors" is like trying to build a house while keeping construction site tidy and clean at all times.

I think you may be overestimating what Zig will count as a compile error. Unused or unreachable code can't possibly be standard compile errors. The entire conditional compilation system is built with the assumption that there will be very large amounts of both of these.

The only issue this proposal mentions that is actually accepted is #224, and even that is susceptible to major conditional compilation problems. The other two, #2304, and #335, can only possibly be validated with multibuilds (#3028), which will be optional. build-exe is never going to fail because of either of these rules. Other checks you might want to turn off in sloppy mode, like #282, #952, #2654, and #5208 are all only enabled with multibuilds.

The only thing left is removing the requirement that switches handle all cases. But I don't think it's worth a whole separate compilation mode to avoid typing else => {}.

[vi]

build-exe is never going to fail because of either of these rules.

Is multi-build being stricter than regular build a design choice or just an implementation detail that may change as Zig compiler gets more clever (i.e. detecting that a thing is obviously unused regardless of various flags because of it is mentioned in the source code exactly once)?

Will there be "open" multi-build mode where Zig should check things for specified list of targets, but not assume that the specified set is complete (so some things can still dangle around)?

In presense of multi-build, will simpler modes like build-exe be still always available or there will be multibuild-exclusive features that lock projects to only building all targets at once?

[SpexGuy]

My assumption was that multibuilds will act more like a linter than a compiler, but I might be wrong about that.

Is multi-build being stricter than regular build a design choice or just an implementation detail

Before 1.0 there will be a language spec that defines exactly what the boundaries are for what does and does not compile. It will err on the side of simplicity, so "clever" checks that are difficult to describe in a specification will likely not be allowed to cause compilation failure, unless they are deemed to be really valuable (like maybe some cases of returning pointer to stack var).

Will there be "open" multi-build mode

Some form of this must certainly exist. The standard library contains code that only compiles on windows, but not all projects which import the standard library will have windows as a target. It would be ridiculous for this to be a compile error. That said, it could be that the plan is to only run multibuild on the "root" package, and not validate external packages. In that case there would be a simple workaround would be to make your root package a stub that references "sloppy" code, but that's a simple enough hole that it will probably make sense to make an open build mode. Additionally, multi-builds inherently require a decent amount of configuration, so I think there will probably always be a simple alternative way to say "build this zig file into an exe and don't do the multibuild validation".

All that said, I haven't actually talked to Andrew about his specific plans for multibuilds, so I might be wrong about the plan here.