Crystal: CRYSTAL_ENV

[crisward]

Kemal has KEMAL_ENV, Amber AMBER_ENV, and other code will have their own way of describing test, dev, production.

It'd be good to agree on this and the language should perhaps take the lead. Hence opening the issue here. Other shards would be able to tap into this to modify their behaviour.

This doesn't need any change to crystal, just perhaps a docs update.

[Papierkorb]

I propose:

• Use CRYSTAL_ENV as environment variable
• production, development and test as recognized values
• Add ENV.production?, ENV.development? and ENV.test? as test getters so that everyone can easily access it

This would practically enforce a standard, and ensure that random shards don't depend on checking ENV["KEMAL_ENV"]. Let's not have another Railstie thing.

The other side of the coin

But there's a catch. What we don't want to happen is that everyone uses these checks deep in their code to "patch stuff out for testing". This is the exact opposite of what this is good for, but it can be abused for it, so I'm eager to learn if someone has a better plan.

I think the documentation would need a stern note on how this feature is there for the initial configuration of the application, and NOT of the library, or any other code components. It needs to be clear on that if someone puts a environment check deep into their code, that they're doing something wrong.

This really leaves a sour taste for me. I think the general idea is good and useful. But this is an issue we shouldn't just ignore.

[robacarp]

The notion of requiring an environment variable to put code into "production mode" is fine when your code is running on a server you control, but it is not a viable paradigm outside of the web world and perhaps a few other places. Crystal does provide some facilities for web work, but is not a purpose built web language like the two frameworks @crisward cited. What would this mean if I was programming a system utility? A video game?

This is hanging community paradigms (or code) off of one use case for a language, which I'm not opposed to, I just want to make it a clear consideration.

The namespace feels weird. To me, an environment variable like CRYSTAL_ feels like it should be configuring crystal itself during a compile rather than a runtime as in JAVA_HOME and GO_PATH. As an aside, I'm very glad Crystal doesn't have a CRYSTAL_UNIVERSE or somesuch.

I understand the fear of the Railstie paradigm, but that feels like a community intent that a community might not have much control over. After all, a big lament in the ruby community is that it's "only used in Rails". Rails just happens to be the framework that landed, stuck, and got used enough to create that inertia. Moreover, I'm not sure how a community standard environment variable is going to prevent Amberties from materializing.

As @Papierkorb mentioned, KEMAL_ENV, AMBER_ENV, UPPERCASE_ENV etc are all kind of brutal to deal with, but it seems like a well designed library shouldn't need to be asking those sort of questions in the first place. Configuration allows those boundaries to be cut cleaner.

🍎

[Papierkorb]

What would this mean if I was programming a system utility? A video game? Is hanging code or community paradigms off of

This would be useful for everything that wants to distinguish environments. Which is everything where you talk to external systems, may that be an REST API or a DB. And this can also be useful for desktop applications (Disclaimer: I develop Qt5 bindings, and I can see a use-case).

And every other program where this doesn't make sense for, 10 more lines in ENV won't change things :)

[paulcsmith]

I love this idea and I love that ENV would have methods for querying against it so you don't need to do string comparison or worry about typos. This is basically what I did in Lucky as well: https://github.com/luckyframework/cli/blob/master/src/web_app_skeleton/config/env.cr

I also like the CRYSTAL_ENV name. It's what I would expect it to be.

I think your warning is spot on. It should be used for initial configuration. Right now I'm using Habitat for configuration and use LuckyEnv to change configuration there, never in the library code itself. Side note: I'd love it if something like this existed in Crystal itself so there is just one way of configuration Crystal projects. Similar to what Elixir has with mix and MIX_ENV, but maybe that can be added later or handled by an external lib (like Habitat)

@robacarp I think this is useful outside of web projects. Knowing whether something is running in test and configuring the application differently is really important for a number of projects. For example, I'm starting on a library for sending emails. In test, I want to use a TestAdapter that doesn't actually send emails for most of the specs. But in dev/production I'd probably use a real adapter. I think that's pretty handy

# An example
MyMailer.configure do
  if ENV.test?
    settings.adapter = TestAdapter
  else
    settings.adapter = RealAdapter
  end
end

[yxhuvud]

Isn't this something that could be developed as a shard and then added to stdlib if it turns out to be commonly used?

There are also other things to consider:

Should it only enumerate a set of environments, or allow people to define their own? I have never worked on a big rails app that was limited to test/dev/production - all major ones have had other defined environments (packaging, asset compilation, multiple staging environments etc).

Should this only be used to enumerate environments, or should it go the full way and provide a tool to provide environent dependent variables, like dotenv?

[robacarp]

Sorry, I didn't mean to say that toggling between program paradigms doesn't have a place outside internet domain programs, but that it often works differently. In web applications, the default environment is often development, while on desktop applications the default is going to be production.

[Wulfklaue]

Isn't this something that could be developed as a shard and then added to stdlib if it turns out to be commonly used?

The issue with outsourcing is that control is lost. I have seen too many new languages where the solution is to outsource into there packages and them issue arise when those packages do not get maintained.

A other issue: People use the Shard version, later it get added to stdlib and then you have two versions in the wild ( assuming that for compatibility the syntax stays the same ). Lets say the stdlib version gets more or less features. Some unsuspecting person new to Crystal reads something and then complains that the option does not work because they use the wrong one.

Critical components like DB drivers, Env options, all belong into a main branch of a programming language.

[robacarp]

@Papierkorb, Something else occurred to me last night about this statement:

And every other program where this doesn't make sense for, 10 more lines in ENV won't change things :)

While I agree, it's not a pattern that scales. I imagine that there are 10 lines of code for N problem domains that could be incorporated into the crystal core, be they into ENV or elsewhere. This creates the same "railties problem", doesn't it?

That said, one concession does not necessarily a bad pattern make.

[asterite]

I agree with the proposal of this issue. This is all about a convention of having some environment variable that all Crystal apps can use to determine mode of execution.

I'm fine with using the name CRYSTAL_ENV for this. We can then use "development", "test" and "production", or any other value for that matter. But "development" will be the default one (Maybe Crystal should set it at the start of each program unless it's set), "test" should be set by test frameworks, and production is set when explicitly setting CRYSTAL_ENV=production. We could use --release for this, but I'm not sure it's a good idea.

We could actually use shorter names like in Elixir's Mix: http://elixir-lang.github.io/getting-started/mix-otp/introduction-to-mix.html#environments . Maybe that's easier to write comparisons. Alternatively we could have Crystal.env and Crystal.env= as something that provides access to that, I wouldn't touch the ENV module for this.

[paulcsmith]

I love the idea of Crystal.env instead of adding it to ENV. I like that it's easier to spot typos with the shorter names in Elixir, so maybe it's worth using short ones. I'm also ok with the long form version. I also like the idea of returning development if it's not explicitly set.

I think that crystal spec should automatically set the env to test though. I think that's what Elixir's Mix does when you run mix test and it seems to work well. What was your thought behind leaving it up to the test frameworks?

[asterite]

I mean that when you do require "spec", spec.cr inside the standard library sets the ENV variable. We can also make crystal spec do that, but you can run spec files without doing crystal spec, just doing crystal spec/some_file.cr works.

I guess we could also have Crystal.dev_env?, Crystal.test_env? and Crystal.prod_env? for these common environments, providing a bit of type safety (or the longer development_env?, but I have no real preference here).

[paulcsmith]

@asterite Ah I see what you mean. Yeah maybe setting it when you require "spec is the best idea since like you said, you can run it with crystal directly. Seems that is a safer approach

As for the type safe env query methods I think it _might_ be easier to remember (and maybe look a bit nicer?) as:

Crystal.env.test?
Crystal.env.dev?
Crystal.env.prod?
Crystal.env.name # Returns the bare string, e.g. "dev", "prod", "test"

However this is a minor suggestion. What you have also work well I think :)

[asterite]

I like that too. It might be a bit confusing that Crystal.env == "custom" won't work, though, but we can always override the == for that object, and maybe also override it for String so that "custom" == Crystal.env still works... hmm...

[paulcsmith]

That sounds like a pretty slick idea to me :)

[Papierkorb]

Overloading String#== for some random data type is doing more bad than good. If people want to create custom environments, they could just access the proposed #name : String.

[paulcsmith]

Maybe one way to handle it so that we don't overload == and people don't get confused is to raise a helpful compile time error if you try to compare it to a String. This way they don't need to Google or search the docs for what they're supposed to call to get the env name.

class CrystalEnv # or whatever it will be called
  def ==(other : String)
    {% raise <<-ERROR
    Can't compare env to a String. Instead call Crystal.env.name

    Example: Crystal.env.name == "my_custom_env"
    ERROR %}
  end
end

[straight-shoota]

@paulcsmith This CrystalEnv#==(String) is totally fine, I think. The issue is the other direction, requiring a special overload String#==(CrystalEnv).

[paulcsmith]

Ah. I should've read more carefully! Thanks for the clarification @straight-shoota

[robacarp]

I don't know what the precedent for using or not using method-missing in std lib, but this would accomplish the same thing without locking in to a standard set of environments?

class CrystalEnv
  macro method_missing(call)
    name = "{{ call.name }}"
    if name.ends_with?('?')
      ENV["CRYSTAL_ENV"] == name[0...-1]
    else
      # I don't actually know what to do here, but I don't think this works
      super
    end
  end
end

It doesn't matter much, but I'd prefer the full name to the abbreviated names like elixir.

[paulcsmith]

@robacarp I think that could work, but then you don't get the safety you'd get with hard coded method names.

For example, with method_missing you could do CrystalEnv.devlopment? and it would compile, but wouldn't work as you expected (unless you have an env called devlopment :P). Whereas if you had hardcoded method names you would get a compile error and it would helpful tell you "did you mean development?".

I think instead if you wanted to add your own custom environments you could open up the class and add a method to it, or maybe there could be a macro for registering new environments. But that would probably be best to figure out later

[asterite]

In general we avoid method_missing at all cost.

[ysbaddaden]

I don't see the point to have a compiler environment variable. There is flag?(:release) which means something, but CRYSTAL_ENV...? It may make sense in specific frameworks that can have different environments (thought this is debatable) but... at the compiler?

IMHO this is a great idea for an env shard, with more capabilities such as populate ENV from .env or config/development.yml or different kind of files.

[asterite]

@ysbaddaden Maybe the name CRYSTAL_ENV is confusing, but it's definitely not a _compiler_ environment variable. It's similar to MIX_ENV in Elixir, RAILS_ENV in Rails, etc. Just an environment variable that tells you what "mode" an app is running. Here we are just defining a convention for all Crystal apps, instead of coming up with names like KEMAL_ENV, AMBER_ENV, etc. (which actually already exist, at least for kemal).

[lachlan]

What about Runtime.environment rather than Crystal.env, since then it's more obvious what it relates to?

Personally, I'd prefer using whole words for the environments: development, test, production.

[straight-shoota]

@asterite For Ruby it's RUBY_ENV - RAILS_ENV is application-specific. That's what we want to avoid 😉 And somewhere in between is RACK_ENV...

[drhuffman12]

APP_ENV?

[drhuffman12]

I like the class approach ['CrystalEnv' or whatever it ends up being called] for a common base; and, then people can sub-class, include, or monkey patch it as applicable for their needs.

Maybe 'CrystalEnv' [or 'Runtime' or 'AppConfig'?] could look something like

  .environment
    :name [default 'dev']
    :id   [default 0]
  .instance
    :name [default 'app']
    :id   [default 0]

The 'AppConfig.environment' and 'AppConfig.instance' objects might be a Hash or Json so people can customize/extend as needed.

I think this would be simple and customizable enough to work for simple use-cases, as well as multi-instance servers or multi-user game clients.

For example, you could end up with something like:

  .environment
    :name            == 'prod'
    :id              == '1'
    :custom_env_key1 == 'USA cluster 1'
    :custom_env_key2 == 'my custom value 2'
  .instance
    :name            == 'server'
    :id              == '123.45.67.89'
    :custom_inst_key == 'my custom value'

  .environment
    :name            == 'dev'
    :id              == '1'
    :custom_env_key1 == 'USA cluster 1'
    :custom_env_key2 == 'my custom value 2'
  .instance
    :name            == 'client'
    :id              == 'beta-tester-123'
    :custom_inst_key == 'my custom value'

[ysbaddaden]

There is no RUBY_ENV, but RACK_ENV. By the way, RACK_ENV doesn't do much, if anything —RAILS_ENV does more, and is actually useful.

I believe CRYSTAL_ENV is confusing. Using Runtime is even more confusing, since it could be understood as "the crystal runtime is in development"... what?

Anyway, this is my last reaction: I believe it would be a great shard, that should be discussed by the different frameworks that use an environment, what they use it for, and how it could be accepted to have a convention over it. Otherwise we'll probably have CRYSTAL_ENV=KEMAL_ENV, just like Rails assigns RACK_ENV=RAILS_ENV.

[Sija]

_/shameless plug/_ but here you go: crystal-environment

[straight-shoota]

Honestly, I'd really hate having to add such dependency with really just a grain of code providing Crystal.env. Please, let's avoid having such a fragmentation of countless microscopic components like in the node.js ecosystem.

[Sija]

@straight-shoota same here. I'll be happy to make a PR to stdlib, @ysbaddaden u ok with it?

[Sija]

@ysbaddaden hello?

[ysbaddaden]

I still believe this is confusing and would prefer a Shard with larger scope (i.e. reading variables from different files and formats), or just each project to have their own, tailored, solution.

Did somebody from Amber, Kemal or another project said they'd use it?

[paulcsmith]

I would definitely use this on Lucky. Right now it's LUCKY_ENV, but I would prefer not to get into the situation in Ruby where there is RAILS_ENV and RACK_ENV that should always be the same because they're used in the exact same way.

I think it would be nice to have one way to set the environment, like how Elixir has Mix.env that all libraries can use to see what environment Elixir is running in.

[bcardiff]

I think I would prefer this to be a separate shard also. Any integration with specs or changes with release mode compilation can be handled even on separate shards.

Other than including the library in the projects spec_helper and in the entry point to set the defaults I see no other needed boilerplate for the user. And this can be done by the the web frameworks when expanding. Maybe the crystal init could also join that idea, but that is another topic.

If the crystal init will use that the shard would need to live in the crystal-lang organization. But, is that also a requirement for the frameworks to agree on a common environment/stage detection?

[paulcsmith]

I think an external shard that on the crystal-lang org would also work well. I think the key is just that there is one "official" way to set/get the Crystal runtime environment. I think we could standardize on dev/test/prod or development/test/production. I think those three are the most common. I personally have a slight preference for the shorthand version, but I think it's fine either way.

And I agree that the tiny boilerplate needed to set the env in spec_helper is not a big deal at all. As long as we have a consistent way to get/set the env it should be fine

[Sija]

@bcardiff external shard under _crystal-lang_ org + adding minimal boilerplate to crystal init sounds gr8 to me, as such solution would cover all of the needs and fears expressed in this issue, without the risks of depending on 3rd party shards.

Let's have One-Way-to-Do-It ™

[RX14]

A shard please, this is way too opinionated and non-universal for the stdlib.

[bcardiff]

What do you think if we start with something like https://gist.github.com/bcardiff/129962b01266dccfa2fc5ee8ba2ea042 ?
The public api is minimum at support default env for specs via requirements. @Sija is almost the api you propose in https://github.com/Sija/crystal-environment . It seems we were both coding at the same time 😅 , yet I haven't push the code to github.

[RX14]

The tiniest of nits, I'd prefer just crystal-lang/env for the git repo, duplicate crystal- prefixes is fairly useless.

[bcardiff]

@RX14 , well you access it via Crystal.env. But, yes there are some inconsistencies with the require "env". I don't like require "crystal-env". So, maybe we shoud use crystal-lang/env. In the cases of db/sqlite3/mysql I use prefix because otherwise is a bit unclear from the url if it is a shard or a fork.

[Sija]

@bcardiff Public API is essentially the same, one difference being Crystal::Environment.setup macro which allows you to extend default set of environments (requested by @veelenga in https://github.com/crystal-lang/crystal/pull/5261#discussion_r149726228), which I find pretty useful for non-standard scenarios.

[bcardiff]

On an api related issue, I prefer to not expose the name. and just use the development?, etc. methods. If we just use that api we could support: dev, devel, development, prod, production, test, spec as values of the environment variable and allow people to setup with short/long names, but have a single way to check.

A getter that will return :development, for dev, devel, development values, could be added for sure.

There is tension between supporting aliases and allowing custom environment names.

[Sija]

So maybe drop .name and have .setup(envs) variant where, _envs_ would be a HashLiteral/NamedTupleLiteral with _alias_ as key, and _env_ as value.

sth along the lines of:

# for aliases
Crystal::Environment.setup({dev: "development"})

# for extending
Crystal::Environment.setup(%w(integration))

On the second thought I'd rather have .name getter for printing the name of current env in logs, debug, wherever...

[bcardiff]

If we free the list of possible environments, who is responsible for defining it? the framework or the app? if it is the app, inner shard won't be able to rely on any set of know environment. If it is the framework, then some shards might turn framework dependent. (yet, shards should have a more fine grained config that environment)

That's why I am more in favor of a fixed set of environments and allowing to change just which is the default.

[RX14]

Please keep this simple: dev, test, prod only.

[RX14]

We've agreed this should be in a shard, closing.

[bcardiff]

I've just published a WIP at https://github.com/crystal-lang/crystal-env . Let's see how the story evolves there.