Wrangler: Add support for multiple environments

[EverlastingBugstopper]

EDIT: A lot of the stuff in this top level is outdated - you can browse through the Environments milestone for a clearer picture :)

This issue was born out of RFC #220 👶

👉 Current Situation

wrangler publish --release only supports a single configuration. It would be very useful to compose a wrangler.toml that allows a project to be deloyed across multiple zones/accounts.

🙏 Proposed Usage

I think it would be good to keep this change backwards compatible with existing wrangler.toml configurations, and also allow a new configuration type. The following would all be valid wrangler.toml files.

👷 Current Configuration Style

name = "worker"
type = "webpack"
zone_id = "12345abcd"
private = false
account_id = "12345abcd"
route = "https://example.com"

This configuration would allow you to wrangler publish for zoneless and wrangler publish --release for zoned. This behavior should remain unchanged with this update.

👷 Add a staging environment to an existing configuration

name = "worker"
type = "webpack"
zone_id = "12345abcd"
private = false
account_id = "12345abcd"
route = "https://example.com"

[env.staging]
name = "staging-worker"
type = "webpack"
zone_id = "12345abcd"
private = false
account_id = "12345abcd"
route = "https://staging.example.com"

This configuration would allow you to deploy zoneless with wrangler publish, wrangler publish --release would deploy the configuration at the top, and wrangler publish --release --env staging would deploy the worker to their staging environment.

👷Configuration that requires explicitly specifying environment

[env.prod]
name = "worker"
type = "webpack"
zone_id = "12345abcd"
private = false
account_id = "12345abcd"
route = "https://example.com"

[env.staging]
name = "staging-worker"
type = "webpack"
zone_id = "12345abcd"
private = false
account_id = "12345abcd"
route = "https://staging.example.com"

This configuration would allow you to publish zoneless with wrangler publish, and wrangler publish --release would ask you to specify an environment to deploy to. You could run wrangler publish --release --env staging, wrangler publish --release --env prod, and wrangler publsh --release --env staging --env prod.

🤢 Problematic Configurations

⚠️ Same name, different routes

[env.prod]
name = "worker"
type = "webpack"
zone_id = "12345abcd"
private = false
account_id = "12345abcd"
route = "https://example.com"

[env.staging]
name = "worker"
type = "webpack"
zone_id = "12345abcd"
private = false
account_id = "12345abcd"
route = "https://staging.example.com"

These workers environments have the same script name. I think we should throw an error here asking them to change the name of one of their workers so they do not overwrite each other.

⚠️ Same routes, different names

[env.prod]
name = "worker"
type = "webpack"
zone_id = "12345abcd"
private = false
account_id = "12345abcd"
route = "https://example.com"

[env.staging]
name = "staging-worker"
type = "webpack"
zone_id = "12345abcd"
private = false
account_id = "12345abcd"
route = "https://example.com"

This configuration is less of a problem because the API will already throw an error if you try to put a script with a different name on a duplicate route. (Right now the error message is kind of ugly though)

* EDIT 08-05-19 *

⚠️ Different types

[env.prod]
name = "worker"
type = "webpack"
zone_id = "12345abcd"
private = false
account_id = "12345abcd"
route = "https://example.com"

[env.staging]
name = "staging-worker"
type = "javascript"
zone_id = "12345abcd"
private = false
account_id = "12345abcd"
route = "https://example.com"

This is a possible config and while it's a user error it kind of smells like poor design to allow this to be a valid config.

[prestonp]

I think it could be error-prone to have prod and staging config in one file. Perhaps wrangler could allow specifying flag for alternative file like -f prod.toml and -f staging.toml

[EverlastingBugstopper]

☝️ open to considering this, it's probably much easier to implement and a cleaner API than what is currently being proposed. I'll speak with the team and gather thoughts. Thanks @prestonp 😄

[xortive]

Things I dislike:

• having to pass --release and an env feels _very_ weird. --release feels like a type of env
• --env should accept lists. wrangler publish --env production staging should be valid.

Things I really want to see:

• some sort of "dry run" functionality. complex publish commands could be an expensive footgun.
• if we are keeping single file config, the current, non-namespaced config type should act as inheritable defaults for the new [env.<env name>] configs. i.e. you could do this

name = "worker"
type = "webpack"
account_id = "12345abcd"
private = true
# no zone_id, or anything else as default.
# if someone had those things, we could use them by default however.

[env.dev]
private = false

[env.staging]
# different zone
zone_id = "not_default"
route = "https://staging.otherzone.com"

[env.prod]
zone_id = "12345abcd"
route = "https://production.com"

[env.other_account]
account_id = "12345abcd"
zone_id = "12345abcd"
route = "https://other.property.com"

and if you don't specify something in one of the envs, it will get that field (if needed) from the "default" env

[signalnerve]

@xortive:

if we are keeping single file config, the current, non-namespaced config type should act as inheritable defaults for the new [env.] configs. i.e. you could do this

I keep going back and forth on this – I understand that having to specify type or account_id every time is annoying, but I also feel like having inherited defaults is an implicit and maybe confusing design choice – I think I might feel less confident about knowing what my config will be when I look in wrangler.toml. Do you think that's a valid concern?

Question around the "Configuration that requires explicitly specifying environment" section, @EverlastingBugstopper:

[env.prod]
name = "worker"
type = "webpack"
zone_id = "12345abcd"
private = false
account_id = "12345abcd"
route = "https://example.com"

[env.staging]
name = "staging-worker"
type = "webpack"
zone_id = "12345abcd"
private = false
account_id = "12345abcd"
route = "https://staging.example.com"

This configuration would allow you to publish zoneless with wrangler publish, and wrangler publish --release would ask you to specify an environment to deploy to. You could run wrangler publish --release --env staging, wrangler publish --release --env prod, and wrangler publsh --release --env staging --env prod.

Wouldn't we want to support environments for zoneless deploys as well? wrangler publish --release --env $env makes sense to me, because there's no implicit/default environment defined. Why/how would wrangler publish work any differently? Does it assume one of the envs? IMO if you have the above TOML config, wrangler publish _and_ wrangler publish --release should require an env, what do you think?

[ashleymichal]

imho if you supply an env, it should be _either_ a workers.dev deployment _or_ a zoned deploy, not both, and we should move toward deprecating the --release flag, since that was sort of an "env" argument to begin with.

[signalnerve]

imho if you supply an env, it should be _either_ a workers.dev deployment _or_ a zoned deploy, not both, and we should move toward deprecating the --release flag, since that was sort of an "env" argument to begin with.

Agreed! I commented on #388 after this ticket and found myself thinking something similar.

[EverlastingBugstopper]

While I agree that --release is less than ideal I think we want to keep this reverse compatible which is why I wrote things this way. That being said, I think maybe a good way to do this is to have --release be an alias for --env and if there is nothing passed to --env (or --release) then deploy to the unlabelled configuration, and then you could pass aliases to --env or --release. This would prevent the ugly --release --env staging API I proposed.

[ashleymichal]

there are acceptable paths for deprecating flags that i think we should employ, such as retaining the --release flag and printing a deprecation warning to the console correcting to the updated use and indicating the release that will remove the flag entirely.

Beyond that, in the case of the updated usage, a default would be nice to have as well; that is, env.default should be used whenever an env is not passed. To preserve backward compatibility we can lump any top-level attributes into that env, and again print verbiage to the screen indicating what is happening. Docs and output can supplement best practices, such as using env.default as test/dev settings and passing specific envs for "prod" etc. This would, again, necessitate a multi-release deprecation path like the one described above.

Strawman for output:

# without flag
$ wrangler publish
No `env` provided, falling back to default values. 
🕵🏻‍♀️ Learn more about configuring wrangler envs: [link]
Publishing myWorker to default env: 
* myWorker.a.workers.dev
<normal publish output>


# with flag
$ wrangler publish --env=staging
Publishing myWorker to staging env:
* tinyhammer.horse/myWorker
<normal publish output>

[ashleymichal]

based on earlier conversation around routes/release flag, we should also enforce that different environments have unique names to prevent confusion when updating your "staging" scripts changes what happens on your "prod" scripts because they have the same script name. I think that this feature will need considerable thought put into how we establish and communicate "best practices" to prevent surprises.

[EverlastingBugstopper]

Love the idea of deprecation warnings and documenting best practices! In your example you have wrangler publish "falling back to default values" which happens to be a zoneless worker. What would that wrangler.toml look like? Currently wrangler publish will publish to zoneless worker, which makes sense, but your comment also said that any top-level attributes would be lumped into env.default. Top-level attributes today are only deployed on wrangler publish --release

[ashleymichal]

here are the notes from our meeting; there's a couple of example tomls (old and new) and some proposed UX flows, BDD-style

old wrangler.toml

name = "worker"
type = "webpack"
zone_id = ""
private = false         // deprecate this
account_id = ""
route = ""

proposed new toml

[project]
type = "webpack"

[env.default]
name = "worker"
zone_id = ""
private = false         // deprecate this
account_id = ""
route = ""

[env.a]                 // a zoned env deploy
name = "worker-a"
zone_id = ""
account_id = ""
route = ""
[[kv-namespaces]]
binding = "foo"
id = "stagingnamespaceid"

[env.workers-dot-dev]   // a workers.dev env deploy
name = "worker-a"
somethingforworkersdotdev = true
account_id = ""
[[kv-namespaces]]
binding = "foo"
id = "prodnamespaceid"

behavior based on context:

context: Wrangler 1.2.0
  context: old toml 
    -> wrangler assumes top level values are "default"
    -> wrangler behaves like it always did (--release goes to zone, none goes to subdomain)
    -> wrangler outputs a warning, suggests action
  context: new toml 
    context: no --env flag
      -> wrangler looks for [env.default]
      -> wrangler asserts unique `name` field for each env.
      -> wrangler can handle route and workers.dev deploy per env.
    context: with --env flag
      -> wrangler looks for the correct env and throws if it is not found
      -> wrangler throws if you also pass --release

example output for old toml -> --release flag:

$ wrangler publish --release
Warn: the --release flag is deprecating, learn more about Wrangler environments: [link]

next release
-> warn
-> update templates
-> update generated toml
-> update tutorials that reference release flag

[EverlastingBugstopper]

For wrangler generate I'm assuming our wrangler.toml will look like this

[project]
type = "webpack"

[env.default]
name = "worker"
zone_id = ""
account_id = ""
route = ""