Deno: Support access token authorisation for remote files

[kitsonk]

Currently there is no way to fetch remote modules from private GitHub repositories without exposing your access token.

For example you could do the following currently:

import * as foo from "https://[email protected]/private_org/private_repo/master/foo.ts";

But you could easily "leak" your access token if you then checked that code in and pushed it to a public repo.

GitHub (and I assume other services) all the token to be passed as an authorisation header:

Authorization: token $TOKEN

If this was somehow passed on the command line, it would become easier to secure.

[pomke]

This is a show stopper for Deno at many companies who need to use private modules and have a policy of not embedding auth tokens into their software.

[kitsonk]

What we need is a decent proposal of what the UI should be for passing it. It should be easy to specify multiple tokens for different hosts, and only when connecting to those hosts, will the header be sent. Maybe [TOKEN]@[HOST] would work? So on the CLI it something like:

$ deno run --auth-token [TOKEN]@raw.githubusercontent.com https://raw.githubusercontent.com/private_org/repo/master/main.ts

Or:

$ deno run --auth-token=[TOKEN]@raw.githubusercontent.com,[TOKEN]@example.com https://raw.githubusercontent.com/private_org/repo/master/main.ts

[Spoonbender]

I like it.

Some points:

• Tokens can be quite long, and command lines do have a length limit, so i think there should also be an option to set them in a file (resembles .npmrc)
• Tokens should also be settable via environment variables, to make life easier ehen integrating with build pipelines. This could be trivial for command line, but requires some more effort for files (finding and replacing placeholders)
• What kind of credentials to support? Just bearer tokens, or also others like basic auth etc.?

[kitsonk]

We have avoided meta data files. I would loath to add one for this. Using an environment variable like DENO_AUTH_TOKEN with the same comma separated format should address the length problem. As far as other auth, personal opinion is we keep it to tokens. Basic Auth encourages bad security practices and I am not aware of something professional grade that doesn't support tokens for things like this. It could be an enhancement down the road if there is a legitimate use case.

[benkeil]

I like the way node was going with something like a .npmrc file. Think also about how to build the code in a CI server. Maybe there's a way to put inside the import map file.

{
  "imports": {
    ...
  },
  "authentication": {
    "github:denoland": "${GITHUB_ACCESS_TOKEN}",
    "github:denoland/deno": "${GITHUB_ACCESS_TOKEN_2}"
  }
}

Without that feature Deno in unusable for us, because all our libraries are in private GitHub repositories.

[qballer]

This is a brilliant idea and would really assist when modules are hosted on private registries. I really like the import map file suggestion as to where to place this config, as far as I can tell, the only problem is, this is nonstandard AFAIK from here. I suggest that any programmatic API will also include authentication tokens and then the community can build there own tooling around it.

[kitsonk]

a way to put inside the import map file

Import maps are based on a draft standard. Modifying that will end up in 😢. There is this issue #3179 for that, but I don't think it will go anywhere any time soon. The feature can easily ship without solving that problem too.

[maciejmaciejewski]

For now everything in this thread is around GitHub, I would strongly suggest to also support other repositories like Azure DevOps, BitBucket, etc.

[kitsonk]

@maciejmaciejewski they support auth tokens as well...

• BitBucket
• Azure DevOps
• Gitlab
• Cloudsmith
• etc...

GitHub didn't invent the standard.

[maciejmaciejewski]

@kitsonk What I meant is all of those can handle the Authorization header in a different way - some can use Bearer token, some other Basic one so just want to make sure that all the use cases are covered.

[benkeil]

We could use providers for authentication like github:denoland or azure:denoland. Deno needs to recognize that an import from e.g. https://raw.githubusercontent.com/private_org/private_repo/master/foo.ts is handled by the corresponding provider.

[pomke]

But where do you end with adding 'providers'? Now there's a magic list of 'providers' who are 'important enough' ? This doesn't seem like a scalable solution. Also +1 for avoiding more dotfiles, most javascript repos have more dotfiles than codefiles these days :crying_cat_face:

[kitsonk]

We need to get it implemented on the command line and with environment variables first, and iterate from there. We don't need to solve every problem at the start.

[pomke]

One issue I see with the [TOKEN]@[HOST] solution is needing to provide tokens for multiple private github projects at the same host. I don't have this issue right now but I cam imagine someone trying to provide a token for two separate private repositories both on the same host (github). The HOST would need to include some path components to avoid conflict?

[kitsonk]

@pomke the tokens are tied to users, not repos... it would be a limitation that the same user has to have read access to all the repos. That is a limitation that seems reasonable.

[keithamus]

As most providers use the Authorization header, one option is to simply provide the raw header value as an environment variable i.e. the env var would include the Bearer or Basic portion of the string.

[benkeil]

For GitHub it’s enough to have an environment variable. Deno just needs to Respekt it.

[silverwind]

I'd suggest something like this:

json "importOptions": { "headers": { "authorization": { "github.com/denoland/deno": "Bearer ${DENO_TOKEN}", } } }

authorization can be replaced with any custom header name. The keys inside authorization could be tested against each import URL and the first partial substring match should be used. In the same vein, one could implement TLS client certificates for import:

json "importOptions": { "clientCertificate": { "github.com/denoland/deno": "${CLIENT_CERT}", }, "clientCertificateKey": { "github.com/denoland/deno": "${CLIENT_CERT_KEY}", } }

The data could also be expose to JS using import.meta.options.

[JerryGreen]

My suggestion is to add -n or --netrc (or similarly named) option, which will handle credentials placed in ~/.netrc, just like curl:

Usage: curl [options...] <url>
 -n, --netrc         Must read .netrc for user name and password
     --netrc-file <filename> Specify FILE for netrc

In ~/.netrc you'll have this:

machine   raw.githubusercontent.com
login     YOUR_TOKEN
password  x-oauth-basic

(YOUR_TOKEN is a personal Github token with "Full control of private repositories")

Now you can easily:

curl -n https://raw.githubusercontent.com/private_org/private_repo/master/foo.ts

Wouldn't work if you won't pass -n option though:

curl https://raw.githubusercontent.com/private_org/private_repo/master/foo.ts # will return 404

Just try it with Github and curl, it's very neat.

Would be cool if it worked the same way in Deno.

[bartlomieju]

Thanks for suggestion @JerryGreen but we're very careful about introducing configuration files and we don't pick them up automatically. I think this problem problem should be solved as part of single "metadata" file:

• #3675
• #3179
• #5897

[lucacasonato]

@bartlomieju We should not put this in the metadata file, because metadata file will be shared between multiple people. This config should be per user.

I do like the idea of netrc. It is relatively standard, and existing parsers for it exist in Rust - we wouldnt have to invent anything new. We wouldn't pick up the file automatically though, (instead you have to explicitly specify the path to it with a flag, as environment variable, or as a key in the metadata file).

[silverwind]

Keep in mind .netrc will probably not be able to support other forms of authorization, like client certificates. Also, storing passwords in plaintext on the disk may be a security risk.