Vscode-restclient: Ability to store Bearer token in a variable automatically

[agjs]

This is not an issue but rather a feature suggestion. Since pretty much any serious API has authorization and security, it would be awesome if we could somehow store a bearer token for example into temp memory or some variable and have it in others requests accessible like {{token}} or similar. This process should be automatic (e.g. a token would be extracted automatically).

Currently, we need to do the login request, copy the token and paste it manually to other requests and since tokens have expiration time, this can become quiet annoying.

Also, thanks a bunch for the awesome work you are putting in :). Cheers

[Huachao]

@agjs it seems the custom variables feature already meets your requirement, that means you can define a variable in the setting file for specific environment, and use it in your http file. The detailed step is like below, you can also view README.md.

First, you should create custom variables in the vscode settings file, like following:

"rest-client.environmentVariables": {
        "dev": {
            "token": "test value"
        }
    }

This piece of code adds a new environment named dev with a custom variables token. The environment name and variable name is whatever you like.

After that, in your http file write the request as before:

GET https://example.com/connect/token HTTP/1.1
Authorization: {{token}}

Finally, switch to the environment named dev as defined in setting file, to switch environment just press Ctrl+Alt+E(Cmd+Alt+E for macOS), or press F1 and then select/type Rest Client: Switch Environment. Or even simple to click the No Environment button in the right bottom status bar.

[agjs]

This is partially true. Yes, I know we can create custom variables but I don't want to paste token manually every time there, I want that token to be automatically stored, that's the difference.

[Huachao]

@agjs do you mean make the extension to extract the token from the login response? Can you show me a sample response?

[agjs]

Yes, pretty much.

Imagine I post to /auth/login and I get a token back in a JSON object or as simple string. That token would then be automatically extracted from that response and stored either in a global variable e.g. {{ token }}, or anywhere else. Doing this, our {{ token }} variable would automatically be populated each time we login (each time new token is generated) and we wouldn't need to worry about our token being valid or invalid.

[agjs]

If this could be possible, in a configuration, we could have something like this:

"rest-client.environmentVariables": {
[someEnv]: {
"extracted_token": "token extracted from the login request"
}
}

GET https://example.com/api/protected HTTP/1.1
Authorization: {{extracted_token}}

This would be super useful feature since the process would be fully automated and no manual copy/pasting would be required.

I assume the example I provided wouldn't work in such form since it would relate file system and writing to a file but coming up with something easier and more efficient shouldn't be hard.

[Huachao]

@agjs the problem is that how can I know which response I need to extract, and which part of the response(e.g., which header) to extract?

[agjs]

@Huachao Since most of the API's today are using JWT's, JWT libraries mostly expect signature like this: Authorization : Bearer + token. That being said, we could either allow in the configuration a user to specify from which header key/value pair his token comes or if that's extra work, simply specify by default Application: Bearer {{token}}.

Most libraries I'm using or the ones I previously used follow this set of rules.

[ahmelsayed]

@Huachao I was about to open a similar issue but with a different proposal. I'm also happy to help if you're open to contributions :)

You can just support running an executable or a script that would return back a jwt token for OAuth authenticated APIs. Kinda like git credential helper if you've ever used that before.

It can be more general than jwt tokens, but the basic idea is the same. To authenticate any API all you'd have to do is write a script or program that knows how to do that, then it'll just return the token as plain text over stdout. That token can then be used by the extension for every request or just added to the current environment as a variable. What do you think?

_ps: I love the extension. Thank you very much! I used to use a similar extension for emacs and was very happy to find this one for vscode_

[Huachao]

@ahmelsayed Thanks for your suggestion, and any PRs and issues are warmly welcomed. And I still have some questions that like to discuss with you:

1. It seems that the task for retrieving the auth token by a script is a separating job for our extension, we can let user manually do this job, and add the result into my extension as an environment variable. Since if it's a one time like job, this seems low cost.
2. If what the script did is a very general pattern, can this logic be moved to our extension, and user just need to prepare some input? Let user write another script or program seems not user friendly.
3. The script will become another place to send some REST call, user seems not easy to maintain all the APIs in one place(the .http file).

Thanks again for your idea.

[ahmelsayed]

Answers:

1. True, this is what I do now. However if the token is short lived (in my case the JWT token is only valid for 1 hour and refresh token for much longer, but you need to keep refreshing the JWT token itself once an hour) that means users have to keep running their script for a new token. Even if it's not really short lived (say 1 day or so) you'll effectively need to keep doing that every time you wanna use the extension, which may not be too bad. _I guess I can keep the script running in the background to keep updating the settings file with a new token every hour._

2. OAuth login has a dance that the browser usually performs. Sometimes the login has to be interactive or requires javascript for things like 2 factor-auth, captcha, etc. These login providers, like Azure Active Directory, Facebook, Google, etc, provide various SDKs for non-browser login flows. It would be nice to support all of them obviously, just thought it would be easier to add something more generic.

[Huachao]

@ahmelsayed great suggestion. And I think for refresh access token, can this be done with single http request within my extension, I mean what extra work did for the script?

[stojce]

I would suggest to have section under body that can run some basic scripting, e.g. regex matching and store results in temporary session variables - until vscode is closed. Also ability to view and delete those variables would be nice.

[stojce]

Now with 0.15.0 values from script can be stored in local variables. Example of use would be like this:

@jwt = ""

// Login User
POST http://{{host}}:{{port}}/api/v1/users/login
Content-Type: application/json

{
  "username": "admin",
  "password": "P@ssw0rd"
}

//Response body
//{
//   "id": "1",
//   "token": "1234567890"
//}

// script part
@{
const pattern = /"token": "(.*)"/;
jwt = $response.body.match(pattern, $1);
}

// Twitter Seciton

// GET list of Twitter Users, use jwt parsed from previous call
GET http://{{host}}:{{port}}/api/v1/twitterusers
Authorization: {{jwt}}

[Huachao]

@stojce my extension doesn't support run script, and it won't save value to variable. How can you achieve that?

[stojce]

My suggestion is to introduce scripting. Script part can be parsed by @{ ... } pattern, and contents can be evaluated. Need to add context to script eval, so the script can have access to global and local variables.

[Huachao]

@stojce nice suggestion, and I will try to implement this when I consider this carefully

[msageryd]

I've played with the plugin for 15 minutes and I'm already in love. I'll probably migrate from Postman soon. At least when token extraction is in place :)

The token extraction idea is great. I would love it. I'm developing an API which I often test with a couple of different user accounts (to test different access roles). Postman has a way to get hold of the token and put it in a global variable which makes this easy to do.

I think the best way would be via scripting. I would love to have a script that populates two or three different token variables (@user1token, @user2token, etc).

[mrchief]

Token extraction would be a great addition. AFAIK, only Runscope currently has this ability. Its not trivial, but if we can get this, then this extension will become a must have!

[lucsky]

This is probably the single reason why I am currently using Paw (https://paw.cloud) on mac, it allows to define variables which values are automatically extracted from other requests response bodies. It's very powerful and extremely convenient.

[flanakin]

Fwiw, I just submitted a PR to add support for Azure AD explicitly. I opted for explicit support due to no experience with other auth providers and support for critical AAD-specific scenarios.

[aphilpotts]

For our purposes, we could handle our Bearer tokens if #140 became real feature, as they are returned in plain text in the JSON returned from a login request.

If we used something more complex, like OpenID, then I doubt that #140 would be enough

[guycnicholas]

We currently use Postman and we use the "tests" functionality to run some JS code. For example ours looks like this

var jsonData = JSON.parse(responseBody);
postman.setGlobalVariable("accessToken", jsonData.access_token);
postman.setGlobalVariable("userId", jsonData.userId);

perhaps something generic like that could be done?

[Huachao]

@guycnicholas this is not supported currently, but is in my roadmap.

[lzp4ever]

How about that, variable without being initialized could be set under the request body.
And I think it would be magnificent to run all request in a file with one click.
Like get token at the first request, and then use the token to finish other request below.
An elegant editor-based automated testing tools!

@auth_token

###
POST https://localhost/login
Content-Type: application/json

{
    "username": "username",
    "password": "password"
}

// from whole body
// Bearer xxxxxxxxxxxxxxxxxxxx
{{auth_token}}=response.body

// from part of json body
// {
//    "token": "Bearer xxxxxxxxxxxxxxxxxxxx"
// }
{{auth_token}}=response.body.token

// from header
// Authorization: Bearer xxxxxxxxxxxxxxxxxxxx
{{auth_token}}=response.header.Authorization

[Huachao]

@lzp4ever, this is similar to what we are doing in #140

[Huachao]

@agjs @ahmelsayed @ndinev @jPrest @thehighlander @chanan @agjs @andyzaharia @neilsmind @lzp4ever @guycnicholas @stojce @borekb @mrchief @lucsky @flanakin @msageryd @jamisliao
@jacob-beltran @eppyjerk @liuxuc In our new version 0.18.0, we introduced the new feature request variable, with this you can achieve something you want in this thread, that you can reference part of the response in another request, although there's still much work towards target of this thread. For more usage details can be found in README. Waiting for your feedback.

[mrchief]

This is great news @Huachao ! Thanks for all the good work!

Will this work across files? Guess I’m being a lil greedy here. 🙃

[Huachao]

@mrchief it's not supported yet, I will consider this

[lushibi]

cool, however, if the expression can be assigned to a temp variable name, it would be more convenient.
e.g.

Authorization: {{login.response.headers.X-AuthToken}}

can be changed to

@token = “{{login.response.headers.X-AuthToken}}”
...
Authorization: {{token}}

or

#@name token = login.response.headers.X-AuthToken
...
Authorization: {{token}}

or something similar

[loverde]

Moved comment to a separate feature request #238

[Huachao]

@lushibi @hudecekd @agjs @ahmelsayed @ndinev @jPrest @thehighlander @chanan @agjs @andyzaharia @neilsmind @lzp4ever @guycnicholas @stojce @borekb @mrchief @lucsky @flanakin @msageryd @jamisliao
@jacob-beltran @eppyjerk @liuxuc @loverde In latest version 0.20.2, you can experiece the feature that value of file variables can be constucted with request variables. The example provided by @lushibi can be supported now:

@token = “{{login.response.headers.X-AuthToken}}”
...
Authorization: {{token}}