Osu-web: Allow client grant API access

[stanriders]

Going through client auth process isn't convenient for serverside services, but currently the only API client that is allowed to use password grant type is lazer and iirc using lazer id/secret isn't allowed.

[peppy]

We were discussing this today, and the general consensus is to do it similar to github:

• Create oauth application
• Authorize as yourself then access endpoints using an (implicit) public scope.

There are still two steps involved here, but does this sound okay to cover your usage needs?

[stanriders]

Creating an application is not much of a problem, but i'm not sure what you mean by "authorize as yourself". If accessed data won't be tied to my account is there a need for authorization at all?

[peppy]

that's basically what we were discussing. the way oauth works, there are two ways we can go forward:

• use the current implementation and have api users which don't care about authing other users just authing once as themselves to use the api (github way). even though you are authing as yourself, as you are in control of what data you use, it's very unlikely you would reveal anything personal.

• add a new method of selecting what kind of application you are when you create the application, skipping the "self auth" step and making it automatic

method 1 is already implemented our end, but requires one extra once-off step from your end

method 2 would require maybe a month or two of development to make possible, but remove that step. conversely, for an application that wanted to auth users and access the api, it would need to register twice as different applications.

we are leaning towards method 1 right now. feedback is welcome.

[stanriders]

First method sounds good, but authing once means token wont ever expire? Then if token somehow gets leaked it'll be much more severe than leak of v1 api key because v2 gives much more info than v1. Second method will make sure that even if token was leaked then it wont give out private info.

Second method sounds more secure and less "hacky" to me tbh. I think you should go that way and since v2 api doesn't yet have enough endpoints (at least all of the v1s) time isn't really a problem

[peppy]

Both tokens will provide relatively similar access. The "public" scope given in both cases will not reveal anything which is not already public information. If you make an application and do not grant specific read permissions, it will not reveal anything about you (the api consumer) that is not already available.

[nekodex]

If your app's secret gets leaked, you can remove the OAuth application (from your account settings) which will invalidate all the tokens associated with it.

(might be nice to add a way to reset the secret without requiring the deletion/recreation of an app though, created an issue to track this #5799)

[omkelderman]

I also was looking for an easier/more straightforward way for serverside applications that dont have a login and dont deal with logged in users (and thus dont have access tokens for said users) to do api requests. Just like api v1 behaves.

I started looking into the laravel passport documentation (that is what osu-web is using right?) and I saw https://laravel.com/docs/7.x/passport#personal-access-tokens. Unless im misunderstanding something, this to me looks like the perfect solution? If there was a possibility to create such personal access tokens with just the public scope, then I think that would be perfect.
I suppose maybe something on the same settings page where you can register oauth clients now? If needed I can look into working on a PR for that.

_(small sidenote @peppy this is what I was talking about in https://github.com/ppy/osu-api/issues/304#issuecomment-631523870, might not have communicated that very clearly)_

[peppy]

Have you read and understood the content of this thread?

Have you attempted to use the existing flow? It's actually not hard once you have it setup, and can serve the purposes you need.

[omkelderman]

I know, I've done it before, I am simply just suggesting an alternative.
If that alternative is not desirable, then I'll do it the other way. I do personally believe however that I am not requesting something super unreasonable, but I guess we differ in opinion about that.

[ThePooN]

This issue has gone under my radar, but I'll still add my grain of salt to this, especially to answer @peppy's March 18th second comment.

Sorry for the wall of text. It is probably fair to say I have a strong connection with the osu! API. I want APIv2 to be as good as it can, and as users expect it to be (especially current v1 users, but not only).

Introduction

I believe the method one you described, which is the current way APIv2 works and the one you're learning towards keeping for this use case, is not the intended method defined by OAuth. It's also very inconvenient for accessing public data in server-to-server communication.

Privacy

Using user-bound tokens for accessing public data is just... not the right way to do it.
None of the current APIv1 use cases involve private/authenticated-user-bound data. All of them should use an equivalent to the API Key, which in OAuth, are access tokens obtained using "Client Credentials Grant" (client id + client secret credentials). These are the right tokens to use for server-to-server communication not involving personal data. They are very well described in section 4.4 of RFC6749.

Major companies that I often integrate with, such as Discord and Twitch do follow this standard as well (except for Twitch bots, as they're normal user accounts, like osu!, and unlike Discord who are doing it the right & best way).
All API services enforcing user-bound tokens for public data access are wrong. It requires giving unnecessary access to user data. Even if you can get a token without giving any personal data-related grant, it's an unnecessary security risk (if a grant isn't checked correctly against, for example).
If GitHub doesn't follow this, it's a shame.

even though you are authing as yourself, as you are in control of what data you use, it's very unlikely you would reveal anything personal.

This may be valid for professional apps, but in the osu! community, not everyone can afford self-hosting. In fact, I have provided myself parts of my personal infrastructure for projects I don't run in the past.
Some major projects (such as Tillerinobot) have shared maintenance as well (by @omkelderman, actually).

Also, if a security breach occurs, you will be tempted to revoke some/all user-bound tokens. This is good practice. But you will also unnecessarily break applications that unnecessarily rely on user-bound tokens, because you're enforcing that single method of authentication.

And finally, beyond privacy, there's ease of use.

Ease of use

Currently, all you need to use APIv1 is an API key. There are a ton of libraries that provide you interfaces where you just feed it the API key, and then all requests are ready-to-use, with all the HTTP/JSON code handled for you. nodesu is one of them, and made my first API experiences ever very painless.

This ease-of-use got many people more into programming. Myself included. I have been an user and have become a maintainer of many osu!-API related projects. I would probably not have been as passionate about programming, and couldn't have given back so much in return to the osu! community, if I had to go through the current OAuth system.

Same goes for Bancho's IRC interface. It's not complex to authenticate against and can also be used for people to learn API programming. I've actually spent over an hour earlier today helping a fairly young person learning the Node.js event pattern, using my bancho.js library. With the method you're currently leaning forward with, this kind of users wouldn't have gotten anywhere.

OAuth Authorization Code Grant is very complex to integrate for beginners, even with the right and many OAuth abstractions out there. We shouldn't have to go through user web-based authentication for a server-to-server application (such as Discord/Bancho bots, statistics tools (GSheets scripts are a very common osu! API use case), or very simple web apps). It makes these projects so much more unnecessarily complex. Everyone working using the osu! API is doing so in their free-time, and will likely not be able to commit to learning OAuth. Many of them are amateurs.

OAuth Client Credentials Grant is also more complex than APIv1 API keys, but if abstracted, is as easy to use as current API keys for the library user. It is entirely automated with no user interaction/authentication, as long as the Client-ID & secret couple is provided. They are the nearest equivalent to current APIv1 keys, and the right tool for this use case.

Client Credentials Grant is implemented by Laravel Passport and doesn't involve method two you described. It is supposed to work with regular, existing applications. Sadly, this grant type is currently reported to be disabled by osu-web.
I cannot recall seeing a service working like method two. I doubt that method is OAuth-compliant.

Conclusion

You have answered "Sounds like you're just being lazy" to @omkelderman when he mentioned this issue in ppy/osu-api#304. To that, I answer that requesting the right tool for the right task is not laziness. It will only result in wasted development & maintenance efforts, security compromises, for everyone willing to use APIv2 in a server-to-server scenario.
More and more people will also have to use APIv2 since you've started denying feature requests, even the most simple ones, like ppy/osu-api#304. I don't believe this is the right thing to do, when you don't provide the right tools to cover the only use cases of APIv1. This should be top priority.

Thanks for your consideration!

PS: this issue has turned into a debate over server-to-server communication since its first comment. It should probably be renamed.

[peppy]

As a heads up, the "security" concern is not one as we have introduced scopes. For your server-side token you can choose to give it zero scope, which will still allow accessing public data without getting anything related to your own user, I believe (@notbakaneko please confirm).

The ease of use is definitely a consideration, but one which is pretty weak in my opinion. There are oauth libraries out there for every programming language around which simplify the process to just inputting two values. The only extra step is getting the initial tokens, which maybe we could add as an automatic step in the settings UI?

[cl8n]

For your server-side token you can choose to give it zero scope, which will still allow accessing public data without getting anything related to your own user

this is exactly why the client credentials grant type exists, why not use it? more standard, less confusing, no need for the initial workaround to get access tokens, already implemented in laravel.

Have you attempted to use the existing flow? It's actually not hard once you have it setup, and can serve the purposes you need.

personally: yes and while it wasn't "hard" to get the tokens (may not be true for someone unfamiliar with oauth), it's definitely not how they're supposed to be used. if all you're using an access token for is public data or information about the client itself, then authenticating a user via auth code is an 100% useless step and pretty misleading too

[ThePooN]

This following point regarding security still stands:

Even if you can get a token without giving any personal data-related grant, it's an unnecessary security risk (if a grant isn't checked correctly against, for example).

If such access tokens and refresh tokens expire, they also still require manual and unjustified user maintenance.

The ease of use is definitely a consideration, but one which is pretty weak in my opinion. There are oauth libraries out there for every programming language around which simplify the process to just inputting two values. The only extra step is getting the initial tokens, which maybe we could add as an automatic step in the settings UI?

Honestly, I don't think you realize how hard OAuth can be to understand for a beginner, especially as the authorization grant is definitely not the easiest to grasp... even more as you're asking everyone to misuse it. This is so confusing!
If you implement it the right way, not only you will make APIv2 way easier to use, but also a good way to learn OAuth for beginners.
People will thank you later, osu! is not only a great game, a great community, but also an excellent way to learn programming, both through osu!lazer and the osu! API. What I learned through APIv1 alone, and OAuth on other platforms, has been very helpful in real-life experiences.

[peppy]

client grant is definitely something we will look at, but the current system is enough to continue building out the API (which is not yet done, surprisingly).

anyone is free to try adding support and we'll try and accommodate it into the ecosystem if it's simple enough.