Gitea: Support federated pull requests

[stevenroose]

_From @stevenroose on May 25, 2016 11:24_

Currently, users can only make pull requests if they have an account on the same Gogs instance. It should also be possible to make pull request from external repositories like GitHub or other Gogs/GitLab repo's.

This could be integrated with https://github.com/gogits/gogs/issues/1297 and https://github.com/gogits/gogs/issues/3130.

_Copied from original issue: gogits/gogs#3131_

[stevenroose]

_From @roblabla on May 25, 2016 12:9_

Somewhat related : https://github.com/gogits/gogs/issues/2210

[stevenroose]

This is the number one feature for a personal hosted Git service!
Would be even greater with #185 !

[ekozan]

This could be integrated with git-appraise integration too ?

[strk]

@ekozan formal proposals are welcome, but I do see git-appraise integration could be a good companion for federated pull requests (to basically have reviews travel across federated nodes with the rest of the code, right?)

[stevenroose]

GitLab is tracking this issue here, so maybe you could word out a streamlined workflow. They are planning to mention the feature during their summit.

[strk]

@bkcsoft maybe you can help with keeping the GitLab specs open enough to allow for federating PR between GitLab and Gitea too ?

[stevenroose]

@strk is @bkcsoft affiliated with GitLab?

[bkcsoft]

@strk I could steer it in the direction of just sending patch-files between servers (maybe using webhooks?). Which is what I suggest Gitea should do as well. Makes it _really_ easy not having to push/pull between servers :)
@stevenroose Yes

[bkcsoft]

Well, seems like they're already thinking or using git request-pull -p (which sends the patch along) so it should be cross-platform compatible :slightly_smiling_face:

[bkcsoft]

They are planning to mention the feature during their summit.

https://gitlab.com/gitlab-org/gitlab-ce/issues/4013 is tagged as moonshot, unassigned, no milestone and no MR in sight. So maybe not get your hopes up just yet :slightly_smiling_face:

[stevenroose]

@bkcsoft We might take the lead here :) If we can get GitLab and GitHub on board, that would end the locking currently imposed by these platforms

[tboerger]

@bkcsoft We might take the lead here :) If we can get GitLab and GitHub on board, that would end the locking currently imposed by these platforms

I can't believe that GitHub wants to solve the lockin issue :P

[stevenroose]

No but if other platforms lead the way, people might demand they follow. And people might migrate :)

Software like Gerrit kind of allows for that

[strk]

@stevenroose do you have reference about the Gerrit support ?

[lunny]

If we implements Gerrit, could we invite Golang team to use Gitea? 😄

[stevenroose]

@strk With Gerrit, you package your commits using git-review and push them to a certain ref and it will show up in Gerrit as a code review. No need to create a Gerrit fork. It does not use git request-pull though.

[strk]

You'd still need write permissions to the repo to push those
reviews to the ref, right ?

In that case the missing bit would still be eventually tracking
a different repo holding the review ref ?

[stevenroose]

Yeah it is different. It pushes individual bits with write permission.

With federated PRs, Gitea should periodically (or on request) check the branch reference for new changes.

[roblabla]

AFAIK, git request-pull does not use git commits at all. It merely generates a list of commits between local and remote, and print it on stdout. We'd need to specify a way to send those changes to remotes/to pull them from remotes

Git request-pull is part of the standard git install though, unlike git appraise or git review.

[stevenroose]

@roblabla Yeah the flow would be to save the git request-pull to a file and upload that (like how submitting patches used to work in the past). Or either enter a URL to a branch of a remote repo so that Gitea can update the PR.

[strk]

Unless I'm missing something git request-pull references a specific commit,
while github/gitea pull requests reference a (possibly moving) branch.

Do we want federated request to track branches rather than specific commits ?
I think we do want a PR (thread of comments/discussion) to track a branch.

[bkcsoft]

it references specific _commits_ yes. So we'd need to continuously re-fetch the data 😒

[strk]

On Tue, Dec 20, 2016 at 12:20:34AM -0800, bkcsoft wrote:

it references specific _commits_ yes. So we'd need to continuously re-fetch the data 😒

We can only fetch new commits if we have a reference to a branch name,
which is not in git request-pull output. So if we want to keep the
track-a-branch approach, we cannot rely on git request-pull.

A PR creator could ask for a remote URL and a branch name, check out the
remote branch locally, perform some checks (ie: refuse to create PR with
a huge number of changes), and create the PR.

Then a button and a webhook could be then made available to fetch more
commits, if any, from the remote branch. Only the PR author should be
given the ability to request updates. Placing a git hook on his fork
to hit the destination webhook would make the update experience smoother.

--strk;

[stevenroose]

I would suggest that when creating a PR as a guest account, you'd have a tab with "external" or "federated' or whatever that has two options:

1. a text input field for a branch URL, with a "fetch" or "test" button maybe to check for reachability and availability of tracking

2. a file upload field to upload a .diff, .patch file or the output of a git-request-pull; this might also be a textarea

In the latter case, once the PR is created, users should be able to overwrite their previous file in order to change the commits in the PR.

Also, in the case of unavailability of automatic branch updates, you could require a user to manually trigger a refetch. In almost all of the cases, users commit to a PR'ed branch exclusively in the light of the commit, so they can just refetch whenever to update the branch. (Take into account that often, updates to a commit are because of feedback in the PR, so they have the page open anyway.)

As my thesis promotor put it: "The real opposition to a change does not come from people who are against it but from people who keep saying it's not enough."

[renothing]

I don't think this feature usefull, you can add two remote for this case. for example branch github for gh, master for gogs, I use both in my work with one repository. and git review is another things. don't make big feature to solve little problem.
anyway, gogs focus on small business or self hosting. not for public cloud service.

[stevenroose]

@renothing I don't think you fully understand the proposal. It has nothing to do with being able to compare or checkout code from different remote sources. Instead, it is about allowing people to submit pull requests or patches without having to be registered to your system. If I publish a piece of software on my Gitea instance, I want others to be able to create a PR without requiring them to go through the hassle of registering, forking, pushing their changes to my instance and then creating a PR.

[renothing]

@stevenroose
as I said, it's quite a little demand scene. think about it, how many people need merge a PR from gist? I don't think it's necessary to add this feature to gitea core. maybe it's ok for a plugin when gitea plugin architecture ready.

[sbrl]

@renothing If I understand the issue being tackled here correctly, it goes far beyond merging just a simple gist. It tackles merging between multiple gitea different instances, and even between a GitHub repository and one on a gitea instance. It allows one to merge changes from any remote branch on any supported git server on the internet.

@stevenroose Correct me if I'm wrong :smiley:

[renothing]

@sbrl
you're right, but if somebody want to do contribution for your repo, he will need fetch your code first, he must have to keep two remote(one for himself, one for yours). it didn't reduce works at all. this feature just reduce login/register step. it didn't help too much, maybe send email path is better choice like linux kernel.

[ekozan]

i'm not ok with you @renothing

1. When you work with git you have always many remote
2. It's a must have
3. login step are fucking boring
4. gitlab will do it too : https://gitlab.com/gitlab-org/gitlab-ce/issues/4013
5. it's a main feature so it's not a plugin

[strk]

I'm all for having support for federated pull requests, anyway
I don't think they could replace logging in, as you still want
to somehow grant/revoke permissions to file PRs and comment on
the corresponding thread.

The login part should be taken care of by OpenID Login (there's
another ticket about that part).

[sbrl]

@renothing As far as I can tell, in actual fact it'll be the server that automatically fetches the code from the remote automatically, meaning that the dude doing the pull request just has to push to a branch on his own server, and doesn't have to have 2 remotes (Again, correct me if I'm wrong).

Besides, if you don't like it, I'm sure there will be an option to disable it - nobody is telling you that you have to use it :smiley:

[stevenroose]

Indeed. You might used GitHub, I use my own Gitea. If you want to contribute to my project, I don't want you to have an account on my Gitea instance, because you use GitHub. I want you to push a feature into a branch on your repo on GitHub, then come back to my instance, the project homepage and file a PR from your GitHub branch into the master one. Without you needing to create an account, all you do is login with your GitHub account, (maybe do a CAPTCHA) and paste the URL to your GitHub branch.

[strk]

As I maybe observed in another PR (the OpenID one) even if someone logs
in with a remote authentication mechanism (OpenID in my case, GitHub/OAuth2
in the example you make) he still needs a record onto the local instance,
if not else to be able to partecipate in the discussion about his proposed
changes, and maybe get notified by mail of answers.

So allowing to reference branches on remote git servers has mostly to
do with removing the need for a contributor to be granted permission to
and actually create a new git repository for code he already hosts
somewhere else. To give basically the freedom to host forks on arbitrary
hosters.

Maybe one day we'll get fine-grained permissions enough to be able
to have users that can send PRs but not create local repositories/forks.

[renothing]

to keep gitea core simple,I don't like this feature at all. it will make core features more complex, not only pull request system, also with permission system and hard to integration with 3rd CI system. anyway, the main authors have rights to choose.

[stevenroose]

@renothing Well, the main authors aren't the ones that should choose, it's the users that should.

On CI, I don't think that's an issue. What CI does is just pull a branch and run some scripts. Since the CI system is probably separate from the Gogs instance anyways, it really makes no difference to pull a branch from the Gogs repo or from any other remote repo.

Also, a permission system should be in place anyways. But it shouldn't be more complicated than just having an extra checkbox for "Guest accounts". You have permissions like _commit, make issues, make PRs and comment_ and then you would have three kind of users _members of the repo, registered accounts at Gogs, and Guests_. Ez πz

[strk]

The idea of a single flag for a "guest account" seems interesting.
But then no guest could be "watching" changes, right ?

You'd be filing a federated PR and not get the comments on it, unless
your email (and prefs) are known by the system.

Do you agree that a local record must be created anyway for the user ?

[stevenroose]

@strk. You would. Guests login with OpenID, so their e-mail address would be known. The user db should just make a distinction between "real" users and guest ones.

With "not member of this instance", I didn't mean that there was no record of them whatsoever in the db.

[strk]

You'd still need to know which email (and thus user) to notify upon receiving comments on a ticket, so there must be an association between a ticket (PR) and a "user" (guest or not) - this means still having a record in database, for that association.

BTW, OpenID provided email isn't necessarely trust/confirmed, so if you want to be sure about email you still need to take care of that

[strk]

Interesting design document by NotABug people with ideas about a federated git hosting service: https://notabug.org/NotABug.org/notabug/src/master/README.md

[bkcsoft]

OAuth for GH/BB/GL could be used to link once account, then if you allow it Gitea lists your remote repos and branches and you can create PRs to any repo that has a common ancestor (finding that is gonna be such a pain...). Only issue I see is that _all_ the other system (GitHub, BitBucket, GitLab) has different APIs for fetching this data, we'd need to support all or none. I say plugins :trollface:

@strk That design doc look nice in theory, but in pratice no one supports it (at least GH/BB/GL doesn't, feel free to point out someone that does, OR at least has _actuall_ intent of doing so) so in reality it's mainly trivia until at least one other has implemented it.
One might counter that with "but we could be first!", if we're first and then everyone else settles on another "standard" we're left stranded and have to take _personal hobby time_ to replace our existing flow with said other standard, possibly breaking the current flow. If we instead use some existing technique that sort-of viable, and the others settle on something we could implement that instead. But until said time I'd refrain from introducing "federation" when it's not exactly federated. Do it once, and do it right 🙂

[roblabla]

This is really a chicken and egg problem. Until someone implements federation, nobody will implement federation.

EDIT: I'd also argue that if we can federate across gogs instances, we've already made a first step towards federation.

[bkcsoft]

@roblabla Right, I'm not against pull requests across instances, what I am saying though is _use existing functionality_, such as the various providers APIs :)

[strk]

I'd prefer a generic solution to one that is bound to known API, especially from proprietary services.

[stevenroose]

I agree with @strk. Git itself is quite powerful, I would try to resort as much as possible to core Git functionality.

If federation is implemented f.e. based on Git URI's, adding UI sugar for hosting providers is not that hard. For Gitea, we use our own API's to extract information from the PR. For GH/GL/... we can either not support them, so just support copy-paste Git URI's, or implement a UI widget to retrieve these URI's from the API's. Or even simpler, map PR URL's to Git URI's.

One issue is that there is no standard way to represent a branch in a remote repo, as far as I'm aware. I think that I saw git://provider/repo.git#refs/mybranchref before, but I'm not sure how standard that is.

[strk]

On Wed, Mar 22, 2017 at 09:47:06AM -0700, Steven Roose wrote:

One issue is that there is no standard way to represent a branch in a remote repo, as far as I'm aware. I think that I saw git://provider/repo.git#refs/mybranchref before, but I'm not sure how standard that is.

I don't see how that's an issue.
Just say that in order to submit a "federated pull request" you'll have
to provide all the informations that "git request-pull" asks you to
provide ...

It doesn't necessarely be a single URL, can be a whole form too...

[sbrl]

How about asking GH / GL for their feedback on a proposed standard? Could be an opening into a wider discussion on the subject

[strk]

Sure, but we need that proposed standard first ...

[davidlt]

Pagure (https://pagure.io/pagure) system supports remote pull request, which I found useful while working on Fedora packages repository. You do need to be registered to make a remote pull request.

See: https://docs.pagure.org/pagure/usage/pull_requests.html#remote-git-to-pagure-pull-request

[link2xt]

Related: https://github.com/gogits/gogs/issues/4437

[vanitasvitae]

Any news on this? There is currently a lot of news coverage about Github being acquired by Microsoft, so federation in context of a git web-service would be super cool :)

[stevenroose]

I literally just came search for this issue a minute before I saw the notification from your comment. This (and federated pull requests) is really the killer feature for the next Git platform. I see an OpenID login tab, but it's not very user friendly with regards to easy access to the big OpenID providers like Google. I only saw an exception made for GitHub.

[Arkanosis]

Hi everyone. I found this issue while looking around what was being done on the subject of federated git infrastructures, following the buying of GitHub by Microsoft.

When it comes to federating user identities, comments, activity… and I guess pull / merge requests as well, I think doing it using the ActivityPub standard would do even more than just federating git infrastructures together: it'd federate git infrastructures with other ActivityPub implementations. For example, it might allow you to comment pull request on a Gitea instance from a Mastodon instance, to follow-up on a bug report within a video on a PeerTube instance or a slide deck on a MediaGoblin instance with a ticket on a GitLab instance…

My 2 cts ;)

[IzzySoft]

Chiming in to what @Arkanosis wrote: seems like that's what GitPub aims at. Maybe some Gitea devs chime in there? That's what's currently asked for:

A specification like this must be agreed upon by at least some of Git web service implementations. If you are a developer of such a software, please join our discussion and speak up. We'll simply add you to the work group.

[KingDuckZ]

I'm not sure if @Arkanosis comment would also apply to Diaspora but it would definitely be useful for myself if it was the case! Looking forward to this feature! :)

[Arkanosis]

I'm not sure if @Arkanosis comment would also apply to Diaspora

@KingDuckZ: I wish! The last time I checked, Diaspora wasn't implementing ActivityPub, though. I'd really like it to do so to federate with other networks (especially with Mastodon). That would, as an added benefit, make it much easier to federate Diaspora and whatever comes out of this proposal (GitPub or anything else).

Sure, it'd be possible to federate directly with Diaspora using the diaspora federation protocol (webfingers + microformats + others), but this protocol does not seem to get as much traction as ActivityStreams / ActivityPub.

[stevenroose]

Please stay on topic. This issue is only related to being able to open pull requests from remote repos. All that is needed to achieve this is a common formulation of "repo and branch".

An example would be https://github.com/go-gitea/gitea.git#federated-prs.

Federation of other data like issues and comments are out of the scope for that. I think as a first step, you should be able to make a PR without creating an account on a git repo.

[IzzySoft]

@stevenroose if you refer to me: apologies, I should have better placed it in #1612 indeed. Meanwhile someone else did mention it there.

[schmittlauch]

There now exists a working group/ project for designing a ActivityPub based git federation protocol. https://github.com/git-federation/gitpub

Join their mailing list if you're interested.

[stevenroose]

GitLab is implementing it!
https://gitlab.com/gitlab-org/gitlab-ce/issues/40830
https://gitlab.com/groups/gitlab-org/-/epics/260

[vanitasvitae]

Niiiice!

[pwFoo]

Federated git would be awesome. Don't like to register uncountable accounts for git repos...
Federated PRs and issues! 👍

[davidak]

@pwFoo i think it will take some time to specify and implement this.

Don't like to register uncountable accounts for git repos...

Would it be an option for you to "Login with GitHub" when it's just one click until then?

[pwFoo]

Could be an workaround, but goal should federated issues / PRs, federated social networks (mastodon, pleroma, misskey, ...) in the future.

[jalcine]

Could be an workaround, but goal should federated issues / PRs, federated social networks (mastodon, pleroma, misskey, ...) in the future.

What's the use case of this? Squishing gitea to forcefully share information between ActivityPub platforms and GitFed or whatever solution is built here would lead to overengineering.

[stevenroose]

@jalcine The use case is that if you work on multiple different software projects, you're not required to create & maintain an account on all the platforms these projects are hosted on (GitHub, GitLab or their own self-hosted Gitea/GitLab/whatever). Ideally you have your own repo (GitHub, GitLab, or your own custom Gitea) and you can make PRs to all of the projects your work on from your own repo.

[jalcine]

Right but what the person I was replying to was saying that you'd sign in using Mastodon to Gitea?

[DevelAngel]

In my case, I have a Gitea instance on a home server. My raspberry pi and my internet connection has limited resources. And I want to prevent that bots can created accounts so that I have less administration in my free time.

Because of that, I disabled the registration page. Thus, nobody is able to add issues to my public repositories.

Currently, the only way to get issues from people is using a mirror repository on Github. But then, I can close my Gitea instance.

The only way where Gitea will be better than Github is that it supports federated issues and pull requests (principle of decentralization).

Social networks like mastodon is another topic and nice to have.

But federated issues and pull requests is a must-have so that people don't need to worry about account creation at each instance, like Nextcloud/Owncloud.

[pwFoo]

@jalcine
"Federated git would be awesome. Don't like to register uncountable accounts for git repos...
Federated PRs and issues! 👍"

Goal is not to login with GitHub or Mastodon Account... I quoted my answer.
Goal is to use one account (ideally my own) to open issues, reply to or open PRs.

[techknowlogick]

I've locked this issue as it is being currently discussed in the forgefed mailing list please redirect all conversation there.

More info: https://github.com/forgefed/forgefed