Gitea: Gitea hosted Gitea

[lunny]

For the first big stage, we would like Gitea's development could be based on a Gitea hosted and github will only a mirror. This will maybe completed in v1.x. So that this issue will list all the features needed to be implemented before v1.x. And of course please discuss them and change my post.

• [x] ~Squash merge (#712 #3188)~
• [x] ~Complete Protected branch (#32 #339)~
• [x] ~Complete API support (#64)~
• [x] ~API Documents (#194)~
• [x] ~Webhooks implementation (#2418)~
• [x] ~Better CI Integration ( (PR #1332~)) ~Drone PR: (#2017)~
• [x] ~Comment on commit and PR (#124 ~#2583~ #3748 )~
• [x] ~Approvals system (#2794 #3748 )~
• [x] ~Approvals limitations (#5251)~
• [x] ~Migrate a throughout github repository to gitea (#6290, #7293, #6200, #7410)~
• [ ] Dump/restore github/gitlab repository data to a local directory and restore to gitea #12244

Migrating Progress Updated:

• [ ] migrate gitea repository -> https://gitea.com/gitea/gitea blocked by #12244
• [x] migrate go-sdk repository -> https://gitea.com/gitea/go-sdk
• [x] migrate git repository -> https://gitea.com/gitea/git
• [x] migrate tea repository -> https://gitea.com/gitea/tea
• [x] migrate website repository -> https://gitea.com/gitea/website
• [x] migrate blog repository -> https://gitea.com/gitea/blog
• [x] migrate theme repository -> https://gitea.com/gitea/theme
• [x] migrate redirects repository -> https://gitea.com/gitea/redirects
• [x] migrate homebrew-gitea repository -> https://gitea.com/gitea/homebrew-gitea

[Lourens-Rich]

Very good idea!

[bkcsoft]

1.2 in February, 1.3 in April, 1.4 in June, 1.5 in August? should be enough time to implement all that 😄

[zellyn]

If you haven't seen it, fantastic and insightful comment supporting your approach to self-hosting only when ready: https://lobste.rs/s/gokjbo/gitea_1_1_0_released/comments/dg9pwe#c_dg9pwe

[bkcsoft]

@lunny Now that I think about it (thanks @zellyn for that link 😂 ) Why do we need oauth-provider, _complete_ webhook support, api-documentation, and a _complete_ API for self-hosting?

OAuth _Consumer_ is required (it's merged AFAIK) so people can login using github auth.
Drone only uses push hooks, so why would we need the other?

As for API, not sure why self-hosting requires that at all TBH :)

[strk]

I agree about trimming that list. Earlier self-hosting will very likely help us setting priorities better :)

[lunny]

@bkcsoft maybe we can setup a hosted site and have a try.

[lunny]

@bkcsoft I updated the issue, do you mean that?

[ekozan]

-> OAuth provider (#27) is not closed

[bkcsoft]

@ekozan not closed, but scratched from the list of "things we need"

[bkcsoft]

Added "Repository Size Limits" since we don't have unlimited storage on the servers...

My proposal for limits:

• 0 Orgs
• 3 Repos
• 1GB/repo

[lunny]

@bkcsoft Did you mean it will be a public service for anyone?

[bkcsoft]

Maybe, maybe not, but _if_ it becomes a public service we can't have it unlimited ;)

[stevegt]

I think dogfooding is important enough that repo size limits may not need to be in the critical path for self-hosting gitea. In the first few days after migrating to gitea, I've run across several feature omissions that made me think self-hosting will help focus effort on getting those things done. Gitea is already a fantastic, highly usable, high-performance tool -- it's a real shame you aren't using it yourselves. ;-)

Rather than depend on hard size limits, it might instead be helpful to think about how the self-hosted server is going to be administered, who's going to police bad behavior, and what tools they will want. For instance, contributor forks of the gitea project ought to be supported on gitea's own server. This exposes the risk that a user forks gitea, then pushes warez to their fork. Size limits might help prevent pushing large binaries, but might not help with a list of passwords or credit card numbers. A tool that might help in that case is something that detects alien files by diff line count or rolling hash.

A nice side-effect of having a diff-size tool available is that the code could be available as an option to run during pushes to flag legitimate commits that should have been broken up into smaller pieces anyway. (Related discussion for ways to do this: https://github.com/go-gitea/gitea/issues/3658#issuecomment-372263759.)

I'd bet there are a lot of other subtle things that will need to be addressed for public-facing servers. It might make sense to use a separate "public hosting" master issue or milestone to track these things.

[stevegt]

Speaking of milestones, should this issue be added to 1.5.0?

[jonasfranz]

@stevegt No, since I think that not all of the PRs will get merged / resolved at 1.5.0.

[lunny]

I removed Repository Size Limits (#3658) from the issue since it will not affect Gitea hosted gitea.

[mxmehl]

I removed Repository Size Limits (#3658) from the issue since it will not affect Gitea hosted gitea.

Great! I'm positive that the sooner Gitea hosts itself, the faster will the whole project profit from real-life experiences, and gain trust and confidence :)

[justinclift]

@lafriks mentions in another thread:

Self-hosted would probably require additional funding/sponsorship to pay for additional virtual machine

And @lunny asks above:

@bkcsoft Did you mean it will be a public service for anyone?

Would it be feasible to combine those thoughts into a "How about setting up an online Gitea service, where people pay for (say) private repos?".

If done ok, that should generate the funds to pay for itself + the public repos.

As a concept, it seems to be fairly well travelled ground. :smile:

[drsect0r]

To promptly add to @justinclift idea; the timing might be right with the current news of Microsoft taking over GitHub.

[mxmehl]

@lafriks mentions in another thread:

Self-hosted would probably require additional funding/sponsorship to pay for additional virtual machine

I'm confident that there will be funding from the community or sponsorship from organisations to make gitea hosting itself possible. Since Gitea is resource-friendly (yes, GitLab, I'm looking at you) this won't be a big deal.

[techknowlogick]

@mxmehl so far there have been 5 individuals that have contributed since the opencollective was open last month: https://opencollective.com/gitea

[lafriks]

@justinclift as Gitea is purely community for driven there is no way we could set up paid private repositories as that requires creating company, dealing with taxes, and have full time staff to deal with technical problems

[mxmehl]

@mxmehl so far there have been 5 individuals that have contributed since the opencollective was open last month: https://opencollective.com/gitea

@techknowlogick Didn't know this page. Now it's 6 ;)

[justinclift]

@lafriks Well.... there are Community projects around - for both software and non-software things - which seem to manage themselves ok, including financial matters, things they pay for, staff (where needed), and so on.

That being said, it does require a level of will to make it happen + keep it going. The people in any needed roles also need to be good custodians (trustworthy, reliable, clueful).

If there's no interest, then it won't go anywhere anyway. Ditto if no suitable "custodian" types can be agreed upon.

From the Open Collective link mentioned above, it looks like some initial seeds are in place. It demonstrates there are people around who are considered ok as custodians. :smile:

[lafriks]

@justinclift I'm not saying it is not possible but just not at current stage but in future it could happen. Currently at least I would better focus on developing new Gitea features and improving documentation :) So any help is much appreciated to move faster to this goal.

[justinclift]

Heh Heh Heh

No worries at all @lafriks. :smile:

[lunny]

First of the goal is Gitea hosted Gitea since github married with Microsoft. :)

[lunny]

I think only #2519 and #3748 need review and merge before we close this issue.

[axifive]

@bkcsoft

Added "Repository Size Limits" since we don't have unlimited storage on the servers...

My proposal for limits:

• 0 Orgs
• 3 Repos
• 1GB/repo

I think, with the repos quantity limit, we can add allows setting for fork only existing repositories for all users not in the Gitea team:

• 0 Orgs
• 3 Repos (allow only forks)
• 1GB/repo

[lafriks]

I don't think fork count need to be limited, it would be limited by gitea org repository count anyway, so that should be ok.
As for repo size, yeah, probably there should be some limits

[lunny]

We should limit creating orgs, creating repos, so repo size limit is not a necessary issue for Gitea hosted Gitea.

[stevegt]

We might consider adding #3134 and #4302 (PR and issue backlinks) to the prereq list for self-hosting -- maybe I'm unique, but our own little gitea install started getting unwieldy without those backlinks as soon as we added more than a few users and issues. We've been able to work around that some with issue search, but that's limited without global issue search (#2434/#3841).

[bkcsoft]

@stevegt While backlinks would be nice to have, I don't see them blocking moving to self-hosting :)

[stevegt]

@bkcsoft Fair enough -- just thought I'd mention it in case it triggered any realizations. In our local case, we've been developing the habit of manually adding the backlinks in issue comments as a workaround.

[jlelse]

I recently heard about the effort of https://teahub.io. They want to start a non-profit org. Can't Gitea use that once they are ready with the setup?

[lunny]

@jlelse No. Gitea will set up a standalone server (i.e. self.gitea.io) to host gitea and mirror to github, gitlab or teahub and etc.

[jonasfranz]

@lunny Do we really require commments on commits since we are currently only using comments on PR at GitHub?

[lunny]

@JonasFranzDEV I mean comments on commits of PR, I think it's necessary.

[ryanburnette]

I'm commenting here because #4108 is closed. When the Gitea Patreon (or Patreon-like alternative) goes up, I need to know about it. I'll be contributing. I'd like to see this project self-hosted and further developed. Once I move all my repositories I won't be spending money with Github anymore, I'll commit that monthly to the Patreon.

[techknowlogick]

@ryanburnette https://opencollective.com/gitea

[mjmlvp]

@lunny looks like 'Approval system' in the topic start can be crossed out. (The issues being respectively closed and merged)?

[lunny]

@mjmlvp I think you are right. I removed #996 #2519 since it should not be a block of this issue. We will set up a server to host our developments.

[strk]

Any news on this ?

[lafriks]

Yes we still need to add approval limitations and then we can migrate to selfhosted code

[raucao]

Would be nice to add the open issues for that to the list. At the moment it looks like all linked issues and PRs have been closed and merged.

[lunny]

@skddc done.

[milahu]

for the record

open gitea servers

• https://try.gitea.io/
• https://teahub.io/
• https://dev.sum7.eu/

trust no one. make regular backups

[ptman]

@giteauser I fail to see how that is relevant

[ptman]

@giteauser and this is about self-hosting gitea development, so that gitea development doesn't have to happen on github anymore. I still fail to see how this is relevant.

[ghost]

Oh I posted these for the wrong issue, I'm sorry.

[davidak]

So, every needed feature should be implemented now. Can we have a new release and migrate away?

[lunny]

Right, we will setup a gitea instance to host gitea's development.

[max-wittig]

I think fixing the backup dump issue and adding proper restore functionality is also an important part to managing infrastructure of a self-hosted gitea

[raucao]

If you're deploying Gitea on Kubernetes, I can recommend Ark for backups and restores.

Generally speaking, backup doesn't seem like it should be a blocking topic for self-hosting Gitea development, because everyone usually has different backup strategies/programs, depending on their choice of platform, method of deployment, existing tools, etc.. It should only be a blocking issue for that very instance's production readiness.

[lunny]

@max-wittig The gitea self-hosted website will run on some public cloud provider. They will provide RDS service and database backup tool and some disk backup function so that backup dump command is not a dependency issue. The dump command is for a single-node gitea service. Of course we should fix that issues.

[lunny]

@skddc It's an interesting tool that we can consider that.

[markg85]

I really hope the 1.8.0 release is going to be the version where gitea becomes self hosted. I personally would love to use it for my own projects but am only using gitea as a mirror for the simple reason that i don't trust it enough if the devs - you guys - don't even use it for the gitea development. Also, i'd like to propose to use gitea in my working environment but can't even propose it as i will just be laughed at when i say: "oh btw, it looks awesome, but the devs of gitea don't use it for their own project" ...

This is not meant as criticism or as a way to push you folks. Just letting you know what i (and probably lots of others) would find troublesome when it comes to using gitea as main source code control system ;)

Lastly, to be a bit more constructive. If you are looking for a very good (and cheap) clous vps for backups or even to host gitea on, take a look at hetzner https://www.hetzner.com/storage/storage-box Though for an important infrastructure as Gitea you'd probably want to back it up at two completely different locations (say hetzner and digitalocean for instance).

[lunny]

@markg85 we are working on https://gitea.com

[zachdoty]

Any update on gitea hosted gitea at gitea.com?

[lunny]

@zachdoty Still are working on that.

[lunny]

I forgot something necessary to move from github to gitea. We need move all data(include git data, issues, comments, pull requests from github to gitea), but in fact I haven't found suitable tools to do that. I have sent PR about move migrating repository from frontend to backend, see #6200 , and also https://gitea.com/gitea/migrator should be merged to gitea because gitea's API will not allow create issue with specify index number.

[raucao]

gitea-github-migrator can migrate almost everything, in case you haven't found that one yet. If something is missing, maybe it can be added there, so other projects also have a good tool for migrating.

[kolaente]

@skddc they actually are the same.

[raucao]

Ah, sorry. Hadn't followed the link.

[lunny]

As @kolaente said, we invited @jonasfranz move gitea-github-migrator to gitea.com and I send a PR https://gitea.com/gitea/migrator/pulls/1 wants to improve that. But I found as a third-party tool, there is one disadvantage. That is it's difficult to keep the issue index as before. (To let all the links on the issue still available.) So I think merge the migrate tool into gitea on the migrate UI is a better idea.

[raucao]

Would definitely be awesome if the internal links in comments etc. could be preserved. Would also be awesome if the PRs that originate from the same repo could be imported as actual PRs instead of issues!

[lunny]

I added new task Migrate a throughout github repository to gitea on issue content and move this to 1.9.0 so that this will not block v1.8's release. But I don't think we should wait until v1.9 released since gitea.com will follow the master.

[BNolet]

Is Gitea.com production ready, as in should I worry about storing code there and it being lost?

[lunny]

@BNolet we have set up a 6 machines ceph cluster to store the repositories. So it should not be lost easy since all the data will be copied 3 times. We are working on set up another backup via ceph mirror. And since git is distributed. Your codes will always store on your or your colleagues disk.

[BNolet]

Wow that's fantastic! Do you all anticipate being able to be a full-on alternative to GitHub or Gitlab?

[lunny]

I don't think so. The main target of gitea.com is to host gitea's developments itself. Although we haven't do any limitation on gitea.com, but we encourage you to set up yourself hosted Gitea instance since it's so easy.

[strk]

Any reason for not enabling OpenID sign-in ?

[BNolet]

@lunny I have one and I love it :D

How likely is it that CI/CD becomes a part of Gitea the product?

[lunny]

@strk just forget it. Will enable OpenID sign-in.
@BNolet Gitea itself will use drone as the primary CI/CD.

[jakimfett]

Getting a 500 error upon attempting to use the Github oauth option on the login page, after I go the permissions request for the go-gitea app on Github.

Should I wait a bit to check out the flagship instance?

[lunny]

@jakimfett that's a known issue. You can try twice and it's OK.

[strk]

@strk just forget it. Will enable OpenID sign-in.

@lunny as it was still off today, is there maybe a deploy script that keeps turning it off ?

[jakimfett]

@jakimfett that's a known issue...

Throw an edit in with the issue number and I'll followup there.

_edit(s): formatting_

[curbengh]

Codeberg is a free Gitea-based service. Perhaps Gitea can set up an official mirror there. Currently it's mirrored at https://codeberg.org/Codeberg/gitea.

[lunny]

Gitea will be hosted on https://gitea.com/gitea/gitea , and we have moved most other packages to https://gitea.com/gitea, and gitea itself is on progress. Mirror is welcome on any other gitea-based service.

[jakimfett]

@lunny commented yesterday:
Mirror is welcome on any other gitea-based service.

I'll hack together an up-to-date mirror once I get back from my weekend.

Related:
Is there a list of mirrors somewhere, and do I add myself to it, or...?

[kolaente]

@jakimfett There isn't, but you could create a pr with a new docs page

[lunny]

For a better user handler when migrating gitea away github, I would like to move this to v1.10 and I think we should implement https://github.com/go-gitea/gitea/issues/7293 before we start to migrate.

[SuperSandro2000]

I would be totally for that move if it wasn't hosted in China or activation email took 10 min to arrive.

[programmerjake]

I would be totally for that move if it wasn't hosted in China or activation email took 10 min to arrive.

Edit: apparently, I was incorrect.

from some googling, apparently gitea.com's IP address is actually in Japan, not China. That IP address is owned by Alibaba though.

[lunny]

We use mailgun.org to send mails. I don't know why it will spend 10 minutes.

Our donates cloud provider is didiyun which is in China and it provides many machines. We have no other choice currently.

And gitea.com's first purpose will not become as a service like github.com or gitlab.com. It will only host gitea itself and we recommend you set up yourself gitea instance in fact.

[lunny]

@programmerjake That's a bridge server.

[SuperSandro2000]

It wasn't my intention to switch away from my instance but I don't like that this project relies on infrastructure donated from a Chinese company that has almost no reputation in the past or search results. You don't really know what they do in the future or if they demand to cut their donation if you don't implement XY or do ZA or if they just go away some day.
You probably know what you are doing and my concerns are just to cautious but you'll never know.

[lunny]

Why a Chinese company will do that but an U.S. company will not? :)

[lunny]

And I'm a Chinese, and maybe you should be careful. Someday I may stole your codes. :)

[lunny]

I think I may have that plan when I started Gogs with other 3 Chinese people on 2014.

[markg85]

It wasn't my intention to switch away from my instance but I don't like that this project relies on infrastructure donated from a Chinese company. You don't really know what they do in the future or if they demand to cut their donation if you don't implement XY or do ZA or if they just go away some day.
You probably know what you are doing and my concerns are just to cautious but you'll never know.

That's a totally idiotic reply.
Now i might be more open minded as i'm Dutch, we kinda embrace that. You should try it too.

As long as this project is open source, there is no risk in my opinion for anything you said.
And even if they do, which probably means forking Gitea, they are fully in their right as the license simply permits it.

The US however... let me remind you that Github has limited their features now due to the "trade war" between the US and China. If anything, the US is more of a risk for free software development at the moment then China has ever been. Heck, @lunny is probably even effected on this very repository due to the trade war.

@lunny and the team developing Gitea. Keep up the awesome work! :)

[kmmndr]

And gitea.com's first purpose will not become as a service like github.com or gitlab.com. It will only host gitea itself and we recommend you set up yourself gitea instance in fact.

Is it something you may consider in the future ?

[SuperSandro2000]

As long as this project is open source, there is no risk in my opinion for anything you said.

The core project has no risk but the infrastructure does. There are no guaranties that the unknown unreputable donor will be there next month no matter if he comes from anywhere or China. GitHub will be here for a long time and won't go anywhere that quick.

Also if the entire infrastructure moves to China the entire user base will suffer due to the difficulties between the US and China.

[guillep2k]

@lunny Regarding e-mails taking ~10 minutes, in my case my company server has an anti-SPAM measure that is a "cool off" timer (20 minutes here) within which it will temporarily reject any mails from unknown sources. So, if it's the first time a user (me) receives an e-mail from that domain (e.g. gitea.com), then my server will respond with "try later" and remember that user/domain combination. The next time gitea.com attempts to deliver my mail within the expected time, the server accepts the message.
Obviously we have some "trusted sources" configured, like GMail, Hotmail, etc. that don't need a cool-off period.

[lathama]

People, if you want more infrastructure spread around for every region then please vote with your wallet at https://opencollective.com/gitea

[guillep2k]

@SuperSandro2000 Are you aware that Gitea is not a service but a _product_? You download the sources, compile Gitea in your servers and install it. No connection whatsoever required to Gitea's hosting.

[techknowlogick]

if they demand to cut their donation if you don't implement XY or do ZA or if they just go away some day.

There are no guaranties that the unknown unreputable donor will be there next month.

A couple of responses to this: We wouldn't host on an unreputable company, and the Gitea leads have placed our trust in this company. If they stop providing sponsorship, or stop being a company we have options available to us, and can move elsewhere.

the entire user base will suffer due to the difficulties between the US and China.

A large part of Gitea team is from China (but we also have maintainers in all other continents too), and if development team can't access code then the user base will suffer, which is why we need development team to be able to access code. We build Gitea so everyone can use it, even users who are banned from GitHub (after recent ban wave from GitHub a lot of those users started using Gitea).

As others have mentioned, there are mirrors of the codebase on other instances around the world, and thanks to git there is a ledger of all changes made to code so everyone can have visibility into all changes.

A note on transparency: I have locked this thread. I don't want to stop this conversation, however it should be moved to a different place as this github ticket is about what enhancements need to be done with software for self-hosting (rather than where we are hosting).