Git-updater: Gitlab with sub-group does not work

Created on 26 May 2017 · 35Comments · Source: afragen/git-updater

Hi @afragen,

With Gitlab new Sub-group feature, the plugin does not handle correctly the path.

If you enter the following plugin URL
https://gitlab.com/issue-packagist-784/my_sub_group/test_project.git

The plugin converts it to :
https://gitlab.com/issue-packagist--84/my_sub_group/repository/archive.zip?ref=master…

Log

Downloading install package from https://gitlab.com/issue-packagist--84/my_sub_group/repository/archive.zip?ref=master…

Unpacking the package…

The package could not be installed. PCLZIP_ERR_BAD_FORMAT (-10) : Unable to find End of Central Dir Record signature

feature-request testing

Source

sadortun

All 35 comments

I'm not sure what a subgroup is nor why it is preferable to a standard repository. Can you provide details?

afragen on 26 May 2017

Well, a group is only a way to organize repositories. Instead of having hundreds of them in the single namespace

https://git.example.com/myCompagny/Mydivision/MyProject/projectA.git
https://git.example.com/myCompagny/Mydivision/MyProject/projectB.git

Gitlab is ahead, and i am sure Github will add them in some time ! It's very usefull !
https://about.gitlab.com/2017/03/22/gitlab-9-0-released/#subgroups-ce-ee

sadortun on 26 May 2017

It doesn't surprise me that it doesn't work. GHU wasn't designed to use subdirectories.

afragen on 26 May 2017

Any possibility of making it work ? Is it a small tweak of a large refactor ?

The only actual change is that the repository URL will contain sub-directories.

Currently looking into https://github.com/afragen/github-updater/blob/develop/src/GitHub_Updater/GitLab_API.php#L417 to fetch the project path

sadortun on 26 May 2017

It's likely a major refactor. It's not the GitLab code but in Base , Plugins, and Themes.

afragen on 26 May 2017

BTW you subgroup repo doesn't even contain a plugin, only a composer.json

afragen on 26 May 2017

Oh.... Because, Once the plugin has been downloaded, the rest of the structure is identical, the only difference is the git repository URL format.

I did submit a PR to have Packagist support this feature and it was a simple tweak in a regex ( https://github.com/composer/packagist/pull/784/commits/df3af192f4358189c2e2642215dc00724f84dbba)

sadortun on 26 May 2017

BTW you subpackage repo doesn't even contain a plugin, only a composer.json

Yes, i know, it was an example to give you the path only.

Do you need a public sample project for testing ?

sadortun on 26 May 2017

No, don't need an actual plugin. The code isn't designed to parse for anything other than https://gitlab.com/owner/repo

Even with Bitbucket Server the URI doesn't work to fit to the repo but parses correctly to create the correct paths.

afragen on 26 May 2017

If your interested I think this is what would need modification, along with parts of Plugin and Theme.
https://github.com/afragen/github-updater/blob/develop/src/GitHub_Updater/Base.php#L845

Remember it still needs to work exactly as it currently does. If you could figure out how to create owner, group, repo variables…

afragen on 26 May 2017

👍1

Correct me if I'm wrong but Teams and Groups are features of GitLab only, not part of the git spec the way submodules are (or to an extent subtrees are). I wouldn't expect others to adopt something identical.

It would be cool if GHU supported more complex repos (like those with sub-modules or sub-tress) but I imagine that's a HUGE rebuild/refactor.

jb510 on 29 May 2017

The way the OP had written the example, it looks to be in the format as follows.

https://gitlab.com/owner/subfolder/repo

As long as this is consistent I might be able to find a solution. Also think subfolder == project

afragen on 29 May 2017

@afragen sorry I came into this thread with the mindset of it being related to my question about managing full sites and thinking the OP wanted to update groups. Disregard.

jb510 on 30 May 2017

@jb510 I interpreted your problem as similar in that the problem isn't some arbitrary name or subgroups or projects but how the repository is organized.

If I understand correctly it's about having the main plugin or theme folder inside a folder of a root and not simply inside the root.

Does that sound right?

afragen on 30 May 2017

If I under understand this issue correctly, no they are very different things. The OP is stating the GHU fails to correctly parse a GitLab repo url when it includes a team/group/sub-group facet. The group is just an organizational feature on GL that adds a facet to the repo url. The url still only references a single repo folder and a single repo, it just look like GL/group/sub-group/repo_name instead of GL/repo_name.

GL does not do anything magic to create a functional relationship between repos placed in a group. GL does not for example provide access to /group/* where that represents multiple repos. That'd be cool, but access like that is not available.

jb510 on 30 May 2017

Somewhere I think I'm confused about this. We should be able to clear it up at WCOC.

afragen on 30 May 2017

@jb510 exact. This is just a matter or repository URL. Nothing more. There should just be no assumsions on the URL path and expect any URL to be a valid repository URL.

sadortun on 30 May 2017

The issue I just thought about was an accurate download link. I'm not sure that a _repo_ in a subfolder of root will be able to have an appropriate download link.

afragen on 30 May 2017

I saw this issue today because I used the Subgroup feature in the GitLab and I tried to use this plugin and really is not already support this GitLab feature.

Well in fact is not a issue but a feature that GitLab made at Mar 2017 as link bellow https://about.gitlab.com/2017/03/22/gitlab-9-0-released/ so is more right to think that is a new feature.

I was looking in the code that filter the URI that is inputed in the Theme URI or Plugin URI and I saw in the API.php the get_api_url() function.

It receives also the provider (Github, GitLab or Bitbucket) parameter so maybe a new algorithm to filter this content when is selected GitLab (between $segments definition and before the switch ($type['repo´]) ) maybe would be a good way. Is only a brainstorm that I had :)

Sorry don´t have now time to test any kind of change in the next 15 days, but in the next month I guess maybe can help with this.

trombini77 on 8 Jun 2017

The issue with support of this is whether or not GitLab can provide a download link for each individual subgroup and is that link syntax consistent?

afragen on 8 Jun 2017

If someone had a group set up that they could grant me read access to that would help.

afragen on 8 Jun 2017

@afragen I just moved all our repos (100) from separate groups to sub-groups. I can discuss this weekend at WordCamp Orange County or in Slack (just ping me)

jb510 on 8 Jun 2017

The issue with support of this is whether or not GitLab can provide a download link for each individual subgroup and is that link syntax consistent?

This is the exact same thing, Project were only in groups, and now can be in groups and sub-groups. Download URL are related to projects, and are still available with the same API as before.

Groups and sub-groups only affects the URL of the repository, the mecanism to access API and download URL remains the same.

sadortun on 8 Jun 2017

Time to test. Let me know how branch gitlab-groups works.

afragen on 8 Jun 2017

👍1

BTW, the URI to put in the header is in the following format: https://gitlab.com/<owner>/<group>/<repo>.

I am not able to support subgroups of groups. I need to have a limit.

afragen on 8 Jun 2017

Just pushed https://github.com/afragen/github-updater/commit/9d1a24f285a13443e6174c2905db7485dcb25e94

It should allow any number of subgroups. Use branch switch to refresh gitlab-groups 👍

afragen on 8 Jun 2017

Has anyone else been able to test the gitlab-groups branch?

afragen on 17 Jun 2017

@afragen I tested the gitlab-groups branch and it is working ok. I've also submitted a new pull request. See if that makes any sense, my thinking could be wrong. see #563

rolandsaven on 24 Jun 2017

👍1

@rolandsaven thanks for the PR. I used concept of groups as an extended owner and updated the branch accordingly.

afragen on 27 Jun 2017

This is now merged into develop. Anyone testing this branch just needs to update when they see the notification and the update will put you back on the develop branch.

afragen on 29 Jun 2017

👍1

Edit: I've just installed the GHU Support plugin and suddenly my plugin's update is available 🤔
I'm wondering if the WP-Cron bypass setting is actually doing anything on my end, so it's possible that the update popped up after a "normal" cron cycle... Will keep my eye on it and open a new issue if I can confirm this with the next update.

~This has become a problem again for me I think. Some time ago GHU has stopped working, but I don't know at what version.~

~I noticed that the code to parse the headers has changed in the latest version and isn't like your initial commit to introduce the feature.~

~For the others in this issue, does GHU still work for you with subgroups?~