Org: Create repositories for azure-disk-csi-driver & azurefile-csi-driver as a SIG AZURE subproject

[andyzhangx]

New Repo, or migrate existing

migrate repository

Requested name for new repository

azure-disk-csi-driver

Which Organization should it reside

kubernetes-sigs

If not a staging repo, who should have admin access

@andyzhangx @khenidak @justaugustus @brendandburns @feiskyer @lachie83

If not a staging repo, who should have write access

@andyzhangx
@khenidak
@feiskyer
@brendandburns
@justaugustus
@lachie83

If a new repo, who should be listed as approvers in OWNERS

@andyzhangx
@khenidak
@feiskyer
@brendandburns
@justaugustus
@lachie83

If a new repo, who should be listed in SECURITY_CONTACTS

@andyzhangx
@khenidak
@justaugustus
@lachie83

What should the repo description be

These drivers allows Kubernetes to use azure disk volume & azure file volume

Approvals

1. SIG-AZURE Group reached consensus to create the repo on Jan 9th 2019 – please refer to the minutes on Jan 9th: https://docs.google.com/document/d/1SpxvmOgHDhnA72Z0lbhBffrfe9inQxZkU9xqlafOW9k/edit
2. Kubernetes code of conduct is adopted: https://github.com/andyzhangx/azuredisk-csi-driver/blob/master/code-of-conduct.md
3. The repo uses Apache License v2: https://github.com/andyzhangx/azuredisk-csi-driver/blob/master/LICENSE
4. CNCF CLA bot is enabled as a webhook on the repo.
5. All contributors have signed the CNCF individual or corporate CLA
6. The topic for the sponsoring SIG is added as k8s-sig-azure on the repository: https://github.com/andyzhangx/azuredisk-csi-driver/blob/master/CONTRIBUTING.md
7. Fossabadge was added for a couple of 3rd party dependencies: https://github.com/andyzhangx/azuredisk-csi-driver/pull/3

TODO:

• Update sigs.yaml file after repo transfer is complete.

cc @kubernetes/steering-committee @kubernetes/sig-azure

[justaugustus]

/lgtm
(SIG Azure approval)

[feiskyer]

/lgtm

[andyzhangx]

cc @lachie83

[justaugustus]

@andyzhangx -- are you also planning to open an issue for the https://github.com/andyzhangx/azurefile-csi-driver repo or is this one supposed to be for both?

[andyzhangx]

@justaugustus is it ok to use this issue for both repos?

[justaugustus]

@andyzhangx -- It's fine by me, but ping @cblecker or @spiffxp, just to be sure.
Count this as my approval for both repos.

When you have a chance, please add me as an admin to the repos, so I can assist with the transfers, if they're happening in US-friendly timezones.

Also, as this issue has been open for a bit and is already approved by SIG Azure, I'd like to proceed with the transfers first thing next week, unless anyone has objections.

[andyzhangx]

@justaugustus thanks, I have added you as collaborator.

[justaugustus]

Christoph -- would you be able to take care of this today?
/assign @cblecker

[justaugustus]

@andyzhangx -- please add @cblecker as a repo admin for each project.

[andyzhangx]

@justaugustus already invited as collaborator.
BTW, I cannot add someone as admin since those two repos are not under an org, instead I could do the admin privilege steps if necessary, thanks.

[Sangeeta04Sharma]

Hi, I would like to work on this task. I have already build CSI driver for GCS fuse on GCP & used AWS EBS csi driver and GCP flie store as well. I am happy to learn Azure stack and work on at as my GSOC project.
Please guide me.

[lachie83]

@andyzhangx @justaugustus what is the current status of this request?

[justaugustus]

@lachie83 -- spoke with Christoph and he's going to tackle this by EOW. Just churning through the rest of his backlog.

[justaugustus]

@kubernetes/owners -- is someone able to pick this repo migration request up? I know Christoph is on break at this point.

[nikhita]

/unassign @cblecker
/assign

I want to verify the requirements first (CLA, etc) but GitHub's been acting flakey for me while loading the list of contributors and commits, so I'll handle this when it's back and healthy. :)

[cblecker]

@andyzhangx @lachie83 @feiskyer @justaugustus --
My sincere apologies to you all. This kept getting bumped on my plate (I'm changing jobs, so everything was in a bit of upheaval).. but this took wayyyy longer than I wanted to to get done.

CLAs have been checked for the following users:

• andyzhangx
• Masquerade0097
• lizebang
• ashishranjan738
• prksu

I've also created #568 to get the initial teams created. Looking at the current OWNERS files and collaborators, only @andyzhangx is listed, so I've only added him to the teams. If direct write access for additional folks is needed, those teams can be PR'd in to.

I think all that's left @nikhita is the following:

• Merge #568 to get teams created and @andyzhangx invited to k-sigs
• Grab a dependency report (way quicker than manual.. I don't have access to FOSSA yet, so maybe @justaugustus can post a link?) and verify it
• Transfer repo
• Swap out direct collaborators for the teams in #568

I've added you as an admin to the repo, so you should be able to get those last steps done.

Sorry again for the delays!

[andyzhangx]

Thanks @cblecker, hope you are doing well in the new job!
Here are the dependency reports:
https://app.fossa.io/projects/git%2Bgithub.com%2Fcsi-driver%2Fazuredisk-csi-driver/refs/branch/master/e6edb2e29134c438c5c0b9b0c11ce0280b3948e9/preview

https://app.fossa.io/projects/git%2Bgithub.com%2Fcsi-driver%2Fazurefile-csi-driver/refs/branch/master/266f9826be14185788b2d4c37c70976ae2ee0d9b

There are 2 unlicensed dependencies failures which uses below two repos:
https://github.com/libopenstorage/gossip
https://github.com/libopenstorage/systemutils

Actually they are using all apache 2.0 license, not sure why it said unlicensed dependencies

[nikhita]

I went through the FOSSA results and also did a manual scan. While checking manually, I saw that both the repos use hashicorp/golang-lru which uses a MPL license: https://github.com/csi-driver/azurefile-csi-driver/blob/master/vendor/github.com/hashicorp/golang-lru/LICENSE.

MPL is not present in the list of whitelisted licenses: https://github.com/kubernetes/sig-release/tree/master/licensing#approved-licenses-for-whitelist. Not sure why this was not shown in the FOSSA result though.

This dependency has been approved by the CNCF IP Committee, but is pending approval by the GB. Ref: https://github.com/kubernetes/steering/issues/57#issuecomment-458968990.

Having said that, k/k also uses this dependency so I'm ok with having this dependency and doing the migration.

But just to make sure that we dot the i's and cross the t's - @swinslow, can you add a comment here confirming that it is ok to migrate a repo containing a MPL license?

[swinslow]

Thanks @nikhita. I think we are fine pulling it in with use of hashicorp/golang-lru under MPL-2.0, since that matches use elsewhere in kubernetes as you noted.

As an aside, I note that the vendor/ folder for this plugin is over 500MB, or more than 5x the size of k/k's vendor/ folder and 2x the size of k/k itself. Not itself a licensing gating item for migrating, but just wondering if there are dependencies being included in the repo that could be stripped out.

[nikhita]

As an aside, I note that the vendor/ folder for this plugin is over 500MB, or more than 5x the size of k/k's vendor/ folder and 2x the size of k/k itself. Not itself a licensing gating item for migrating, but just wondering if there are dependencies being included in the repo that could be stripped out.

Good catch! :100:

Looking into it, this happens because of two reasons:

• both of them vendor k/k (which alone contributes about 138M)
• and https://github.com/Azure/azure-sdk-for-go (which contributes about 159M)

I can't speak about the latter, but ideally "downstream" repos shouldn't be vendoring k/k....but I'm probably digressing and this is out of scope for github-management.

@spiffxp have y'all faced something like this from a github-management perspective before?

[nikhita]

cc @dims ^^
to stay in the loop for the licensing thing

[spiffxp]

vendoring k8s.io/kubernetes is really discouraged, but I agree on scope... enforcing or mandating this sort of thing IMO falls on the SIG Architecture code-organization subproject

I would encourage the subproject owners to consider their contributor experience, and prune their repo of unnecessary dependencies. But we're not here to do it for them, nor gate on this.

[spiffxp]

Out of idle curiousity

$ for d in kubernetes-{csi,incubator,sigs}/*; do du -sh $d/vendor 2>/dev/null; done | sort -r | grep M | grep -v \\. | head -n 20
297M    kubernetes-incubator/cluster-capacity/vendor
262M    kubernetes-csi/external-snapshotter/vendor
226M    kubernetes-incubator/descheduler/vendor
221M    kubernetes-csi/driver-registrar/vendor
166M    kubernetes-sigs/aws-iam-authenticator/vendor
143M    kubernetes-incubator/apiserver-builder/vendor
129M    kubernetes-sigs/cluster-api-provider-digitalocean/vendor
128M    kubernetes-incubator/custom-metrics-apiserver/vendor
122M    kubernetes-sigs/application/vendor
102M    kubernetes-incubator/kube-aws/vendor
 81M    kubernetes-sigs/kubebuilder/vendor
 63M    kubernetes-sigs/cluster-api-provider-aws/vendor
 62M    kubernetes-sigs/sig-storage-local-static-provisioner/vendor
 56M    kubernetes-sigs/gcp-compute-persistent-disk-csi-driver/vendor
 56M    kubernetes-incubator/external-storage/vendor
 50M    kubernetes-sigs/kube-batch/vendor
 50M    kubernetes-sigs/cluster-api-provider-vsphere/vendor
 50M    kubernetes-incubator/apiserver-builder-alpha/vendor
 49M    kubernetes-sigs/cluster-api-provider-gcp/vendor
 49M    kubernetes-sigs/cluster-api-provider-azure/vendor

We definitely have other repos that import a lot, but it's true, nothing as large as this.

[nikhita]

I'm gonna go ahead and migrate because that's not a gating factor but it would be cool to see the repo get pruned a bit. :)

[nikhita]

@andyzhangx @justaugustus sorry for the bumpy journey :) ...but the migration is now complete! :tada:

• https://github.com/kubernetes-sigs/azuredisk-csi-driver has been created and azuredisk-csi-driver-admins had admin access and azuredisk-csi-driver-maintainers has write access.

• https://github.com/kubernetes-sigs/azurefile-csi-driver has been created and azurefile-csi-driver-admins has admin access and azurefile-csi-driver-maintainers has write access.

Could you create a PR to add the two repos to sigs.yaml? Once that's done we can close this issue. :+1:

[justaugustus]

Thank you SOOOOO much!
I've updated OWNERS here: https://github.com/kubernetes/community/pull/3369

@nikhita -- you can feel free to /lgtm that one. The addition is approved by me for SIG Azure.

[nikhita]

@nikhita -- you can feel free to /lgtm that one. The addition is approved by me for SIG Azure.

Done. But you'll still need a root owner for approval. Assigned Aaron since he was involved with this issue anyway.

[andyzhangx]

Thanks guys!
about "vendoring k8s.io/kubernetes" issue, actually these two CSI repos depends on azure cloud provider, and that provider is still in k8s core repo, so when that provider is removed out of core repo, and we will switch to use a standalone azure provider, that will reduce the size a lot.

Currently we don't want to maintain an exactly same azure cloud provider out of k8s core repo, that's why we don't switch now.

[nikhita]

@andyzhangx Ack, thanks for the update! :)