Pwa-studio: Migrate PWA Studio to magento Organization

Created on 25 Jul 2019 · 17Comments · Source: magento/pwa-studio

https://help.github.com/en/articles/transferring-a-repository#whats-transferred-with-a-repository
https://help.github.com/en/articles/transferring-a-repository#transferring-a-repository-owned-by-your-organization

Plan of action:

🔁 denotes an item in progress

Before the Move

[x] Prune branches of pwa-studio repository
[x] Identify CICD integrations that may potentially break
Done: https://github.com/magento-research/pwa-studio/issues/1472#issuecomment-516067641.
[x] Notify the community (Slack, Community Sync)
[x] Create a full list of outside collaborators to add to the Community Contributors team.
Done: https://github.com/magento-research/pwa-studio/issues/1472#issuecomment-515589081.
[x] Record project collaborators' permission set (will not transfer to the new repo automatically). Done: https://github.com/magento-research/pwa-studio/issues/1472#issuecomment-516133700.
[x] Create a full repository copy using GitHub Mirror approach
Done: https://github.com/magento-research/pwa-studio-copy.
[x] Create GitHub Pages migration plan
Done: https://github.com/magento-research/pwa-studio/issues/1472#issuecomment-516498022.
[x] Prepare a PR to redirect the old docs site to the new one
Done: https://github.com/magento-research/pwa-studio/pull/1494.
[x] Ensure the magento organization has the bundlesize, Danger-JS, and Now Apps that the magento-research organization has
Done: It does not, but we can install them

Move

[x] Modify the http://pwastudio.io redirect to point to the new docs site location (@ericerway)
[x] Rebuild / Deploy the docs site from this repo (@jcalcaben)
[x] Verify the http://pwastudio.io and https://magento-research.github.io/pwa-studio/ redirects are working (they should be 404ing)
[x] Update the pwa-studio-copy mirror with the latest changes

git fetch -p origin
git push --mirror

[x] Move pwa-studio to magento organization on GitHub. Instructions here.
[x] Rebuild, publish, and verify the docs site (https://magento.github.io/pwa-studio/)
[x] Verify that pwastudio.io and the old docs site redirect to the new docs site
[x] Install the required GitHub Apps
[x] Add the full list of outside collaborators to add to the Community Contributors team with write permissions
[x] Notify the community (Slack, Community Sync), especially about the docs site move. Monitor Community Slack for problems
[x] Merge #1488
[x] Merge #14 in pwa-studio-cicd repo

After the Move

[ ] Review the team's workflow to ensure that it is community oriented, make changes if necessary

Owned by Us

[x] Update repository references in pwa-studio markdown
Done: #1511 .
[x] Update repository references in pwa-studio wiki
[x] Update repository references in dev-docs
[x] Update repository references in pwa-tests repo
Update: none found
[x] Update repository references in internal tools (Confluence, etc.)
Created Issue: #1524.
[x] Delete https://github.com/magento-research/pwa-studio-copy repository
Created Issue: #1525.
[x] Move other repositories to the magento organization:
- upward-php
- magento2-upward-connector
- pwa-tests
- express-sharp
- venia-consumer-example

Created Issue: #1526.

Known Issues

[x] Now.sh
Issue Created: #1519.
[x] Coveralls is not reporting correctly
Issue Created: #1520.
[x] CLA check
Done: Folks may have to re-sign the CLA in the magento org. Working for new contributors.
[x] Old docs site 404s

Owned by EngCom

[ ] Add pwa-studio as a pinned repository in magento
[ ] Create GitHub project for pwa-studio
[ ] Update EngCom's Portal tool
[ ] Check contributions points mechanism
[ ] Community Portal statistic. PWA project name
[ ] Community Portal project card

Source

supernova-at

Most helpful comment

Suggestion: after the move, add PWA Studio to the pinned repositories for the magento org (top middle slot looks good).

brendanfalkowski on 1 Aug 2019

👍2

All 17 comments

Are there any inbound/outbound links that we need to worry about breaking during the move? Docs, wiki, confluence, etc.?

awilcoxa on 25 Jul 2019

👍1

Are there any inbound/outbound links that we need to worry about breaking during the move? Docs, wiki, confluence, etc.?

Added to the list 👍

We should definitely update those but GitHub will redirect for us in the meantime. The way I understand it, nothing will break. 🤞

supernova-at on 25 Jul 2019

If we have a not-clone repo with a copy of all the code, and magento-research/pwa-studio is a remote that we forcefully pull from to get upstream changes...

As I understand it, that's how quite a few projects are keeping track and using PWA Studio today.

No idea if Git or GitHub gracefully handles that redirect. Probably not I'm guessing. Could be a confusing day weeks down the line if Git starts saying remote not found.

Not sure there's any way around it if the redirect doesn't apply.

brendanfalkowski on 25 Jul 2019

As I understand it, that's how quite a few projects are keeping track and using PWA Studio today.

Good point - @Jordaneisenburger that's how fallback studio works, right?

From what I understand, GitHub will redirect HTTP(S) and SSH requests to magento-research/pwa-studio to magento/pwa-studio. Those remotes should still be updated, but (at least in the short term) they will be redirected.

supernova-at on 26 Jul 2019

From https://help.github.com/en/articles/transferring-a-repository:

All links to the previous repository location are automatically redirected to the new location. When you use git clone, git fetch, or git push on a transferred repository, these commands will redirect to the new repository location or URL. However, to avoid confusion, we strongly recommend updating any existing local clones to point to the new repository URL. You can do this by using git remote on the command line:

$ git remote set-url origin new_url

supernova-at on 26 Jul 2019

@supernova-at Fallback Studio is a full wrapper around the whole PWA Studio repo. It's used as a vendor and so Studio lives inside the fallback studio repo. Meaning that it wouldn't crash if the Studio repo changed. I just have to update my upgrade script to get the latest Studio version from the new repo i think

Jordaneisenburger on 26 Jul 2019

👍1

Okay here is EngCom's checklist from previous repository transfers:

Create full repo copy
Check redirects: GitHub, Wiki, Git console
Check list of outside collaborators. Move them to Community Contributors team with write permissions
Move to Magento org
Create DevOps tickets to update Jenkins configuration to use new location
Check CLA service integration
Backlog GitHub app integration
EngCom portal tool Update
Check contributions points mechanism
Community portal statistic. MSI project name
Check release packaging builds
Check all documentations (DevDocs, Wiki)
Community Portal project card

Here are some thoughts / questions on the above:

Create full repo copy

✅ Added to our checklist

Seems like a good idea "just in case". I don't think we want to officially duplicate a repository, but it would be smart to make sure we have a copy of the full repo _somewhere_ (even if it's just locally on someone's machine) before we initiate the transfer.

Questions:

What advantages does duplicating the repository have?
Am I correct in assuming we'd have the original magento-research/pwa-studio and the duplicated magento-research/pwa-studio-new and then we'd transfer magento-research/pwa-studio-new to magento/pwa-studio?
Where would that leave folks who fetch / pull / push to magento-research/pwa-studio? Would that still update the old repository?

Check redirects: GitHub, Wiki, Git console

❌ Did not add to our checklist

If we don't do the full duplication we don't have to do this step, as the redirects won't kick in until we do the actual transfer.

Check list of outside collaborators. Move them to Community Contributors team with write permissions

✅ Added to our checklist

I think we can get a list of folks who've contributed but where does that leave new contributors? Is there a process for them to get themselves added to Community Contributors team?

Create DevOps tickets to update Jenkins configuration to use new location

❌ Did not add to our checklist

Not needed for PWA.

EngCom portal tool Update

✅ Added to our checklist

I'm assuming pwa-studio will have to be added to EngCom's portal tool

Check contributions points mechanism

✅ Added to our checklist

Questions:

How do we do this?

Community portal statistic. PWA project name
Community Portal project card

✅ Added to our checklist

I'm assuming EngCom will run point on these 😄

Check release packaging builds

❌ Did not add to our checklist

We should probably plan for a hiccup or two next time we use the automated release scripts. Updates can happen at that time.

supernova-at on 26 Jul 2019

Here is the full list of outside collaborators to add to the Community Contributors team:

Question: should we strike EPAM folks off this list?

~0: {name: "zetlen", url: "https://github.com/zetlen"}~
~1: {name: "DrewML", url: "https://github.com/DrewML"}~
~2: {name: "jcalcaben", url: "https://github.com/jcalcaben"}~
~3: {name: "jimbo", url: "https://github.com/jimbo"}~
~4: {name: "supernova-at", url: "https://github.com/supernova-at"}~
~5: {name: "sirugh", url: "https://github.com/sirugh"}~
6: {name: "Starotitorov", url: "https://github.com/Starotitorov"}
~7: {name: "tjwiebell", url: "https://github.com/tjwiebell"}~
8: {name: "mage2pratik", url: "https://github.com/mage2pratik"}
9: {name: "AlexeyKaryka", url: "https://github.com/AlexeyKaryka"}
~10: {name: "sharkySharks", url: "https://github.com/sharkySharks"}~
~11: {name: "dpatil-magento", url: "https://github.com/dpatil-magento"}~
12: {name: "codeAdrian", url: "https://github.com/codeAdrian"}
13: {name: "pcvonz", url: "https://github.com/pcvonz"}
14: {name: "vdiachenko", url: "https://github.com/vdiachenko"}
15: {name: "Serunde", url: "https://github.com/Serunde"}
16: {name: "JStein92", url: "https://github.com/JStein92"}
17: {name: "yogeshsuhagiya", url: "https://github.com/yogeshsuhagiya"}
18: {name: "vishal-7037", url: "https://github.com/vishal-7037"}
19: {name: "lewisvoncken", url: "https://github.com/lewisvoncken"}
20: {name: "rowan-m", url: "https://github.com/rowan-m"}
21: {name: "jahvi", url: "https://github.com/jahvi"}
22: {name: "ennostuurman", url: "https://github.com/ennostuurman"}
23: {name: "dani97", url: "https://github.com/dani97"}
24: {name: "brendanfalkowski", url: "https://github.com/brendanfalkowski"}
25: {name: "shakyShane", url: "https://github.com/shakyShane"}
26: {name: "rossmc", url: "https://github.com/rossmc"}
27: {name: "ronak2ram", url: "https://github.com/ronak2ram"}
28: {name: "real34", url: "https://github.com/real34"}
29: {name: "mtbottens", url: "https://github.com/mtbottens"}
30: {name: "mhhansen", url: "https://github.com/mhhansen"}
31: {name: "khoa-le", url: "https://github.com/khoa-le"}
32: {name: "jissereitsma", url: "https://github.com/jissereitsma"}
33: {name: "gil--", url: "https://github.com/gil--"}
34: {name: "belbiy", url: "https://github.com/belbiy"}
35: {name: "abrarpathan19", url: "https://github.com/abrarpathan19"}
36: {name: "VitaliyBoyko", url: "https://github.com/VitaliyBoyko"}
37: {name: "LucasCalazans", url: "https://github.com/LucasCalazans"}
38: {name: "Igloczek", url: "https://github.com/Igloczek"}
39: {name: "zengang", url: "https://github.com/zengang"}
40: {name: "zanilee", url: "https://github.com/zanilee"}
41: {name: "vitalics", url: "https://github.com/vitalics"}
42: {name: "sudeep-cedcoss", url: "https://github.com/sudeep-cedcoss"}
43: {name: "speedy008", url: "https://github.com/speedy008"}
44: {name: "shashidesilva", url: "https://github.com/shashidesilva"}
45: {name: "sanjay-wagento", url: "https://github.com/sanjay-wagento"}
46: {name: "realchriswells", url: "https://github.com/realchriswells"}
47: {name: "raith-hamzah", url: "https://github.com/raith-hamzah"}
48: {name: "prakashpatel07", url: "https://github.com/prakashpatel07"}
49: {name: "pradeep-cedcoss", url: "https://github.com/pradeep-cedcoss"}
50: {name: "philwinkle", url: "https://github.com/philwinkle"}
51: {name: "niklas-wolf", url: "https://github.com/niklas-wolf"}
52: {name: "neeta-wagento", url: "https://github.com/neeta-wagento"}
53: {name: "mzeis", url: "https://github.com/mzeis"}
54: {name: "mrtuvn", url: "https://github.com/mrtuvn"}
55: {name: "michaelyu0123", url: "https://github.com/michaelyu0123"}
56: {name: "matthewhaworth", url: "https://github.com/matthewhaworth"}
57: {name: "marcneubauer", url: "https://github.com/marcneubauer"}
58: {name: "marcin-piekarski", url: "https://github.com/marcin-piekarski"}
59: {name: "mageprince", url: "https://github.com/mageprince"}
60: {name: "kanhaiya5590", url: "https://github.com/kanhaiya5590"}
61: {name: "jflanaganuk", url: "https://github.com/jflanaganuk"}
62: {name: "jaimin-ktpl", url: "https://github.com/jaimin-ktpl"}
63: {name: "greenkeeper[bot]", url: "https://github.com/apps/greenkeeper"}
64: {name: "gavin2point0", url: "https://github.com/gavin2point0"}
65: {name: "fooman", url: "https://github.com/fooman"}
66: {name: "edwinbos", url: "https://github.com/edwinbos"}
67: {name: "davidverholen", url: "https://github.com/davidverholen"}
68: {name: "camdixon", url: "https://github.com/camdixon"}
69: {name: "bobmotor", url: "https://github.com/bobmotor"}
70: {name: "bobbyshaw", url: "https://github.com/bobbyshaw"}
71: {name: "bgkavinga", url: "https://github.com/bgkavinga"}
~72: {name: "bbatsche", url: "https://github.com/bbatsche"}~
73: {name: "artKozinets", url: "https://github.com/artKozinets"}
74: {name: "ankitsrivastavacedcoss", url: "https://github.com/ankitsrivastavacedcoss"}
75: {name: "andreas-ateles", url: "https://github.com/andreas-ateles"}
76: {name: "adrian-martinez-interactiv4", url: "https://github.com/adrian-martinez-interactiv4"}
~77: {name: "PWAStudioBot", url: "https://github.com/PWAStudioBot"}~
78: {name: "MarynaVozniuk", url: "https://github.com/MarynaVozniuk"}
79: {name: "Jakhotiya", url: "https://github.com/Jakhotiya"}

supernova-at on 26 Jul 2019

@supernova-at.
About "Create full repo copy", we did it the first time, and yes, it's needed just to protect yourself from some extra cases.

We did duplicate using GitHub mirror approach.
No, you should transfer origin repository, duplicate it's just back up copy for an extra case.
The fetch / pull / push and other manipulation with "origin" will be automatically redirected to the new repository. ( You will see the notice in the terminal that remote repository migrated to another organization, but all functionality will work correctly)

Take attention to project collaborators, because they permissions will not be transferred automatically.

Community portal statistic. PWA project name
Community Portal project card 
EngCom portal tool Update

we will check these points.

VladimirZaets on 27 Jul 2019

👍1

Identify CICD integrations that may potentially break

From Transferring a Repository:

If the transferred repository contains webhooks, services, or deploy keys, they will remain associated after the transfer is complete.

Webhooks

ZenHub, CLA, AWS Webhooks, AWS Code Build

Won't post details here, have info locally.

GitHub Apps

⚠️ The magento organization must have the bundlesize, Danger-JS, and Now Apps that the magento-research organization has. ⚠️

Danger-JS

Code Update

Covered by steps in pwa-studio-cicd.

API Token Update

AWS > Services > Systems Manager (ensure N. Virginia region) > Parameter Store

Update the API Token.

@sharkySharks , how can we get a new token for this?

Update Repository Access

Settings > Integrations & services > Danger-JS

Now

Code Update

I think the only thing we have to update is https://github.com/magento-research/pwa-studio/blob/a294b1fbad4c04b5a13e5af34ea26b1050f765f0/now.json#L7.

Edit: PR opened here: https://github.com/magento-research/pwa-studio/pull/1488.

Update Repository Access

Settings > Integrations & services > Now

bundlesize

Update Repository Access

Settings > Integrations & services > bundlesize

Miscellaneous

Docker

Nothing to do here. (Can you verify, @sharkySharks ?)

AWS CodePipeline

Update:

Builds: all of them
Pipelines: pwa-demo-deploy, pwa-stable

AWS CodeBuild

Update the primary repository

pwa-studio-cicd repository

Update all references to magento-research/pwa-studio, including dangerfile.js.

Edit: This PR is located here: https://github.com/magento/pwa-studio-cicd/pull/13

supernova-at on 29 Jul 2019

Record project collaborators' permission set

Admin

Note: we should probably restrict this list. I have starred each person who is not granted Admin privileges by way of GitHub Teams, and should be examined on a case-by-case basis.

Write

| User | Access |
| --- | --- |
| https://github.com/lighthousebot | Write |
| https://github.com/PWAStudioBot | Write |
| https://github.com/cherdman | Write |
| https://github.com/dmtrkad | Write |

Read

supernova-at on 29 Jul 2019

When we communicate the move to the community, this will be useful:

git remote set-url origin new_url

https://help.github.com/en/articles/changing-a-remotes-url

supernova-at on 29 Jul 2019

GitHub Pages Migration Plan

tl;dr

The existing docs site may 404 or serve stale content after the repository moves.

We will update the docs site before the move to redirect to the new location. This way if stale content is served, it will redirect to the new content.

We will also have to be proactive about communicating that the docs site has moved because GitHub will not automatically redirect traffic to it.

Overview / Context

Currently, our repository publishes to a GitHub Pages site.

It accomplishes this by maintaining a gh-pages branch. When this branch is edited, GitHub automatically updates https://magento-research.github.io/pwa-studio/.

We also have a custom domain http://pwastudio.io which redirects to https://magento-research.github.io/pwa-studio/. @ericerway owns http://pwastudio.io.

The Problem

In what's transferred with a repository, GitHub says:

If the transferred repository contains a GitHub Pages site, then links to the Git repository on the Web and through Git activity are redirected. However, we don't redirect GitHub Pages associated with the repository.

When we move the pwa-studio repository to the magento organization, the gh-pages branch will move along with it. Any edits to the gh-pages branch after the move will result in https://magento.github.io/pwa-studio/ being updated, not https://magento-research.github.io/pwa-studio/.

Which means that anyone that has bookmarked https://magento-research.github.io/pwa-studio/ will not be redirected to the new https://magento.github.io/pwa-studio/.

Because http://pwastudio.io is a simple redirect, most / all bookmarks will directly point to https://magento-research.github.io/pwa-studio/, not http://pwastudio.io itself.

Further, once we move the repository and there no longer exists a repo at magento-research/pwa-studio, we anticipate that the existing https://magento-research.github.io/pwa-studio/ site will 404.

Edit: In https://github.com/isaacs/github/issues/492, a user commented that they moved their repository to a different organization but their old repo's GitHub Pages site still stands (over two years later). Old: https://kanaka.github.io/noVNC/, New: https://noVNC.github.io/noVNC/

Long story short, we will be leaving a lot of folks with a bookmark to a site that won't exist or has stale content.

Possible Solutions

Wait

The idea here is that we put up a banner on the existing https://magento-research.github.io/pwa-studio/ site that lets visitors know that the docs site will be moving.

We'd need to decide how much time is "enough", but we wouldn't actually perform the move until the banner has been up for that amount of time.

Communicate

This is the same as Wait except we only let the banner sit for a couple days. We'd have to be much more proactive about communicating because it would be more likely that fewer people would see the move notice.

Another downside is that anyone who has the docs site bookmarked but isn't in our Slack wouldn't be notified of the move.

This is the same outcome as if they didn't visit the site and read the banner, however.

Pre-Move Update

In a GitHub issue comment, the kanaka team says:

We transferred kanaka/noVNC to noVNC/noVNC, but now the site http://kanaka.github.io/noVNC/ has stale data from the original account.

Indeed, two plus years after that comment, http://kanaka.github.io/noVNC/ is still serving stale content (http://noVNC.github.io/noVNC/ is the updated site).

The idea here is to update magento-research/pwa-studio's GitHub Pages site _before the move_ to redirect to the new https://magento.github.io/pwa-studio/ location.

In this way we can preserve all existing bookmarks to https://magento-research.github.io/pwa-studio/.

Note that there will be a short amount of time between when we make this change and when the repository gets moved that the docs site will 404 (the new site won't exist yet).

Re-create PWA Studio in magento-research

The repo would only contain the gh-pages branch which would keep up the banner that links visitors to the new docs site.

This would prevent https://magento-research.github.io/pwa-studio/ from 404ing.

Unfortunately, this would probably also break any automatic redirecting that GitHub would do for us so that HTTP(S) / SSH traffic gets redirected to magento/pwa-studio. Therefore we do not think this solution is a good one.

supernova-at on 30 Jul 2019