Polis: Run Apr 3 community call

Created on 4 Apr 2020  ·  22Comments  ·  Source: compdemocracy/polis

Re-ticketed from https://github.com/pol-is/polis-issues/issues/127

We ran a call today (🙋‍♂featuring @ballPointPenguin, @crkrenn, @david-nadaraia, @metasoarous @patcon) and some notes were taken 🎉 We can use this issue to track small follow-up actions, spin out larger items into new tickets, and generally plan next steps. Call out any todos I missed

I'll add all potential actions that came up, but some might not get attention in near-term.

📝 Notes: https://hackmd.io/1U58Q-KOSpeDUagCMCDwzQ?view

Decisions

  • Gitter continues to be the synchronous chat space
  • Everyone +1 on monorepo
  • Open to multiple deploy tactics for now
  • docs in repos (docs/ dir), managed via pull request for now
  • testing is good
  • ...
  • @patcon tentatively nominated as community steward

To Do (Candidates)

(Pls feel free to consider the above as shared content, and editable if you have push access)

picture patcon
👍1

Most helpful comment

To clarify, I don't think we're expecting all the above to be checked off before we close this, but just wanted to capture what was talked about in an issue :)

As for monorepo, how's this look as a general structure? (polis-issues untouched for now)

polis/
├── CONTRIBUTING.md
├── README.md          # minimal root readme
├── docker-compose.yml
├── ...
├── docs
│   └── deployment.md  # from polis-deploy
├── polisClientAdmin
│   ├── Dockerfile
│   ├── README.md
│   └── ...
├── polisClientParticipation
│   ├── Dockerfile
│   ├── README.md
│   └── ...
├── polisMath
│   ├── Dockerfile
│   ├── README.md
│   └── ...
└── polisServer
    ├── Dockerfile
    ├── README.md
    └── ...
picture patcon on 4 Apr 2020
👍2 ❤1 👎1

All 22 comments

To clarify, I don't think we're expecting all the above to be checked off before we close this, but just wanted to capture what was talked about in an issue :)

As for monorepo, how's this look as a general structure? (polis-issues untouched for now)

polis/
├── CONTRIBUTING.md
├── README.md          # minimal root readme
├── docker-compose.yml
├── ...
├── docs
│   └── deployment.md  # from polis-deploy
├── polisClientAdmin
│   ├── Dockerfile
│   ├── README.md
│   └── ...
├── polisClientParticipation
│   ├── Dockerfile
│   ├── README.md
│   └── ...
├── polisMath
│   ├── Dockerfile
│   ├── README.md
│   └── ...
└── polisServer
    ├── Dockerfile
    ├── README.md
    └── ...
patcon picture patcon on 4 Apr 2020
👍2 ❤1 👎1

Thinking it'd be rad to just merge bare structural changes first, and unblock other stuff asap so won't be merge conflicts later -- the rest of the monorepo'izing will involve poaching pieces from readmes.

Does this look to merge in?
https://github.com/pol-is-trial-balloon/polisServer/tree/monorepo

flagging that any non-master branches from non-polisServer repos are currently lost. That ok? Maybe we could inventory ones worth saving, if needed?

EDIT: used this https://medium.com/altcampus/how-to-merge-two-or-multiple-git-repositories-into-one-9f8a5209913f

patcon picture patcon on 5 Apr 2020

Hey @patcon,

This looks great!

A note for others, you need to use "git log --follow " to see the commit history on the command line.

I think the following recipe could maintain all branches. I have started but not yet finished.

https://saintgimp.org/2013/01/22/merging-two-git-repositories-into-one-repository-without-losing-file-history/

-Chris

crkrenn picture crkrenn on 5 Apr 2020

Ah thanks Chris, good call :) We could bring them all in and the clear them out after.

Makes me realize I could probably capture this whole process in a bash script, and commit that for review/contribution instead :) Will do that now

patcon picture patcon on 5 Apr 2020

@patcon, FYI, I sent an email to your gmail address listed in your git commits.

crkrenn picture crkrenn on 6 Apr 2020

thanks @crkrenn -- haven't been as on top of email, but seeing it now.

As context, @crkrenn wrote a script: https://gist.github.com/patcon/3fc1e346f6c45a95ffdcdc08bc7cb21d

Looks great, but what if we left dirs same for now, and deal with namespace convo later (prob more things to change related to that).

The thing I was stuck on, that perhaps you'd have more luck with, was finding a batch way to rebase or merge the feature branches with the subdir layout. Related:

As-is, checking out any of the branches (e.g., polisClientParticipation/explore-styles) means we'd then be looking at the subdir's contents in the root of the repo (e.g., the contents of polisClientParticipation are in the polisCode root). Was trying to get it working as if the changes were against the subdir, and so any PR's wouldn't involve shuffling things around in repetitive and confusing ways.

The main hitch is that this could only be batched for existing feature branches that didn't already have merge conflicts even before all this

Question for others: Does the above feel worth doing? The main reason I was trying to do it, was because I was thinking that it'll be hard to evaluate languising branches when the diffs from our new master are massive and not representative of the actual changes.

EDIT: oh, and the results of @crkrenn's script is: https://github.com/pol-is-trial-balloon/polisCode

patcon picture patcon on 6 Apr 2020

@metasoarous et al, should documentation, reports, and notebooks be a part of the monorepo? I'd just been assuming math, clientadmin, clientparticipation, server, and deploy.

patcon picture patcon on 6 Apr 2020

Oh hey, this was my pending script fwiw: https://gist.github.com/patcon/267c5fc9ffa2abb319e73e54992b769c

patcon picture patcon on 6 Apr 2020

TL;DR; A suggestion for moving forward with the monorepo. Please let me know what y'all think. (@patcon, @metasoarous, @ballPointPenguin, @david-nadaraia, and @colinmegill in particular!)

  1. All current "pol-is" repos except for "openData" and "polis-issues" should be moved into the polisServer repo.
  2. The directory structure and naming of the new repo should be as follows:

    • clientAdmin

    • clientParticipation

    • deploy

    • documentation

    • math

    • notebooks

    • report

    • server

  3. Previous branches and file history will persist in the monorepo; a sub-tree merge is not necessary.
  4. The old repos (polisClientAdmin, etc.) should be renamed with a prefix of "xlegacy".
  5. There can be advantages to nominating one or more of @colinmegill, @mbjorkegren, & @metasoarous as benevolent dictator(s) for life BDFL.
  6. Red Hat's open decision framework can be valuable.

I can adapt my script to implement this recommendation (or modifications thereof).

Many details on other options and my decision making process follow ;).

Options and decisions for the monorepo:

  1. What should be in the monorepo?

    1. Everything
    2. Everything except openData
    3. Everything except polis-issues
    4. Just the code (polisServer polisMath polis-deploy polisClientAdmin polisClientParticipation polisReport)
    5. I would vote for everything but openData and polis-issues. A precedent is the LimeSurvey project. They have one main repo including examples. Their smaller repos are for side projects that might get merged in at some point, or for forks of other projects that they may modify for their own purposes.
    6. The advantage of leaving polis-issues out of the monorepo is that we don't have to worry about migrating issues, which seems even more complicated than merging repos google search.

  2. Do we use a subtree merge or not?

    • I recommend to not use a subtree merge because the purpose of subtree merges is to allow pulling from and pushing to existing repos, and we are proposing to stop development on polisClientAdmin, etc. after the merge.

  3. Do we keep old branches?

    • I'm in general a hoarder, so I vote yes. As @patcon pointed out, if you check out earlier branches, the subdir's contents will appear in the root of the repo. This is imperfect, but I don't think there is a way that git could maintain history in a consistent way. The code history is there if we need it.

  4. How should the monorepo relate to existing repos?

    1. It could be a new repo, for example https://github.com/pol-is-trial-balloon/polisCode
    2. One existing repo could be chosen to absorb the others:

      1. polisServer: this is @patcon's proposal and is a good choice because it has been forked and starred the most

      2. polis-issues: this has the advantage that most issues could stay where they are. After the merge this could be renamed to something more appropriate.

    3. After I learned that you can rename repos, I'm leaning towards ii. I don't know how important stars and forks are, but I would lean towards ii/a and to suggest leaving polis-issues its own repo for now

  5. Should the directory naming structure stay the same, or should it change?

    1. @patcon suggests staying the same for now to avoid additional needed changes

    2. I suggest changing now because the only place the current name space is referenced in the codebase are in readme files, comments, the docker-compose.yml file, and two places in the server/package.json file:
      ```
      (base) CRKFrankenBook:polisCode crkrenn$ grep -e polisMath -e polisServer -e polisClientAdmin -l -R *
      ClientAdmin/package.json
      ClientAdmin/README.md
      ClientParticipation/polis.config.template.js
      ClientParticipation/README.md
      Math/README.md
      Report/README.md
      Server/app.js
      Server/docker-compose.yml
      Server/package.json
      Server/README.md
      deploy/README.md

      (base) CRKFrankenBook:polisCode crkrenn$ grep -e polisMath -e polisServer -e polisClientAdmin -R Server/package.json
      Server/package.json: "url": "git://github.com/pol-is/polisServer.git"
      Server/package.json: "polisServerBrand": "git+https://github.com/colinmegill/polisServerBrand.git#master"
      ```

  6. Should the "polis" prefixes be removed?

    • I recommend yes based on the "don't add gratuitous context" principle from Clean Code and for consistency (polis-documentation vs. polisServer vs. notebooks)

  7. Should directories be in camelCase or hyphenated or a mixture?

    1. If we are changing directory names, consistency seems to be a good thing.
    2. The choice between camelCase and hyphenated seems to me to be an arbitrary decision that could be potentially delegated to a BDFL.
    3. I recommend camelCase because the only two places where it would still matter after removing polis prefixes are clientAdmin and clientParticipation.

  8. What do we do with the old repos?

    • As I mentioned before, I'm a hoarder, so I probably would rename them with a prefix of "legacy" (or "xlegacy")

Finally, some meta issues:

  1. A proposal for more efficient decision making: to nominate one or more of @colinmegill, @mbjorkegren, & @metasoarous as benevolent dictator(s) for life BDFL. I don't suggest that the BDFL would cut short discussion, but perhaps speed up decision making. (The choice between camelCase and hyphenated is an example.)

  2. I suggest it may valuable to refer to Red Hat's open decision framework

crkrenn picture crkrenn on 8 Apr 2020

Generally :+1: on your first 1-6. Is it ok to also leave out notebooks for now? (in context below)

re: old repos. :+1: on all name-changes, incl legacy archives. Also, can mark repos "archived" in github settings, and update readmes to point to right place. (can be done later)

re: open decision framework. looks neat! I feel in alignment with everything in there on first blush. As for what it means to "use" it, can we kick that to another issue, to keep this issue from becoming a mega-issue (any more than I already made it one :) )

re: BDFL. Not opposed :) though wondering if this namespacing convo could be doable without. Moving forward via "agreement" vs "delegating decision to an authority" both truly have a place. Admittedly, I'm keen to flex our ability with former, since we're still small. I feel hopeful we can sort it!

re: your script. I'm eager to use choices that you've made :) As aside, I'll often suggest/ask about something else (like my last msg), but pls don't think of it as "stalemate". If something I share doesn't resonate, I usually have a _"good enough for now; safe enough to try"_ philosophy of trying someone else's pref. I'm super happy to figure out my own balance between voicing my thoughts and restraining them, if you have patience for that :)

For next 1-8, assume agreement, though further comments:
4.ii.b) re: moving issues. this is doable for someone with permissions, and imho shouldn't affect which repo we choose to merge into. +1 for polisServer 4.ii.a

Proposal

how about this:

  • all docs and app code merged in (server, math, clientAdmin, clientParticipation, report, documentation)

    • these repos are a coherent deployable unit. E.g., a new feature in an app will expect doc changes.

  • repos merged into polisServer (conserving stars), renaming to polisCode in pol-is-trial-balloon org for now
  • after upstream merge, transfer issues between repos as appropriate (nothing gets lost)
  • leave notebooks/openData out (they're another unit, but not tied to the rest)
  • naming as you suggested
  • all branches preserved, without rebasing (too complicated)

    • start talking about culling them after we move to polisCode

Would the above be acceptable to you? also happy to have colin or metasoarous just make a call on any of this.

This is what the above would look like (but pushed to polisServer eventually):
https://github.com/pol-is-trial-balloon/polisCode-patcon-v2020-04-08

Here's the script and logs from the push that created the repo:
https://gist.github.com/patcon/267c5fc9ffa2abb319e73e54992b769c

The one downside of all this "preserve branches" thinking is that either (1) I need to have push access to pol-is/polisServer, or (2) someone with write permissions needs to do some force-pushing and branch cleanup. (since we're not going to open a PR for each of the 75+ merged branches)

patcon picture patcon on 8 Apr 2020

Please leave out documentation and notebooks

Documentation will soon be replaced and the repo should only really need a
readme with links to it

Notebooks really is a separate deliverable that only interacts with
exported data

Otherwise great!

On Wed, Apr 8, 2020, 3:24 PM Patrick Connolly notifications@github.com
wrote:

Generally 👍 on your first 1-6. Is it ok to also leave out notebooks
for now? (more below)

re: old repos. 👍 on all name-changes, incl archives. Also, can mark
repos "archived" in github settings, and update readmes to point to right
place. (can be done later)

re: open decision framework. looks neat! I feel in alignment with
everything in there on first blush. As for what it means to "use" it, can
we kick that to another issue, to keep this issue from becoming a
mega-issue (any more than I already made it one :) )

re: BDFL. Not opposed :) though wondering if this namespacing convo
could be doable without. Moving forward via "agreement" vs "delegating
decision to an authority" both truly have a place. Admittedly, I'm keen to
flex our ability with former, since we're still small. I feel hopeful we
can sort it!

re: your script. I'm eager to use choices that you've made :) As aside,
I'll often suggest/ask about something else (like my last msg), but pls
don't think of it as "stalemate". If something I share doesn't resonate, I
usually have a "good enough for now; safe enough to try" philosophy.
I'm super happy to figure out my own balance between voicing my thoughts
and restraining them, if you have patience for that :)

For next 1-8, assume agreement, though further comments:
4.ii.b) re: moving issues. this is doable for someone with permissions,
and imho shouldn't affect which repo we choose to merge into. +1 for
polisServer 4.ii.a
Proposal

how about this:

  • all docs and app code merged in (server, math, clientAdmin,
    clientParticipation, report, documentation)

    • these repos are a coherent deployable unit. E.g., a new feature

      in an app will expect doc changes.

  • repos merged into polisServer (conserving stars), renaming to
    polisCode in pol-is-trial-balloon org for now
  • after upstream merge, transfer issues between repos as appropriate
    (nothing gets lost)
  • leave notebooks/openData out (they're another unit, but not tied to
    the rest)
  • naming as you suggested
  • all branches preserved, without rebasing (too complicated)

    • start talking about culling them after we move to polisCode

Would the above be acceptable to you? also happy to have colin or

metasoarous just make a call on any of this.

This is what the above would look like (but pushed to polisServer
eventually):
https://github.com/pol-is-trial-balloon/polisCode-patcon-v2020-04-08

Here's the script and logs from the push that created the repo:
https://gist.github.com/patcon/267c5fc9ffa2abb319e73e54992b769c

The one downside of all this "preserve branches" thinking is that either
(1) I need to have push access to pol-is/polisServer, or (2) someone with
write permissions needs to do some force-pushing and branch cleanup. (since
we're not going to open a PR for each of the 75+ merged branches
https://github.com/pol-is-trial-balloon/polisCode-patcon-v2020-04-08/branches
)

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/pol-is/polis-issues/issues/128#issuecomment-611145265,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AANQGGL7QQBTUBCDRNCFSJLRLTFNVANCNFSM4L5L7DCA
.

colinmegill picture colinmegill on 8 Apr 2020
👍1

Makes sense! Updated gist script to remove polis-documentation, and force-pushed results to pol-is-trial-balloon/polisCode-patcon-v2020-04-08 demo repo

Are you +1 on how that repo looks @crkrenn? If so, I'll push it to pol-is-trial-balloon/polisServer and rename to polisCode, and then we can start clearing out junk branches among other things :)

Noted for @crkrenn:

patcon picture patcon on 8 Apr 2020

@crkrenn thoughts on just ignoring all automated snyk-fix-* branches while migrating?

If we turn snyk back on in the future, then most recent PRs will regenerate anyhow. When we drop those auto-generated branches (and the original master branches, in favour of the merged monorepo branch), then there are only 7 remaining branches aside from the monorepo mainline (instead of 70)

patcon picture patcon on 9 Apr 2020

Dropping those branches is reasonable

On Wed, Apr 8, 2020, 9:47 PM Patrick Connolly notifications@github.com
wrote:

@crkrenn https://github.com/crkrenn thoughts on just ignoring all
automated snyk-fix-* branches while migrating?

If we turn snyk back on in the future, the most recent PRs will regenerate
anyhow. When we drop those auto-generated branches (and the original
master branches, in favour of the merged monorepo branch), then there are
only 7 remaining branches aside from the monorepo mainline.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/pol-is/polis-issues/issues/128#issuecomment-611281810,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AANQGGPDGG3KEEUIKQVNDRLRLUSJZANCNFSM4L5L7DCA
.

colinmegill picture colinmegill on 9 Apr 2020
👍1

@patcon +1 overall on the proposal. I still suggest/request lower case "server", "math", etc.

A benefit of keeping snyk branches is that they provide a tractable way of prioritizing work. (It is easiest to fix the oldest snyk branch first.) But, I'm happy to keep the snyk branches in my fork and would support removing them from the monorepo.

A suggestion: state the importance of using "git log --follow " in the README.md because of the merge. And, do gui git clients need a similar user change?

Start pull requests after downcasing?

crkrenn picture crkrenn on 9 Apr 2020
👍1

@patcon, all good. Incrementalism in general is safe and good. But “killing two birds with one stone” can also be good when the added cost is small.

Very respectfully, -Chris

re: your script. I'm eager to use choices that you've made :) As aside, I'll often suggest/ask about something else (like my last msg), but pls don't think of it as "stalemate". If something I share doesn't resonate, I usually have a "good enough for now; safe enough to try" philosophy. I'm super happy to figure out my own balance between voicing my thoughts and restraining them, if you have patience for that :)

Q: Why is this email five sentences or less?
A: http://five.sentenc.es

crkrenn picture crkrenn on 9 Apr 2020

Going to move monorepo convo over to existing issue and reply there, just to make convo more manageable :)
https://github.com/pol-is/polis-deploy/issues/8

patcon picture patcon on 9 Apr 2020

In the process of spawning most of the todos from this meeting into their own tickets, so we can close this out :)

patcon picture patcon on 30 Apr 2020

@metasoarous are you able to help me close out action item (5)?

patcon picture patcon on 3 May 2020

And for (24) re: sharing emails with you.

Do this suffice? https://docs.google.com/spreadsheets/d/1ASDJ_9hgCnfLkX_rSzQ6m3hYpqVKQxY7rIFZg5djxiY/edit#gid=0

patcon picture patcon on 3 May 2020

Ok, I think this is good enough to close. The rest will just happen in the flow of things, and is hard to lose track of :)

🎉🎉🎉🎉🎉🎉🎉

patcon picture patcon on 9 May 2020

ansible-related: https://github.com/ballPointPenguin/polisServer/blob/playbook/playbook.yml

patcon picture patcon on 13 May 2020
Was this page helpful?
0 / 5 - 0 ratings

Related issues

patcon picture patcon  ·  6Comments
joshsmith2 picture joshsmith2  ·  9Comments
patcon picture patcon  ·  4Comments
joshsmith2 picture joshsmith2  ·  6Comments
patcon picture patcon  ·  7Comments