Google-api-python-client: Consider vendoring uritemplate

[theacodes]

We currently depending on uritemplate, however, there is another library named uritemplate.py that unfortunately has the same import path. Because of this if someone uses this library and another library that depends on uritemplate then neither work.

uritemplate is one file and hasn't changed since 2013. - should we just consider vendoring it?

[nathanielmanistaatgoogle]

Were you around for issue 57 and pull request 146?

I'm open to vendoring it, but let's exhaust all better options first? I'm tempted to write the maintainers of both packages and just say "yo, what's up?".

[theacodes]

As luck would have it @jcgregorio (a fellow Googler) is the maintainer of uritemplate and @sigmavirus24 is the maintainer of uritemplate.py. I think it's just an unfortunate coincidence. They seem to have both implemented an RFC at the same time.

It seems the uritemplate has been installed significantly more than uritemplate.py, but I don't think that number really means anything.

There's only two ways to resolve this - have us vendor one of them so our users don't run into conflicts, or find some way to convince Joe or Ian to rename their package.

[theacodes]

@nathanielmanistaatgoogle so @sigmavirus24 just notified that uritemplate.py is api-compatible with uritemplate. I think that @jcgregorio has moved on to other things, so it might be our best move to switch?

[jcgregorio]

@jonparrott Yes, it's been years since I did Python, so finding a better maintained package would be a good idea.

[sigmavirus24]

Someone suggested merging the two. That discussion never went anywhere. I'm also open to resolving any problems people have. As it is, the same test examples that uritemplate uses pass against uritemplate.py so ... you should run into very few problems. If you do, I'm more than happy to fix and release them.

[nathanielmanistaatgoogle]

While we have both @sigmavirus24 and @jcgregorio here let's shoot the moon: should the two projects be merged, with uritemplate.py entirely replacing uritemplate?

[theacodes]

It seems like since uritemplate's test suite passes against uritemplate.py they are effectively merged. I vote we switch - it seems it would be trivial for us.

[theacodes]

We could further 'merge' by turning uritemplate into an alias package that just depends on uritemplate.py if @jcgregorio is willing to give up the pypi package.

[nathanielmanistaatgoogle]

@jonparrott: that's where I'm trying to drive the conversation - why solve it for us if we can solve it for everyone?

@sigmavirus24: why and how was it that that conversation never went anywhere last time?

[sigmavirus24]

@nathanielmanistaatgoogle it was on uritemplate's tracker. Someone else took uritemplate.py and sent a PR with it to "merge" them. That annoyed someone over there and they shot down further conversations on merging somehow. I didn't want to annoy people so I didn't try bringing it back up because uritemplate.py already does what I need it to for my use cases and is compatible with uritemplate.

[sigmavirus24]

To be clear, I'm happy to merge things in either direction. I'm still maintaining uritemplate.py and have been meaning to cut a release, but the main purpose of that release is to have 1.0 to signify that it's stable and there should be 0 API breakage. I think the problem in the past was that someone initialized the merge by sending a PR and that surprised, confused, and irked the other project.

[nathanielmanistaatgoogle]

Is there anyone other that @mnot and @graingert who should be brought into a merge-uritemplate-and-uritemplate.py conversation?

Based on the facts and feelings shared so far, I propose:

1. @jcgregorio and @mnot share or transfer control of the uritemplate PyPI package with @sigmavirus24.
2. @sigmavirus24 pushes code to one of the uritemplate PyPI package and the uritemplate.py PyPI package and sets the other to redirect to the first.
3. The (original) uritemplate codebase is marked as deprecated/superseded/no longer active.
4. One of the two packages is "blessed" to be the package name by which the code will be known going forward and pip install examples are updated to reflect that choice.

Some questions:

1. Both codebases have merged changes since their last release. Are we certain that it's still the case today that uritemplate.py is universally substitutable for uritemplate?
2. It's great to reduce two implementations to one, but if uritemplate has tests that uritemplate.py passes, those tests should be retained, right? Unless uritemplate's tests are demonstrably a subset of uritemplate.py's tests, right?
3. Does PyPI even allow one project to be marked as a "redirect" to another, or would we have to accomplish the same effect hackily by having one of the packages be a no-code package with a dependency on the other?
4. As the author of a package with a name other than the name by which the code is imported in the Python code that uses it, I've been told that it's preferable to for a PyPI package name to match the Python package name used within the package. Would it be reasonable to ask that uritemplate be the PyPI package that becomes authoritative after the merge?

My having proposed merging one way doesn't mean that I haven't missed anything or that other proposals shouldn't be made. What are some alternate proposals? What have I overlooked? What feathers have I ruffled, and whose?

[theacodes]

would we have to accomplish the same effect hackily by having one of the packages be a no-code package with a dependency on the other?

That's essentially what we'd have to do.

[mnot]

As long as it's truly compatible, this WFM. @jcgregorio?

[jcgregorio]

Sounds good to me. Let me know the ids of the people to add to the pypi package.

[nathanielmanistaatgoogle]

@jcgregorio: "graffatcolmingov", as listed at the bottom of the uritemplate.py PyPI page.

[graingert]

that's @sigmavirus24 or [email protected]

[graingert]

v1 was just released: https://github.com/sigmavirus24/uritemplate/commit/a8e3c38d80f31c280da42f4c7d6b565640d24bd1

[sigmavirus24]

So the two libraries are absolutely API compatible. The one problem we'll have is that they're licensed differently. Granted, they're both OSI Approved and permissive (Apache 2 and BSD 3-Clause) but this may be an issue for some folks.

I can probably get permission from the other 4 people to relicense as Apache (or maybe dual-license?) that way people who are expecting an Apache licensed project can continue to use uritemplate.py under that license.

Do y'all think it's worth the effort?

[nathanielmanistaatgoogle]

It doesn't strike me as worth the effort.

I seriously doubt that anyone is going to look at uritemplate 0.6, see that it is Apache-licensed, look at uritemplate 1.0.1, see that it is BSD-3-Claused-licensed, and say "whoa, I'd better put a <=0.6 on my dependency!".

I don't think I could even describe the differences between the two licenses without looking them up.

[jcgregorio]

Strongly disagree, this should be done on a major version # boundary and should be announced. It would be problematic for people that need to roll up all the licenses of the libraries that they depend upon.

[nathanielmanistaatgoogle]

@jcgregorio: that surprises me but I'm perfectly happy to defer to your judgement. Luckily we're at a major version number boundary and a license change notice would fit perfectly well into a "hey we've unified the two independently-written separate implementations" announcement. Right?

Via what channels should the announcement be... announced? Does uritemplate have a mailing list associated with it?

[jcgregorio]

Hmm, I thought I had a list setup. _shrug_ I guess a major version boundary bump will have to be sufficient. I'll also announce on twitter and on my blog.

[sigmavirus24]

So there has been back and forth but no decision as to which package will live on as _the_ package. Frankly, I don't care. It seems like @jonparrott is leaning towards having uritemplate depend on uritemplate.py. I don't think we can rely on download counts because they're inaccurate measures of usage in the first place and apparently close enough to make it hard to figure out which is better.

Sorry for the delay in responses. I'm at Abstractions.io right now. I suppose if anyone else is here we could chat about this in person.

[theacodes]

Indeed I am leaning towards uritemplate.py being the _real_ package. It's historically tied to what is now the canonical repository, so I'd prefer that.

[sigmavirus24]

It's historically tied to what is now the canonical repository, so I'd prefer that.

I'm confused as to which is the canonical repository. I do think it's funny though that uritemplate has the repository name uritemplate-py while uritemplate.py (which PyPI will reify to uritemplate-py under the hood) has the repository name uritemplate.

Are we talking about unifying in one repository or another too or just having one package point to another? I think I've lost track of what we're discussing at this point (could also just be travel fatigue).

[graingert]

Clearly there should be one package uritemplate: hosted at
uritemplate/uritemplate, imported as uritemplate and pip installed as
uritemplate.

uritemplate.py should depend uritemplate for backwards compatibility.

On 18 Aug 2016 10:18 pm, "Ian Cordasco" [email protected] wrote:

It's historically tied to what is now the canonical repository, so I'd
prefer that.

I'm confused as to which is the canonical repository. I do think it's
funny though that uritemplate has the repository name uritemplate-py
while uritemplate.py (which PyPI will reify to uritemplate-py under the
hood) has the repository name uritemplate.

Are we talking about unifying in one repository or another too or just
having one package point to another? I think I've lost track of what we're
discussing at this point (could also just be travel fatigue).

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/google/google-api-python-client/issues/266#issuecomment-240860133,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAZQTFpkduN201AWyuQ_L4Ftzy4UlGf3ks5qhMw8gaJpZM4JkwQC
.

[theacodes]

I'm confused as to which is the canonical repository. I do think it's funny though that uritemplate has the repository name uritemplate-py while uritemplate.py (which PyPI will reify to uritemplate-py under the hood) has the repository name uritemplate.

lol that is funny.

So, if uritemplate.py (the package) is to become the "one true package" I would prefer that we leave it alone. It should have an unbroken history.

On the other hand, uritemplate is being superceeded by uritemplate.py. It's the package that's actually 'going away' so to speak, so it makes sense for its history to end with a package that just depends on uritemplate.py.

[theacodes]

Seems like @graingert disagrees maybe?

[nathanielmanistaatgoogle]

@sigmavirus24: I did propose (question four, toward the end) that uritemplate be the package that lives on as _the_ package. I think the long-term simplicity of the shorter, simpler name is of more value than the historical detail of "this implementation was once distributed under a different name".

I'm also attracted to the idea of using uritemplate/uritemplate as the repository location.

So @jonparrott, I also disagree. Let's look forward to _future_ developers knowing the package by one, simple, un-suffixed-in-all-contexts name.

[theacodes]

Fair enough, consider me a turncoat.

[sigmavirus24]

Okay, so does someone here own the uritemplate organization? Because what is currently in use is uri_-_template_s_.

[sigmavirus24]

Also, it seems odd for me to have authored the code that will be uploaded to uritemplate when the author of that package since 2013 has been a different person. I get the "one true name" desire, but it makes me uncomfortable to upload code Joe didn't author to that package. I get it's "just metadata" but I think it's important metadata. ¯\_(ツ)_/¯

[graingert]

The author metadata comes from your setup.py

On 18 Aug 2016 10:47 pm, "Ian Cordasco" [email protected] wrote:

Also, it seems odd for me to have authored the code that will be uploaded
to uritemplate when the author of that package since 2013 has been a
different person. I get the "one true name" desire, but it makes me
uncomfortable to upload code Joe didn't author to that package. I get it's
"just metadata" but I think it's important metadata. ¯_(ツ)_/¯

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/google/google-api-python-client/issues/266#issuecomment-240867539,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAZQTNsPQM0Y5FGebRv3Q2bz3Qt4GKvRks5qhNLugaJpZM4JkwQC
.

[nathanielmanistaatgoogle]

@sigmavirus24: From the way it has zero users and zero repositories I have a sinking feeling that the organization was just created in the last ten minutes. :-P Anyway, it's completely separable from actions we're taking with PyPI.

@sigmavirus24: The "author" field on a PyPI package can be set manually, set by a setup.py file, or not set at all. It can be one name or many. Across the life of a typical open-source codebase it starts at one, grows to two or three, then five, then says "refer to CONTRIBUTORS.md", and then is removed as redundant and unnecessary. If PyPI were such that only one author could ever be named on a package, that would be vulnerable to cookie-licking. I counsel you not to worry about it.

[sigmavirus24]

The author metadata comes from your setup.py

I know where it comes with. The thing is, some people (e.g., packagers) look at that and when it changes their going to be suspicious. I won't be maintaining that code base, I'll be replacing it. Regardless of blog posts, tweets, or anything else, it's going to cause some non-trivial amount of confusion. If everyone here thinks this isn't going to be a problem, I'll push uritemplate.py to uritemplate when I get PyPI privileges

[nathanielmanistaatgoogle]

I think it'll be a tiny bit confusing, but less confusing than the continued existence of two feature-equivalent, API-equivalent, separately-implemented, similarly-named, cannot-both-be-dependencies-of-the-same-third-codebase codebases and packages. I think it certainly won't be a _problem_. I'm keeping my eyes on the prize. :-)

[sigmavirus24]

So let's do the first thing:

• Create an upload to uritemplate. I still don't have that permission which you can see here.
• Then let's shave yaks over which repository it should live in.

Also we're waiting on the last person to consent to relicensing to hopefully make this as painless for users who care about licenses as possible.

[jcgregorio]

Added graffatcolmingov as owner:
https://pypi.python.org/pypi/uritemplate/0.6

[sigmavirus24]

So https://pypi.org/project/uritemplate/ has a 2.0.0rc1 version uploaded for people to test. I realized after the fact that I need to update some references in the README, but those are minor.

A week from Monday, I will release the final version of uritemplate and then release a version of uritemplate.py that depends on uritemplate.

[theacodes]

thanks, everyone! :D

[sigmavirus24]

Thank you, @jonparrott, and @nathanielmanistaatgoogle for properly mediating this discussion. 🍰

[theacodes]

Closing this issue, as I think we got the best resolution that could be hoped for. :)

[nathanielmanistaatgoogle]

@jonparrott: I was holding off on "A week from Monday [so, today], I will release the final version of uritemplate and then release a version of uritemplate.py that depends on uritemplate.".

@sigmavirus24: Is that still going to happen today?

[sigmavirus24]

Yes, I just didn't get the chance to do it this morning.

[sigmavirus24]

uritemplate 2.0 was released and uritemplate.py 3.0.1 was released depending on the former.

🎉

[nathanielmanistaatgoogle]

@sigmavirus24: any inclination to update the documentation at the old uritemplate.py PyPI page?

[grovduck]

Forgive this (likely naive) question, but based on this conversation, should setup.py for this package be changed to uritemplate>=3.0. It's very possible that I'm not understanding all that's involved here ...

[graingert]

It should be '>0.6 <4'

[theacodes]

yep. we'll update it on the next release.

[nathanielmanistaatgoogle]

@jcgregorio: I like to think of myself as at least as against censorship and compelled speech as anyone, but... should the content at this page and this page be updated to reflect the project's and code's end-of-life status and to suggest that the reader look to the currently-authoritative repository for supported code?

[nathanielmanistaatgoogle]

@sigmavirus24: now that I learn of @jcgregorio's uritemplate-test repository I'm curious to learn whether or not the now-unified uritemplate implementation runs against and passes those tests?

[sigmavirus24]

@nathanielmanistaatgoogle it does. We integrated those tests a long time ago. (Before we released the projects under both names.)