Google-cloud-python: RTD build is broken

[waprin]

Can look at this, leaving as note as reminder.

[daspecster]

I'm not sure what this means?

They appear to be populated?
https://googlecloudplatform.github.io/google-cloud-python/latest/storage-client.html

[waprin]

Sorry opened issue with customer next to me.

@daspecster realizing its' the read the docs docs that are missing.

https://google-cloud-python.readthedocs.io/en/latest/storage-client.html

[dhermes]

Indeed, another example: https://google-cloud-python.readthedocs.io/en/latest/datastore-transactions.html

Latest build is green (but must be secretly not green): https://readthedocs.org/projects/google-cloud-python/builds/5028739/

[dhermes]

@tseaver @daspecster We should tackle this. I'm not sure if there is another issue, but the RTD build has been b0rken for a few days:

https://readthedocs.org/projects/google-cloud-python/builds/

e.g.

https://readthedocs.org/projects/google-cloud-python/builds/5060236/

[dhermes]

OK the fix is easy, we just need to get all subpackages on the same version of google-cloud-core. DERP!

[bjwatson]

This is a P0 issue that must be fixed ASAP. It's a poor user experience.

How can we avoid this with future google-cloud-core releases?

[daspecster]

@bjwatson #3056 should do it AFAICT.

[dhermes]

@bjwatson It's not the release, it's just a broken build.

[daspecster]

Looks like it's still broken :(
https://readthedocs.org/projects/google-cloud-python/builds/5066476/

[dhermes]

Jinx.

I think it's because of cached deps, so i'm blowing up the cache.

[dhermes]

It was because of cached deps, but now it's because of an accidental missing comma in proto-google-cloud-pubsub-v1==0.15.0.

@lukesneeringer is on the case!

[lukesneeringer]

Waiting on an LGTM and I will push the packages.

[lukesneeringer]

Packages to be added / pushed to 0.15.1 (proto only):

All except noted are in https://github.com/googleapis/api-client-staging/pull/176.

• [x] datastore (0.90.1)
• [x] errorreporting
• [x] language
• [x] monitoring
• [x] pubsub (https://github.com/googleapis/api-client-staging/pull/177 -- changed: was 174)
• [x] spanner-admin-database
• [x] spanner-admin-instance
• [x] spanner
• [x] speech (https://github.com/googleapis/api-client-staging/pull/175)
• [x] vision

[dhermes]

Whoa that's a lot! Thanks @lukesneeringer

[lukesneeringer]

Sooner or later I am going to automate this... :-)

[lukesneeringer]

All launched except Speech.

[lukesneeringer]

Speech launched.

[dhermes]

Builds still failing, but it's because the umbrella package refers to versions other than the ones in source (this is an argument against getting too far out of sync)

[lukesneeringer]

Okay. Let's make a concerted effort to get this across the finish line tomorrow morning.

[dhermes]

Still b0rken but now it's not our fault: https://readthedocs.org/projects/google-cloud-python/builds/5071900/

pkg_resources.UnknownExtra: proto-google-cloud-datastore-v1 0.90.1 has no such extra feature 'grpc'

@lukesneeringer and I are aware of this, but we weren't aware it'd keep RTD from building

[bjwatson]

At least the /latest/ RTD is showing content now: https://googlecloudplatform.github.io/google-cloud-python/latest/datastore-usage.html

[daspecster]

@bjwatson I was looking at /latest/ the other day and it was fine? Was there a time when it wasn't showing content?

[dhermes]

@bjwatson That isn't RTD, this is: http://google-cloud-python.readthedocs.io/en/latest/datastore-usage.html

[bjwatson]

Ugh, you're right. Sorry, I'm juggling too many things at once right now.

Is there a plan to resolve the remaining issue?

[bjwatson]

@daspecster Yes.

[lukesneeringer]

Yeah, I know what needs to happen to fix that. It is just datastore, so it should be pretty easy.

[bjwatson]

Ok, cool.

[lukesneeringer]

Correction: It is every proto- package, so after I fix it I will have to regenerate all of them, and every GAPIC.

[bjwatson]

Got it. Will we make grpc an option for every proto- package?

[lukesneeringer]

Yeah, that is exactly what needs to happen. We thought we were thorough when we changed grpc to optional but clearly I made a small mistake here (which does not break installs but does break RTD).

Side note: Apparently something like six separate problems were _all_ sufficient individually to break RTD builds. There is no way these have been good for some time now.

[lukesneeringer]

Another note: A high priority needs to be that RTD builds from CI. We need to know about failures without having to check RTD.

[lukesneeringer]

Correction to my correction: Just Datastore.

[dhermes]

@lukesneeringer We do build docs on CI, we just use different flags than RTD uses.

[daspecster]

We had one build pass a couple days ago but we're back to this it looks like.

pkg_resources.UnknownExtra: proto-google-cloud-datastore-v1 0.90.1 has no such extra feature 'grpc'

See: https://readthedocs.org/projects/google-cloud-python/builds/5080674/

[lukesneeringer]

@dhermes You said this is fixed, correct?

[dhermes]

As of https://readthedocs.org/projects/google-cloud-python/builds/5081679/ it seems to be fixed (a new build after it also passed: https://readthedocs.org/projects/google-cloud-python/builds/5081815/)

[guy-shahine]

Is it fair to say that documentation links are broken again?

https://googlecloudplatform.github.io/google-cloud-python/latest/pubsub/usage.html throws a 404 error

I got the link from here https://github.com/GoogleCloudPlatform/google-cloud-python/tree/master/pubsub by clicking on the link that says Documentation

[dhermes]

@chlela No, that URL no longer has documentation (Pub/Sub recently had a very large change, docs included).