Rubygems.org: Reverse dependencies page error 500

[markets]

Hi 👋

I'm trying to visit the /gems/GEM_NAME/reverse_dependencies page for different gems and I get a 503 error:

Unicorn!
We are having a bad problem and the app server will not talk to us.

Examples (gems under my account):

• https://rubygems.org/gems/mini_i18n/reverse_dependencies
• https://rubygems.org/gems/invisible_captcha/reverse_dependencies

Then, I realized this page crashes also for other gems:

• https://rubygems.org/gems/rails/reverse_dependencies
• https://rubygems.org/gems/bundler/reverse_dependencies
• https://rubygems.org/gems/sinatra/reverse_dependencies

[aleksandrs-ledovskis]

Interestingly enough, REST API seems to be unharmed / can be used while web page is down.

https://rubygems.org/api/v1/gems/mini_i18n/reverse_dependencies.json

[dwradcliffe]

Sorry for the delayed response here. We had to disable this endpoint for now because the performance was just soooo bad that it was breaking the entire site.

It would be great if someone could look at this and see what we can do to make these queries faster.

[sonalkr132]

Half the response time is count(distinct ...) query, for total entries count used by will_paginate. I am working on migration to kaminari, which has option to only show next and previous page navigation.
Feature was requested on will_paginate but it was not accepted.

[markets]

Thank you guys for your response and taking a look!

@aleksandrs-ledovskis maybe this is because API doesn't respond with pagination? Since @sonalkr132 is pointing out that pagination is what causes the issue...

It seems so: https://github.com/rubygems/rubygems.org/blob/e847fdd30f428da2909fc22bf2bfeb94fa6e4b0c/app/controllers/api/v1/versions_controller.rb#L30-L36

[sonalkr132]

pagination is half the problem. Other half is by_downloads (ORDER BY gem_downloads.count) used in controller. I don't think we can keep order by downloads either.

[simi]

@sonalkr132 are there any easy steps to reproduce this problem? Is loading data dump and visitng that url locally enough?

[sonalkr132]

Is loading data dump and visitng that url locally enough?

yes

[sonalkr132]

I don't think we can keep order by downloads either.

My PR cuts response time in half but it is still going to be above 700-800ms for popular gems. Removing by_downloads brings it down to ~200ms.
@dwradcliffe what would be acceptable response time here? Could we use rails cache?

[dwradcliffe]

Our overall goal for the service is 100ms, but 200ms should be ok for this endpoint. The before disabling it, this endpoint was averaging 2700ms.

Also we were getting some traffic with garbage query string values which bypasses Fastly. So I think it would make sense to use Rails cache to ensure that repeated calls for the same gem are cached.

[dwradcliffe]

Well it seems like the kaminari PR was a huge help! I've re-enabled this endpoint and it's looking better over the last 12 hours.

[markets]

Thank you guys for working on this! 👏