Graphql-ruby: Order-dependent spec failures with "No type found for <X>" using rspec / capybara / phantomjs

[khamusa]

Hi guys,

first of all thanks for this amazing job on graphql-ruby!

In a rails 4.2 setup our test suit is now failing randomly according to the order in which the graphql type definition files are loaded (currently by rails autoload mechanisms).

The following two spec files fail when running on the given order but pass when inverting the order:

# root_query_type_spec.rb
require 'rails_helper'

RSpec.describe 'RootQueryType' do
  subject(:type) { RootQueryType }

  it 'has a name' do
    expect(RootQueryType.name).to eq('LetsEventsAPI')
  end
end

And this capybara feature test running with phantomjs:

require 'rails_helper'

RSpec.feature 'On event page', js: true do
  # ... fixtures

  scenario 'can create list' do
    visit event_path public_event
    within '[data-events-show-stage]' do
      expect(page).to have_selector(:css, 'a[href="#admin/lists/new"]')
    end
  end
end

This second spec trigger an ajax request to retrieve data from the graphql server. The relevant part of the query:

query EventAdminLists(..., $listsFilter: ListPermissionsFilterEnum, ...) { ... }

Which fails with "#" on the controller handling the graphql request.
ListPermissionsFilterEnum is defined in a separate file and is just a normal enum. It is only used once in EventType, to which I added a few debug statements to understand the order in which files are loaded:

puts 'DEBUG ######## Loading EventType (file)'

EventType = GraphQL::ObjectType.define do
  puts 'DEBUG ######## Defining EventType'
  name 'Event'

  field :lists, !types[EventListType] do
    puts 'DEBUG ######## Defining :lists field on EventType (which has an argument of type ListPermissionsFilterEnum)'
    argument :filter, ListPermissionsFilterEnum # this is the enum type that cannot be found
    # ...
  end
end

I've also added similar statements to the RootQueryType which has a field 'event' of type EventType. Here is the output when the specs run in the failing order:

DEBUG ######## Loading RootQueryType (file)
DEBUG ######## Defining RootQueryType
DEBUG ######## Loading EventType (file)
DEBUG ######## Defining EventType
  has a name

On event page
  signin user
    when user is list_creator
DEBUG ######## Defining :lists field on EventType (which has an argument of type ListPermissionsFilterEnum)
Exception caught: #<RuntimeError: No type found for 'ListPermissionsFilterEnum'>
      can create list (FAILED - 1)

Now the execution with a different seed, which does not fail:

On event page
  signin user
    when user is list_creator
DEBUG ######## Loading RootQueryType (file)
DEBUG ######## Defining RootQueryType
DEBUG ######## Loading EventType (file)
DEBUG ######## Defining EventType
DEBUG ######## Defining :lists field on EventType (which has an argument of type ListPermissionsFilterEnum)
DEBUG ######## Defining ListPermissionsFilterEnum
      can create list

RootQueryType
  has a name

So it seems the issue is caused by rails's on-demand loading of graphql types. Does anyone has a clue on how to ensure types are loaded properly?

_EDIT_: fixed typos and trimmed unrelated code

[rmosolgo]

Thanks for this great writeup! I totally agree with your diagnosis of the situation, now let me think about it for a moment...

[rmosolgo]

My instinct is that it has _something_ to do with the Schema.define { ... } block.

Could you add a debug statement to that block, like the other debugs that you inserted?

I wonder if some specs _don't_ load the schema definition, and then something gets messed up. (I'm not exactly sure what would be messed up, but it seems like a possibility!)

[rmosolgo]

If it _is_ an issue with Schema.define { ... }, you could work around it by loading the schema inside rails_helper.rb. That might work in the short-term while we look into a proper fix!

[khamusa]

Sure! Thanks for the super fast reply :)

debug statements:

puts 'DEBUG ######## Loading LetsGraph::Schema (file)'
module LetsGraph
  module Config
    DEFAULT_MAX_PAGE_SIZE = 150
  end

  Schema = GraphQL::Schema.define do
    puts 'DEBUG ######## Defining LetsGraph::Schema'
    query RootQueryType
    mutation RootMutationType
  end
end

Failing output:

DEBUG ######## Loading RootQueryType (file)
DEBUG ######## Defining RootQueryType
DEBUG ######## Loading LetsGraph::Schema (file)
DEBUG ######## Defining LetsGraph::Schema
DEBUG ######## Loading EventType (file)
DEBUG ######## Defining EventType
  has a name

On event page
  signin user
    when user is list_creator
DEBUG ######## Defining :lists field on EventType (which has an argument of type ListPermissionsFilterEnum)
Exception caught: #<RuntimeError: No type found for 'ListPermissionsFilterEnum'>
      can create list (FAILED - 1)
  HTML screenshot: file:///home/sam/projects/lets/screenshot_2016-12-28-17-31-45.777.html
  Image screenshot: file:///home/sam/projects/lets/screenshot_2016-12-28-17-31-45.777.png

Passing output:

On event page
  signin user
    when user is list_creator
DEBUG ######## Loading LetsGraph::Schema (file)
DEBUG ######## Defining LetsGraph::Schema
DEBUG ######## Loading RootQueryType (file)
DEBUG ######## Defining RootQueryType
DEBUG ######## Loading EventType (file)
DEBUG ######## Defining EventType
DEBUG ######## Defining :lists field on EventType (which has an argument of type ListPermissionsFilterEnum)
DEBUG ######## Defining ListPermissionsFilterEnum
      can create list
RootQueryType
  has a name

This actually gave me an insight. The RootQueryType spec shouldn't actually need to load the schema, since it's a type tested in isolation. But looking at the type definition I just realized this:

RootQueryType = GraphQL::ObjectType.define do
  puts 'DEBUG ######## Defining RootQueryType'
  # ...
  connection :events, max_page_size: LetsGraph::Config::DEFAULT_MAX_PAGE_SIZE do
  #...
  end

Both the LetsGraph::Schema and LetsGraph::Config constants are defined in the same file.
Because of this Config submodule, the failing scenario loads the schema, which it actually doesn't need.

if I remove the reference to LetsGraph::Config::DEFAULT_MAX_PAGE_SIZE in RootQueryType, the specs pass:

DEBUG ######## Loading RootQueryType (file)
DEBUG ######## Defining RootQueryType
# Schema is not loaded here anymore
DEBUG ######## Loading EventType (file)
DEBUG ######## Defining EventType
  has a name

On event page
  signin user
    when user is list_creator
DEBUG ######## Loading LetsGraph::Schema (file)
DEBUG ######## Defining LetsGraph::Schema
DEBUG ######## Defining :lists field on EventType (which has an argument of type ListPermissionsFilterEnum)
DEBUG ######## Defining ListPermissionsFilterEnum
      can create list

If I could guess anything from this, I'd say the issue is on loading the root node (schema) between the loading of two intermediate nodes. Unsure we can generalize though.

For the time being I'll just separate the two constants in two different files, but I'd be glad to help on finding a more robust solution.

Thanks for helping out! :tada:

[rmosolgo]

Glad you found something that works for now!

Obviously graphql-ruby should support this case, so I'll keep this ticket open until I can nail down the issue and fix it.

Thanks again for the great bug report :)

[rmosolgo]

I haven't heard more reports of this, so i'm closing as stale.