Relay: [DX] Improve testability of containers.

[alloy]

After trying and thinking about various solutions, which are described here, these are some of my suggestions that could ease fully testing containers.

• Generate a Flow/TS type that describes the shape of the full response for an operation. This would be helpful in knowing all the fixture data that the dev needs to provide _without_ reading all of the fragments of the containers involved and looking up the types of the fields selected in those fragments and manually putting that all together.

  In our tests we've been creating query renderers for subsets of an entire product view, so that we can test containers more in isolation and the data tree would be significantly smaller.

• Additional utility to stub a relay environment with the response fixture data, which allows a query renderer to then render synchronously (using the STORE_THEN_NETWORK cache config).

• What isn't clear yet is how to ease testing mutations. I think it could largely work the same, but thus far we have only done so using mocks and expectations, so any thought on that would be appreciated.

Example

An example of what I’d imagine a test would look like:

// ArtworkMetadata.tsx

import { createFragmentContainer, graphql } from "react-relay"
import { ArtworkMetadata_artwork } from "__generated__/ArtworkMetadata.graphql"

export const ArtworkMetadata: React.SFC<{ artwork: ArtworkMetadata_artwork }> = createFragmentContainer(
  ({ artwork }) => <span>{artwork.title} – {artwork.artist.name}</span>,
  graphql`
    fragment ArtworkMetadata_artwork on Artwork {
      title
      artist {
        name
      }
    }
  `
)

// ArtworkBrick.tsx

import { createFragmentContainer, graphql } from "react-relay"
import { ArtworkBrick_artwork } from "__generated__/ArtworkBrick.graphql"

export const ArtworkBrick: React.SFC<{ artwork: ArtworkBrick_artwork }> = createFragmentContainer(
  ({ artwork }) => <div><img src={artwork.image.url} /><ArtworkMetadata artwork={artwork} /></div>,
  graphql`
    fragment ArtworkBrick_artwork on Artwork {
      image {
        url
      }
      ...ArtworkMetadata_artwork
    }
  `
)

// ArtworkBrick.test.tsx

import { graphql, stubEnvironmentWithStore , QueryRenderer } from "react-relay"
import { mount } from "enzyme"
import { ArtworkBrickTestQuery, ArtworkBrickTestQueryUnmaskedResponse } from "__generated__/ArtworkBrick.test.graphql"

const query = graphql`
  query ArtworkBrickTestQuery($artworkID: ID!) {
    artwork(id: $artworkID) {
      ...ArtworkBrick_artwork
    }
  }
`

const artworkFixture: ArtworkBrickTestQueryUnmaskedResponse = {
  artwork: {
    title: "Monkey Sign",
    image: {
      url: "https://d32dm0rphc51dk.cloudfront.net/L52FHh7v_f9mZ3QkC_2YJA/thumbnail.jpg"
    }
    artist: {
      name: "Banksy"
    }
  }
}

describe("ArtworkBrick", () => {
  it("renders synchronously", () => {
    const tree = mount(
      <QueryRenderer<ArtworkBrickTestQuery>
        dataFrom={STORE_THEN_NETWORK}
        environment={stubEnvironmentWithStore(artworkFixture)}
        query={query}
        variables={{ artworkID: "banksy-monkey-sign" }}
        render={props => <ArtworkBrick artwork={props.artwork}}
      />
    )

    expect(tree.find("span").text).toEqual("Monkey Sign - Banksy")
    // etc
  })
})

[zth]

Hi!

I'm also very interested in this, and have worked quite a lot on trying to improve the DX around testing.

I've written a lib called graphql-query-test-mock (https://github.com/zth/graphql-query-test-mock) that's designed to help testing by being as non-invasive as possible. The elevator pitch is that it mocks at the outer most level (the http layer using nock) which removes all needs to alter your code or mock any actual code when writing your tests. Together with some strict handling of variables etc in order to ensure the right thing is actually sent to your backend, I've had quite a lot of success using it to mock entire or parts of views/apps.

It does however come with one major downside:

• It requires you to provide the actual data you'd get back from your backend. This is great for accuracy in the tests, but quite quickly becomes a maintenance burden when the entire query, parts of the query or other data change, which it does rapidly in development.

However, there's work ongoing in providing an automocking feature, using graphql-tools for instance (or anything really that can take your query/variables/schema and return mock data). You can check out the work going on here: https://github.com/zth/graphql-query-test-mock/issues/3 (thanks @jnak for your work!).
Ideally, this means that when we're done you can:

• Provide actual data only for the part of your query you're interested in testing.
• Have the rest of the data be automatically mocked.

You could also set up quite extensive default mocks yourself (we have lots of things on the viewer field that we need resolved in a particular way for example).

Like I said, this is very interesting to me as well, and I'd like to contribute to how this workflow can be improved. Is the solution proposed with graphql-query-test-mock something that'd be interesting to you as well?

[alloy]

@zth Thanks for joining in the conversation!

At Artsy we have been using a somewhat similar setup as the one you’re describing (and all of that is available in our OSS repo). You can take a look at the ticket I link to at the top for an overview of the pros and cons I see with the various solutions.

I’ll summarize by saying that in our experience mocking a schema using graphql-tools has proven to not be a user friendly setup as it requires quite some knowledge of GraphQL itself, knowledge of the graphql-tools specific API for defining schemas, and finally the auto mocking has only made it harder for people to understand what to mock; so we have started removing most of that functionality and are replacing it with just accepting an object literal, something that all devs understand easily.

With regards to full stack testing, I don’t see a real need for testing through the network layer but I do see a downside which is that it makes the codepath async (we'd have to poll the react tree to know if the expected render results are available) and I want our test suite to stay as fast as possible.

[alloy]

@zth I forgot to add that regardless of the difference in how we’d inject data into the system, the emitted types for the operation response would still be usable for your network case as well.

[jstejada]

looping in @alunyov who have been thinking about this too and might want to share some ideas (cc @josephsavona)

[estrada9166]

My solution will be using easygraphql-tester on the environment, it'll mock the query/mutation... also, you can set fixtures to set your data if you don't set a fixture it will mock the requested fields for you.... the only thing is that it doesn't support fragments so the complete query should be specified Also, it supports fragments.

You have to pass the GraphQL schema

How to use it:
Environment.js

import fs from'fs';
import path from'path';
import EasyGraphQLTester from 'easygraphql-tester';

import { Environment, Network, RecordSource, Store } from 'relay-runtime';

import handler from './handler';
import { fixtures } from './fixture'

const schema = fs.readFileSync(path.join(__dirname, 'schema',  'schema.graphql'), 'utf8');
const tester = new EasyGraphQLTester(schema);

let networkQuery = handler

if (process.env.NODE_ENV === 'test') {
  networkQuery = mockQuery
}

function mockQuery(operation, variables) {
  const mockedQuery = tester.mock({
    query: operation.text,
    variables,
    fixture: fixture[operation.name]
  })

  const response = {
    "data": mockedQuery
  }

  return response
}

const network = Network.create(networkQuery);

const source = new RecordSource();
const store = new Store(source);

const env = new Environment({
  network,
  store,
});

export default env;

fixture.js

export const fixtures = {
  UserListQuery: {
    pageInfo: { 
      hasNextPage: false
    },
    edges: [{
      node: {
        name: 'yay'
      }
    }]
  }
}

Mocked result

{
  "users": {
    "pageInfo": {
      "hasNextPage": false,
      "endCursor": "lkasnslm"
    },
    "edges": [
      {
        "node": {
          "id": "51",
          "name": "yay",
          "__typename": "User"
        }
      }
    ]
  }
}

Here is a gist with this info and the complete query!

[zth]

@alloy thanks for your insight, that's very valuable! Yeah, I do see the downsides, especially about the speed of the tests and things requiring too much knowledge of graphql in order to be effective.

I'm curious, you mention that it's the polling of the rendered tree that's slowing things down when going the async route - is that the main problem, or are there more in terms of speed? Is the primary downside that you'll need multiple scans, or is it the actual re-rendering/waiting for the mocked data to return that seemed to be the main problem?

[estrada9166]

I just created a PR on the relay example that maybe can be useful to solve this!

[alunyov]

A quick update on the topic. We've added new APIs to make testing a little bit easier.

The main ideas that we have for testing tools, that should improve the DX, were:

• Simplify maintaining the mock data as much as possible, by generating payloads based on the selection in the queries, mutations, and subscriptions. Relying on other tools (flow, lint) to make sure that the selection in the component is actually correct.
• Better API for controlling the flow of the mock operations: createMockEnvironment that allows resolving operation, queue operation resolvers, etc.

https://facebook.github.io/relay/docs/en/testing-relay-components.html

Please let us know how this is working for you

[mrtnzlml]

Hi @alunyov! The new utilities are really great. I have 2 questions:

1. Is it OK to run Relay Compiler on __tests__ (remove it from the excluded dirs list)? I am asking because docs recommend using graphql tagged template in tests but tests are excluded by default in Relay Compiler.
2. I was playing with generateAndCompile and unwrapContainer combo as well. What suprised me is that generateAndCompile doesn't validate the query and therefore you can write some fake one. Is it intended? Example:

generateAndCompile(
  `
  query ThisDoesntMatterAtAll {
    ...CountryFlag_location # <<< Location type would not be allowed here normally
  }
  fragment CountryFlag_location on Location {
    name
  }
  `,
  buildSchema(fs.readFileSync(path.join('some.schema'), 'utf8')),
).ThisDoesntMatterAtAll;

I can imagine this can be handy when you want to test some very deeply nested component but not sure if it's intentional and what is the motivation behind it.

Thanks in advance.

[alunyov]

@mrtnzlml thank you! Great questions!

1. Yes, it's OK to include the __tests__ to the compiler list (that's were you may use @relay_test_operation).

tests are excluded by default in Relay Compiler.

Thanks for reporting. Fixing that.

1. generateAndCompile(...) - we use internally to test Relay, so we assume that the query is valid, and we skip all the graphql-js validation.
   But, if you will use graphql tag and relay-compiler for tests, then queries should be validated.

[alloy]

@josephsavona @tyao1 Is the @raw_response_type directive what I describe in the first item, which can be used to type-check response test fixtures?

Generate a Flow/TS type that describes the shape of the full response for an operation. This would be helpful in knowing all the fixture data that the dev needs to provide without reading all of the fragments of the containers involved and looking up the types of the fields selected in those fragments and manually putting that all together.

[tyao1]

Yes, it describes the shape of the response. I think it can be used to type the response test fixtures as well, although the original purpose is to flow type the optimisticResponse.

[alloy]

This is great, thanks!