Gatsby: Schema Customization for gatsby-transformer-csv & gatsby-transformer-json

[melpers]

Description

Neither gatsby-transformer-csv nor gatsby-transformer-json honor Schema Customization.

Also, there is inconsistent inference of fields for these 2 transformers.

Steps to reproduce

1. Install the Gatsby starter.
2. Install & configure both gatsby-transformer-csv & gatsby-transformer-json
3. Add one .csv and one .json file under /src/data/csv/ and /src/data/json respectively. Include at least 1 field with a 4-digit number in each file.

// CSV file
name,year,code
Test1,2019,9999
Test2,2017,1600
Test3,2016,7654

// JSON file
[
  {
    "name": "Test1",
    "year": 2019,
    "code": 9999
  },
  {
    "name": "Test2",
    "year": 2017,
    "code": 1600
  },
  {
    "name": "Test3",
    "year": 2016,
    "code": 7654
  }
]

1. npm run develop

At this point there are issues with the 4-digit field in GraphQL being assigned different field types (See below)

1. In gatsby-node.js add a createSchemaCustomization definition to declare the CSV & JSON types created to use type Int! for the 4-digit number field.

// gatsy-node.js
module.exports.createSchemaCustomization = ({ actions }) => {
  const { createTypes } = actions
  const typeDefs = `
    type csvTestCsv implements Node @dontInfer {
      name: String!
      date: Date! @dateformat
      code: Int!
    }
    type jsonTestJson implements Node @dontInfer {
      name: String!
      date: Date! @dateformat
      code: Int!
      }
  `
  createTypes(typeDefs)
}

1. npm run develop again

No change in results.

Expected result

1. I would expect the 2 transformers to infer the same data as the same type.
2. I would not expect a 4-digit number to be inferred as type Date.
3. I would expect both transformers to honor the schema customization.

Actual result

After Step 4 the 4-digit field in GraphQL is being assigned different field types from the two transformers:

• The CSV is assigned the type Date.
  
  
  
  • Note that if a 5-digit number is used, it is assigned the type String.
  
  
• The JSON transformer assigns the type Number.

Build message:

warn There are conflicting field types in your data.

If you have explicitly defined a type for those fields, you can safely ignore this warning message.
Otherwise, Gatsby will omit those fields from the GraphQL schema.

If you know all field types in advance, the best strategy is to explicitly define them with the `createTypes` action, and skip inference with the `@dontInfer` directive.
See https://www.gatsbyjs.org/docs/actions/#createTypes
SitePage.context.code:
 - type: date
   value: '9999'
 - type: number
   value: 9999

error  Variable "$code" of type "Int!" used in position expecting type "Date"  graphql/template-strings

Environment

System:
OS: macOS 10.14.6
CPU: (4) x64 Intel(R) Core(TM) i5-4670 CPU @ 3.40GHz
Shell: 3.2.57 - /bin/bash
Binaries:
Node: 10.15.3 - /usr/local/bin/node
npm: 6.4.1 - /usr/local/bin/npm
Languages:
Python: 2.7.16 - /usr/local/bin/python
Browsers:
Chrome: 80.0.3987.163
Firefox: 75.0
Safari: 13.1
npmPackages:
gatsby: ^2.20.12 => 2.20.12
gatsby-image: ^2.3.1 => 2.3.1
gatsby-plugin-manifest: ^2.3.3 => 2.3.3
gatsby-plugin-offline: ^3.1.2 => 3.1.2
gatsby-plugin-react-helmet: ^3.2.2 => 3.2.2
gatsby-plugin-sharp: ^2.5.3 => 2.5.3
gatsby-source-filesystem: ^2.2.2 => 2.2.2
gatsby-transformer-csv: ^2.2.1 => 2.2.1
gatsby-transformer-json: ^2.3.1 => 2.3.1
gatsby-transformer-sharp: ^2.4.3 => 2.4.3
npmGlobalPackages:
gatsby-cli: 2.5.12

Test Repo: https://github.com/melpers/gatsby-csv-test

[d4rekanguok]

Hi @melpers, in createSchemaCustomization hook, type name has to be in title case. Also your data doesn't have date, but year:

// gatsy-node.js
module.exports.createSchemaCustomization = ({ actions }) => {
  const { createTypes } = actions
  const typeDefs = `
-   type csvTestCsv implements Node @dontInfer {
+   type CsvTestCsv implements Node @dontInfer {
      name: String!
-     date: Date! @dateformat
+     year: Date! @dateformat
      code: Int!
    }

-   type jsonTestJson implements Node @dontInfer {
+   type JsonTestJson implements Node @dontInfer {
      name: String!
-     date: Date! @dateformat
+     year: Date! @dateformat
      code: Int!
      }
  `
  createTypes(typeDefs)
}

It works properly for me after fixing these. Perhaps docs need to make this more clear?

[d4rekanguok]

Also, there is inconsistent inference of fields for these 2 transformers.

I also noticed this, but I think it's due to the nature of CSV being text-based, while JSON can have numbers. I agree that the CSV should be able to recognize those fields as number.

[melpers]

@d4rekanguok Thank you, the title case was indeed what I missed. I had used the graphql name.

The one thing that still strikes me as unexpected is the csv transformer setting any 4-digit number to Date (unless some other pattern is seen). But since the customization addresses that I'll consider this closed.

[d4rekanguok]

Np! In case you missed it, a field's type name is listed in the right sidebar in the graphql explorer:

[melpers]

@d4rekanguok Excellent, thanks again for that tidbit. That would be good to include in the documentation as well I think.