Gatsby: Schema customization resolved data are not present in page component

[Simply007]

Description

I have a site using schema customization for extending the types:

• https://github.com/Simply007/gatsby-starter-kontent-lumen/tree/vNext

My use case is that I want a backreference from kontent_item_category to kontent_item_article to be able to show how many articles are in the category.

Steps to reproduce

If you check Gatsby preview on site:
https://gatsby-starter-kontent-lumen-8311752784.gtsb.io/categories

The result returns (N/A means the data in backreference are not present):

design-inspiration (N/A)
gatsby (N/A)
typography (N/A)

But if you try to run the query on the same environment vie GraphiQL

query MyQuery {
  allKontentItemCategory(filter: {preferred_language: {eq: "en-US"}}) {
    nodes {
      elements {
        title {
          value
        }
      }
      used_by_articles {
        system {
          codename
        }
      }
    }
  }
}

Implementation

I am using createSchemaCustomization hook in gatsby-node.js https://github.com/Simply007/gatsby-starter-kontent-lumen/blob/vNext/gatsby-node.js#L10 to extend the kontent_item_category type.

The implementation of schema customization is here:

https://github.com/Simply007/gatsby-starter-kontent-lumen/blob/vNext/src/usedByContentItemsField.js#L27..L66

Expected result

I would expect to have the same result in the page data as in GraphiQL.

Actual result

used_by_articles is null in the categories https://github.com/Simply007/gatsby-starter-kontent-lumen/blob/vNext/src/pages/categories.jsx#L35 page.

Environment

The same problem is on my local machine:
gatsby info --clipboard

System:
OS: Windows 10 10.0.18363
CPU: (8) x64 Intel(R) Core(TM) i5-8350U CPU @ 1.70GHz
Binaries:
Node: 12.13.0 - C:Program Filesnodejsnode.EXE
Yarn: 1.21.1 - C:Program Files (x86)Yarnbinyarn.CMD
npm: 6.12.0 - C:Program Filesnodejsnpm.CMD
Languages:
Python: 2.7.17
Browsers:
Edge: 44.18362.449.0
npmPackages:
gatsby: ^2.20.36 => 2.20.36
gatsby-cli: ^2.11.22 => 2.11.22
gatsby-plugin-catch-links: ^2.0.4 => 2.2.1
gatsby-plugin-google-analytics: ^2.0.6 => 2.2.2
gatsby-plugin-google-fonts: latest => 1.0.1
gatsby-plugin-react-helmet: ^3.0.0 => 3.2.1
gatsby-plugin-sass: ^2.0.1 => 2.2.1
gatsby-plugin-sharp: ^2.5.3 => 2.5.3
gatsby-plugin-sitemap: ^2.0.1 => 2.3.1

[vladar]

So I tried debugging this and it has something to do with preferred_language filtering. This is probably because it doesn't have its own resolver but uses resolver from the extension.

The query works if you remove this filter: https://github.com/Simply007/gatsby-starter-kontent-lumen/blob/99af7d223c0ca32ca7cf14830d9aab5cb043b00c/src/usedByContentItemsField.js#L41-L44

I will dig deeper into why this is happening a bit later. But what you need to know is that fields with custom resolvers have a special treatment in nodeNodel. And somehow the resolver from the extension is not considered a custom resolver and so the field is not pre-processed.

One option how you can try to fix this is by adding resolver to the preferred_language field explicitly just to confirm this is the issue (it might be more complicated than that).

In general, the rule of thumb is to try to avoid runQuery on fields with custom resolvers (but yeah, it should still work, so it does look like a bug).

[vladar]

I did more digging and it seems things might be even more complicated than what I described above. But my general impression is that you might be using nodeModel in a way that was not anticipated. I'll try to explain what I mean. So we have two layers in the Gatsby data store:

1. Nodes Layer. Raw JS objects passed through createNode action and stored in memory.
2. GraphQL layer on top of it which mirrors the structure of those nodes. It can also add "virtual" fields, transform values based on arguments using schema customization, etc

This is similar to how they use GraphQL with a database for a regular web API. You have a GraphQL API but it uses SQL, MongoDB, or ElasticSearch queries under the hood to fetch data from the underlying store.

So the truth is the nodeModel is like a database. You can use it from your resolvers to query raw data from layer 1. We added a little bit of magic on top of it to be able to query for data coming from custom resolvers. But it was meant to be used for simple use cases (like formatted dates, etc)

But if we look at this query we see that you are trying to filter nodes by "virtual" fields from custom resolvers. I am talking about this part:

                    value: {
                      elemMatch: {
                        preferred_language: {
                          // depends on language fallback preferences
                          eq: source.preferred_language,
                        },
                        system: {
                          codename: {
                            eq: source.system.codename,
                          },
                        },
                      },
                    },

Raw nodes have value: [String!] structure there. It was overridden by schema customization but the behavior of nodeModel in this case is somewhat a gray area.

I think we can consider this as a bug but I would strongly advise against using nodeModel this way. Most of the time you should treat it as a query engine of the underlying raw node store.

We should have probably restricted the ability to query for fields with custom resolvers via nodeModel (maybe only allow to query for scalars this way).

Sorry for this confusion.

[Simply007]

Hello @vladar,

thank you for the insights and the explanation!

Could you please recommend me what approach to use?

It is possible to use context.nodeModel.getAllNodes() and then filter them in memory and return them, but it seems a bit fishy to load all nodes.

I haven't found any method to load the nodes by specific type (other then runQuery).

Or how should I construct the query?
I have tried something like this in the example, but I wasn't successful to construct the query that could return the data I need.

          const linkedNodes = await context.nodeModel.runQuery({
            query: {
              filter: {
                elements: {
                  [linkedElementCodename]: {
                    value: {
                      in: source.system.codename,
                    },
                  },
                },
              },
            },
            type: parentGraphqlType,
            firstOnly: false,
          })

The only working way I came up is to use runQuery and the filter out data in memory.
https://github.com/Simply007/gatsby-starter-kontent-lumen/pull/2

[vladar]

Thank you for opening this, @Simply007

It should be fixed in [email protected] via https://github.com/gatsbyjs/gatsby/pull/26644

We're marking this issue as answered and closing it for now but please feel free to reopen this and comment if you still experience this issue or would like to continue this discussion.

[Simply007]

Hello @vladar,

Unfortunately when trying to test out the solution, I found probably related one.

I have tried to update gatsby from 2.20.36 to 2.24.51 and ran gatsby develop and internal call failed here: https://github.com/gatsbyjs/gatsby/blob/master/packages/gatsby/src/schema/schema.js#L403 failed, because node was null.

See the pull request draft with error in CI: https://github.com/Kentico/gatsby-starter-kontent-lumen/pull/13

---

When I debugged the issue, I found out, that innerValue is [null, null, null] when calling resolveRecursive https://github.com/gatsbyjs/gatsby/blob/66111f490b9d6677642abd98b564c2bb2ea16ddc/packages/gatsby/src/schema/node-model.js#L659

These three nulls come from @kentico/gatsby-source-kontent plugin - there is a field extension that is taking care of linking modular content elements:
From the value of the element, which is an array of codenames, plus preferred language value from the ancestor node, it queries the data for the linked nodes:

https://github.com/Kentico/kontent-gatsby-packages/blob/master/packages/gatsby-source-kontent/src/createSchemaCustomization.items.ts#L42

... // see the link above 👆
const promises = nodesCodeNames.map(codename => context.nodeModel.runQuery({
    query: {
        filter: {
            system: {
                codename: {
                    eq: codename,
                },
            },
            [naming_1.PREFERRED_LANGUAGE_IDENTIFIER]: {
                eq: nodesLanguage,
            },
        },
    },
    type: naming_1.getKontentItemInterfaceName(),
    firstOnly: true,
}));
const nodes = await Promise.all(promises);  // nodes are in this case [null, null, null] 

I have tried to test out if the data are present -> and they were in the model

context.nodeModel.getAllNodes({ type: getKontentItemInterfaceName() }).filter(i => i.system && nodesCodeNames.includes(i.system.codename) && i.preferred_language === nodesLanguage);

Before update gatsby from 2.20.36 to 2.24.51 there was no problem and the constructed queries returned the data as expected.

[Simply007]

I can't re-open this issue, but I can create a new one if necessary.

• It is probably because I am not in Gatsby Github org anymore

[vladar]

Ok then it means there are two problems:

1. Gatsby fails with nullish values in an array of interface type (should be fixed via #26677)
2. Somehow query running doesn't return what you expect.

I couldn't figure out the source of the 2nd problem yet because so many queries are running that it's hard to debug it. Maybe you can extract a problem into a minimum reproduction (with a minimum possible dataset and custom types)? That would help a lot to debug this further.

[vladar]

I've found that the 2nd problem was caused by https://github.com/gatsbyjs/gatsby/pull/21010

Should be fixed via #26681

[vladar]

Published in [email protected]

[Simply007]

Applied fix and resolution works as expected:

• https://deploy-preview-13--gatsby-starter-kontent-lumen.netlify.app/articles/upgraded-to-gatsby-v2 -> the tags from the bottom are loaded from data extended using schema extensions in source plugin
• backlink using schema extension right in the app (via local plugin) - https://deploy-preview-13--gatsby-starter-kontent-lumen.netlify.app/tags/

Thanks, @vladar!