Gatsby: Allow non-existing fields in GraphQL Queries

[Unforgiven-wanda]

Hello,
I'm using Gatsby with the Wordpress source plugin. So far so good. But the thing is I am querying fields that might or might not be there, in this case, the featured image of a post.
Here is my query:

{
allWordpressPost(sort: { fields: [date] }) {
      edges {
        node {
          title
          excerpt
          slug
          featured_media
          better_featured_image {
            wordpress_id
            alt_text
            caption
            description
            media_type
            post
            source_url
          }
        }
      }
    }
}

It works well when the featured image is set, but fails miserably otherwise.
And so my question: Is there any way in GraphQL to query an optional field? To add a default value to a required field?

Thank you.

[m4rrc0]

I have/had the same problem with contentful. Here is my PR proposal #2037. It still needs to be polished.
Unfortunately it does not handle images atm because with this approach we would need a default image to fill in the field and it is not the expected behavior to have an image there if the user didn't put one. I don't know how we could set a 'falsy-like' value on an image field...
I am wondering how the graphiql interface handles this kind of stuff. If we query an unset field there it just returns null on that field.
Anyway, if someone has an idea to move this forward I am interested too.

[m4rrc0]

And the issue associated for reference: #1517

[Unforgiven-wanda]

@MarcCoet I just noticed that this particular issue makes it also impossible to publish an empty blog. Meaning if you don't have any post in your Wordpress site for example, you run into Cannot read property 'allWordpressPost' of undefined

We really need some way to throw a warning, instead of a full-blown error in these cases

[m4rrc0]

Totally agree but I don't know how.
I feel this is something that should be solved in gatsby core because every CMS-like source plugin is going to fall in the same traps.
Sorry @KyleAMathews but I need to mention you again here. ^^'

[sebastienfi]

We solved this by publishing a dummy post with all features set.

[Unforgiven-wanda]

This issue remains my biggest gripe with Gatsby. Simply returning nulls to every non-existant queried field in a GraphQL query would be the best approach in my opinion, as it is trivial to check whether the field exists in React
Just a use case that I encountered this week: One of my clients' first reflex was to delete all his posts, expecting to see a no posts page, until such a time he would start filling his blog with content, and that simply broke the build process. I had all the trouble in the world to explain to him why it worked that way.

Another, more recent one had to do with the menu, where I fetch wordpressWpApiMenusMenusItems to display the menu he creates in the Wordpress dashboard. Again, ideally not having a menu to fetch I would not display one, instead the build process breaks, same thing when trying to fetch all menus children, something like:

wordpressWpApiMenusMenusItems(name: { eq: "Main Menu" }) {
      items {
        wordpress_id
        title
        url
        type
        wordpress_children {
          wordpress_id
          title
          url
          attr
          target
          type
          wordpress_children {
            wordpress_id
            title
            url
            attr
            target
            type
          }
        }
      }
    }

If the menu does not have children, we run into the same issue.

So, I'm not sure if this is in line with Gatsby's philosophy, so maybe I'm assuming too much, but it would be much appreciated if unsuccessful GraphQL queries did not break the build, simply returning empty or null values for the fields required, then it would be up to the dev to check whether the values exist before displaying them.
What do you think of this?

[KyleAMathews]

@Unforgiven-wanda these are great examples of when this causes trouble! I have been somewhat confused why people ran into this trouble as I always build out my sites while adding content so don't ever have the problem of wanting to write a query against something that doesn't yet exist.

Thinking about this some more — I think it could be possible to turn what's now a build error into a warning. @jquense any thoughts on this that come to mind?

@Unforgiven-wanda would you like to investigate a possible solution to this? We use the Relay compiler for validating queries and we'd either need to somehow lower the warning level or catch its errors and not exit when it's a query that includes fields that don't exist.

[jquense]

Unfortunately there isn't really a good way to silence this sort of thing since it's a hard error in GraphQL. Even if you could tho you'd lose the validation for cases were you do want it, e.g. notification you are trying to query some field you spelt wrong or something. I agree that is can be frustrating and annoying in the development, but this is a hard constraint of how graphql works, it can _only_ query stuff that is defined in the type system.

Ultimately the issue here is that if you only infer the schema from the input data, you are necessarily constrained to what the data contains. There isn't any way to tell graphql to just return null or something for fields that don't exist, because it needs to know a a lot of info _waaay_ before that to even try and resolve that field. You also can't work backwards from the query and do inference at that point because queries don't contain enough info to do inference (e.g. you can't distinguish a List from an Object)

In terms of what Gatsby can do, the API's for handling this already exist luckily, you need to extend the graphql types manually and specify which fields you always want to exist. We might be able to make that a bit friendly for quickly specifying additional fields, but honestly, defining GQL schema is verbose and time consuming sort of inherently. (note Apollo Launchpad has a neat way of defining schemas quickly)

[Unforgiven-wanda]

@KyleAMathews Of course. i'll do my utmost to resolve this issue, and if I am successful, I'll be sure to put it a pull request
I'll be closing this issue then, since as demonstrated by @jquense there is no obvious solution for now.
Thank you all.

[sebastienfi]

This also looks promising https://github.com/marmelab/graphql-schema-from-json

[diegolamanno]

@jquense I am trying to follow your recommendation to extend graphql types manually. However, I am running in some issues that I would appreciate your input.
1) I am using setFieldsOnGraphQLNodeType very similar like some of the gatsby-transform plugins do. Am I in the right path or that's not the gatsby API you were referring to?

2) So far I have been able to add more types using ☝️ that API. But I haven't found a way to setup a default value for a UnionTypes without completely overwriting the ones coming from the API,

I hope this makes any sense, thanks

[sedubois]

defining GQL schema is verbose and time consuming sort of inherently

How about the kind of GraphQL IDL syntax like in Graphcool GraphQL-up? https://github.com/graphcool/graphql-up
(NB: I never actually built a GraphQL backend myself and am just planning to use Graphcool, Apollo, Gatsby where needed.)

[hubertron]

Hey Kyle - We are running into this issue as well. It's preventing build of more modular layouts that have varying content but one template.

[m-allanson]

@hubertron there's some great work happening at #3344 to fix these 'missing field' errors.

[Kikobeats]

@Unforgiven-wanda how did you resolve that?

[Unforgiven-wanda]

@Kikobeats Unfortunately no. For my personal projects I keep a hidden page with all fields to circumvent this issue. However I had to move my clients projects to React-static (a decent alternative to Gatsby, although somewhat inferior imho) as I could not explain to them why it worked this way and the build process kept breaking because someone would either delete the hidden page or mess with it.

[Kikobeats]

The only thing necessary is to extend the GraphQL schema for the add Nullable over the value

[Unforgiven-wanda]

@Kikobeats I wasn't aware of that as it's been some time since I've used Gatsby.
Thank you for the info.

[jannispaul]

@Kikobeats Could you please explain how to extend the GraphQL schema and add Nullable over the value or provide a link? I have a similar problem and can't figure it out

[Kikobeats]

Unfortunately I didn't find how to 😢

[LouiseReid]

I'm running into this issue with Gridsome and GraphQL and it seems ridiculous that you can't delete the last entry in the collection without the build failing. Surely this is a massive use case?

[kuworking]

This is indeed problematic:
I'm re-using a core theme to be able to re-use it with many child themes, everything around the idea of writing posts in mdx.

__This is an expected use case I would say__. Then now, with my __FORTH__ child theme I want to expand the mdx frontmatter post files I was using so far

So until now I was using a frontmatter like

date: 2019.05.16
title: 'hey world'
tags: general
snippet: 'hey snippet'
abstract: 'first post'
---

which was processed by the core theme

Now with the __FORTH__ child theme I wanted to add another field, the type

type: 'javascript'
date: 2019.05.16
title: 'hey world'
tags: general
snippet: 'hey snippet'
abstract: 'first post'
---

So in the core theme I've updated the query to

          frontmatter {
            type
            date
            title
            snippet
            abstract
          }

And of course, this causes an error with my __FIRST__ theme, since mdx files there don't have the new field.

The only workaround now is to add an empty type field in all the posts of __all__ the previous mdx posts, something I can only do if I own these sites.

Is this like this? Isn't there anything I can do apart from what I've said?

[giacomoalonzi]

Any solution found? I'm experiencing the same issue with ACF if any repeater is empty.

[devpascoe]

I'm using Contentful for site data. Similar issue, an optional short text field, if not set to required and no text entered then querying for the now missing field breaks.
Anyone reading this does Contentful have a default value setting i'm not aware of??
Regardless it would be amazing if:

• value could return null (but understand the by design issues mentioned above)
• we had an option to inherit and override the Query fields and therefore specify a default value
• manually define all the Query fields as above
  There was some mention about overriding with setFieldsOnGraphQLNodeType - can some kind person point me in the right direction where i might be able to manually define the nodes (bonus points for something specific to Contentful)?

ta ❤️

[tweetzal]

@KyleAMathews Just thought I'd let everyone know aswell that we are experiencing the same issue here in London, UK - using gatsby-source-wordpress

I have 700 pages populated from a content migration.

I added 4 new custom fields to pages in wordpress after 300 pages where populated.

I now have 300 pages without the 4 new fields and 400 pages with the 4 new fields, and I need this data to run logic for the primary navigation/ menu.

Any suggestions or work arounds we can use.

?

[Ozerich]

@tweetzal, next code can help you

if (!function_exists('acf_nullify_empty')) {
    function acf_nullify_empty($value, $post_id, $field) {
        if (empty($value)) {
            return null;
        }
        return $value;
    }
}

add_filter('acf/format_value', 'acf_nullify_empty', 100, 3);

[giacomoalonzi]

I solved with a resolver on gatsby
gatsby-node.js

exports.createResolvers = ({ createResolvers, schema }) => {
  const resolvers = {
    wordpress__PAGEAcfHero_text: {
      list_of_words: {
        resolve(source, args, context, info) {
          if (!source.list_of_words) {
            return info.originalResolver(
              {
                ...source,
                list_of_words: []
              },
              args,
              context,
              info
            )
          } else {
            return info.originalResolver(source, args, context, info)
          }
        },
      },
    },
    wordpress__PAGEAcfMember: {
      socials: {
        resolve(source, args, context, info) {
          if (!source.socials) {
            return info.originalResolver(
              {
                ...source,
                socials: []
              },
              args,
              context,
              info
            )
          } else {
            return info.originalResolver(source, args, context, info)
          }
        },
      },
    }
  }
  createResolvers(resolvers)
}

[tweetzal]

Thanks guys. Much appreciated

On Tue, Jul 16, 2019 at 4:11 PM Giacomo Alonzi [e-mail]
wrote:

I solved with a resolver on gatsby
gatsby-node.js

exports.createResolvers = ({ createResolvers, schema }) => {
const resolvers = {
wordpress__PAGEAcfHero_text: {
list_of_words: {
resolve(source, args, context, info) {
if (!source.list_of_words) {
return info.originalResolver(
{
...source,
list_of_words: []
},
args,
context,
info
)
} else {
return info.originalResolver(source, args, context, info)
}
},
},
},
wordpress__PAGEAcfMember: {
socials: {
resolve(source, args, context, info) {
if (!source.socials) {
return info.originalResolver(
{
...source,
socials: []
},
args,
context,
info
)
} else {
return info.originalResolver(source, args, context, info)
}
},
},
}
}
createResolvers(resolvers)
}

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/gatsbyjs/gatsby/issues/2392?email_source=notifications&email_token=AAS5DVTSGDLWT4GA6K6VZFDP7XQKBA5CNFSM4D6G56NKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2BFQLI#issuecomment-511858733,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAS5DVXDMLLA75XOGONHRTDP7XQKBANCNFSM4D6G56NA
.

[robmarshall]

I solved with a resolver on gatsby
gatsby-node.js

exports.createResolvers = ({ createResolvers, schema }) => {
  const resolvers = {
    wordpress__PAGEAcfHero_text: {
      list_of_words: {
        resolve(source, args, context, info) {
          if (!source.list_of_words) {
            return info.originalResolver(
              {
                ...source,
                list_of_words: []
              },
              args,
              context,
              info
            )
          } else {
            return info.originalResolver(source, args, context, info)
          }
        },
      },
    },
    wordpress__PAGEAcfMember: {
      socials: {
        resolve(source, args, context, info) {
          if (!source.socials) {
            return info.originalResolver(
              {
                ...source,
                socials: []
              },
              args,
              context,
              info
            )
          } else {
            return info.originalResolver(source, args, context, info)
          }
        },
      },
    }
  }
  createResolvers(resolvers)
}

Would this mean creating a resolver for each ACF field? I am working on something that uses a hell of a lot of potentially empty fields.

[Glinkis]

I just ran into this very issue with the Contentful source plugin. I have a lot of optional fields, and potentially empty content types.

[kimbaudi]

I'm also having this issue when migrating my Jekyll site to GatsbyJS. Wish GraphQL wasn't so strict. In fact, it would be great if there was a config option that would allow GraphQL to just simply warn instead of throwing a build time error.

[hidanielle]

Running into this issue while using Gatsby and Kentico Cloud. Can't seem to get it working with the createResolver solution. Has anyone else here had any luck since?

[robmarshall]

I found writing out an entire schema (with correct values - 0 for int, false for bool, ect), and then merging entities over that worked. Bit of a long way round.

[ttstauss]

I just ran into this very issue with the Contentful source plugin. I have a lot of optional fields, and potentially empty content types.

I'm in the same boat.

[briangonzalezmia]

I was able to use the following from the Gatsby Site to take care of the nested nullable:

exports.createSchemaCustomization = ({ actions }) => {
  const { createTypes } = actions
  const typeDefs = `
    type MarkdownRemark implements Node {
      frontmatter: Frontmatter
    }
    type Frontmatter {
      tags: [String!]!
    }
  `
  createTypes(typeDefs)
}

* Don't forget to restart the dev server to see the results*

Ref: https://www.gatsbyjs.org/docs/schema-customization/#nested-types

[anonimoconiglio]

I was able to use the following from the Gatsby Site to take care of the nested nullable:

exports.createSchemaCustomization = ({ actions }) => {
  const { createTypes } = actions
  const typeDefs = `
    type MarkdownRemark implements Node {
      frontmatter: Frontmatter
    }
    type Frontmatter {
      tags: [String!]!
    }
  `
  createTypes(typeDefs)
}

Is this snippet of code complementary to the rest of gatsby-node.js?
You added this before or after the exports.createPages module?

[bowenac]

Was having the same issue pulling in WordPress posts that have optional fields. I expected they would just return null if they didn't exist not blow up the whole query... that really puts a damper on things.

Seems pretty common to have optional fields. Will keep looking around for a solution, but having to create a schema for every possible optional field seems more like a band aid fix.

@Ozerich thanks for the solution, that worked.

[AntoineFSG]

With Wordpress and ACF, GraphQL queries the fields and returns null if at least one of the posts has a value for this field.
Let's say you have an optional acf field beautiful_image, if at least one of your posts actually has a beautiful_image then GraphQL will return null for all the posts that don't but if an optional field is never used through the site you can't query it without triggering an error.

[hades200082]

Same issue here with Contentful.

Is it possible to extend the graphQL interface to allow specifying of a default value on a per-field basis such that if the field doesn't exist it will fill with the default specified in the query instead of failing the build?

[Ricky-NSW]

@benrobertsonio You provided some email support for me on April 2nd regarding the same issue (I think).
I have the Contentful plugin pulling in content. One content item in Contentful was missing data which was breaking the build with a "Compile Error" and you said a "Safety Check" would fix it - by adding a "?" in the data request, like this (notice the ?):

Doesnt work: biography.mediaApprovedQuote.childMarkdownRemark.html
DOES work: biography.mediaApprovedQuote?.childMarkdownRemark.html

When a Content Item is missing that data it is left blank.

Could this be applied elsewhere?
I haven't been able to find any more information on this technique. Ben do you know where we can learn more about this as a possible fix?

Thanks (AGAIN).

[collegewap]

I had the same problem when using staticman comments and had an optional field "replyFor"

I have added the following section in gatsy-node.js and it fixed the issue

exports.createSchemaCustomization = ({ actions }) => {
  const { createTypes } = actions
  const typeDefs = `
    type CommentsYaml implements Node {
      replyFor: String
    }
  `
  createTypes(typeDefs)
}

Explicitly defining data types

[bughishor]

I had the same problem when using staticman comments and had an optional field "replyFor"

I have added the following section in gatsy-node.js and it fixed the issue

exports.createSchemaCustomization = ({ actions }) => {
  const { createTypes } = actions
  const typeDefs = `
    type CommentsYaml implements Node {
      replyFor: String
    }
  `
  createTypes(typeDefs)
}

Explicitly defining data types

I can confirm that the same approach works with Strapi (e.g type StrapiElement).