gatsby-source-contentful: Maximum call stack size exceeded

[marcneander]

Not sure if this is a bug or not. And unfortunately I don't really have much time this week to investigate. But I thought I might as well put this up here.

I'm using the gatsby-source-contentful plugin. In my contentful content model I'm having two content types, both with the new Richtext editor. Problems occurs when I have entry links to each other.

To be more clear. Say I have two pages, home and about. If I link from home -> about and from about -> home inside the Richtext editor I get this error.

Like I said before. Not sure if this is a bug.

Here's a callstack https://pastebin.com/fD676e6j

Environment (if relevant)

  System:
    OS: macOS 10.14.3
    CPU: (4) x64 Intel(R) Core(TM) i5-8210Y CPU @ 1.60GHz
    Shell: 3.0.0 - /usr/local/bin/fish
  Binaries:
    Node: 10.15.0 - /var/folders/vh/cx86q1vs2xz5rxlf8b8cny2r0000gn/T/yarn--1548706327929-0.8647198662198463/node
    Yarn: 1.13.0 - /var/folders/vh/cx86q1vs2xz5rxlf8b8cny2r0000gn/T/yarn--1548706327929-0.8647198662198463/yarn
    npm: 6.4.1 - ~/.nvm/versions/node/v10.15.0/bin/npm
  Languages:
    Python: 2.7.10 - /usr/bin/python
  Browsers:
    Chrome: 71.0.3578.98
    Safari: 12.0.3
  npmPackages:
    gatsby: 2.0.98 => 2.0.98 
    gatsby-plugin-layout: 1.0.11 => 1.0.11 
    gatsby-plugin-offline: 2.0.21 => 2.0.21 
    gatsby-plugin-react-helmet: 3.0.5 => 3.0.5 
    gatsby-plugin-styled-components: 3.0.4 => 3.0.4 
    gatsby-source-contentful: 2.0.26 => 2.0.26 
    gatsby-transformer-remark: 2.2.0 => 2.2.0 

gatsby-config.js:

plugins: [
        {
            resolve: 'gatsby-plugin-styled-components',
            options: {
                displayName: false
            }
        },
        'gatsby-plugin-react-helmet',
        {
            resolve: 'gatsby-plugin-layout',
            options: {
                component: require.resolve('./src/components/Layout')
            }
        },
        {
            resolve: 'gatsby-source-contentful',
            options: {
                spaceId: '<spaceID>',
                accessToken: process.env.CONTENTFUL_ACCESS_TOKEN
            }
        },
        {
            resolve: '@contentful/gatsby-transformer-contentful-richtext',
            options: contentfulRenderOptions
        },
        'gatsby-plugin-offline'
    ]

[karenhou]

When using reference fields, be aware that this source plugin will automatically create the reverse reference. You do not need to create references on both content types. For simplicity, it is easier to put the reference field on the child in child/parent relationships.

https://www.gatsbyjs.org/packages/gatsby-source-contentful/?=contentful

Not sure if this would help, but try to disconnect one of your reference.

[marcneander]

@karenhou It's not reference fields that's the problem. It makes sense to have references one way. What I'm talking about is the Rich text field which has Entry hyperlink support. If you want to create a <a href> to another entry you'd do it that way. Problems occurs when two content entries link to each other. Which would be a very valid case.

Say you have a 3 part article that would link to each other in the beginning. That would never work right now if you're not fine hardcoding the links.

[sidharthachatterjee]

My best guess is some recursive handling of references is exploding the call stack because of the lack of an exit case that covers this maybe?

@marcneander Can you try to create a minimal reproduction repo for this (with a contentful instance setup to break it) and link to it? That'll help us debug this faster

[marcneander]

https://discovery.contentful.com/entries/by-content-type?delivery_access_token=809074d7507cf609df4dd9210903910a91c6f35d3ddc7ecbc2cb2cd4d5a260c4&preview=false&preview_access_token=&space_id=7b9jienzv022

https://github.com/marcneander/gatsby-contentful-bug

[simoneb]

I've been looking into this issue because we're also experiencing it. I'm not very familiar with the contentful source plugin codebase but the first impression is that the logic in place to avoid circular references only accounts for field types which are not richText. The latter is a lot more complex to analyze because it can contain a "link" to another entry in deeply nested places. I _think_ that the logic to handle this should live in the buildForeignReferenceMap function in the gatsby-node.js module, but I'm not sure about the feasibility of recursively checking the complex data structure of richText fields.

At the current time, two entries cross-referencing each other via hyperlinks inside a richText field lead to infinite recursion.

This is a perfectly valid use case, of a series of articles referencing each other at the top of the article content for instance, and @marcneander repro is a perfect one.

[dkhuntrods]

@sidharthachatterjee Thanks for your work on this. Would just like to add my upvote to this issue - we've encountered the exact same scenario, and I would second @simoneb's comments about this being a very valid use-case and @marcneander's repro an accurate reduced test-case of our situation. We've got a very urgent situation on our hands at the moment - do you think it likely you'll be looking at this soon? Otherwise we will look at a PR of some sort. Thanks again.

[simoneb]

Hi @marcneander, given the lack of support around here we've worked around the issue by forking the gastby repo, changing the gatsby-source-contentful plugin and publishing our own version on npm.

Because it was really hard to figure out how to handle rich text nodes with links to other contentful entries, we've come up with a workaround which consists in the plugin accepting an array of whitelist field names to include when processing rich text field nodes. All other fields are omitted. For the lack of a better and generic solution, this basically shifts the responsibility of which fields won't cause circular references on the shoulders of developers.

For instance, we use the gatsby-transformer-contentful-richtext plugin to process rich text fields and transform hyperlinks to entries into plain <a> tags, in a similar way as shown in the docs:

[INLINES.ASSET_HYPERLINK]: node => {
  return `<img src="${
    node.data.target.fields.file['en-US'].url
  }"/>`
}

So, I configure our custom gatsby-source-contentful plugin with the option richTextFieldEntryHyperlinkWhitelist: ['file'], because it's the only field we use and we know it won't cause circular references.

You can find our fork here. The plugin is published as @nearform/gatsby-source-contentful.

Hope it helps!

[snide]

@simoneb TY for your fork. This fixed the issue for me as well. It's pretty easy to get into this kind of reference, so although it's not an optimal fix I really appreciate you publishing your fork.

[simoneb]

@snide you're welcome!

[toby-bushell]

@sidharthachatterjee Thanks for your work on this. Would just like to add my upvote to this issue - we've encountered the exact same scenario, and I would second @simoneb's comments about this being a very valid use-case and @marcneander's repro an accurate reduced test-case of our situation. We've got a very urgent situation on our hands at the moment - do you think it likely you'll be looking at this soon? Otherwise we will look at a PR of some sort. Thanks again.

I'd also like to upvote this, this is currently making internal links impossible to use in the RichText. If the reference contains any link back to the content at any level this causes the stack size explosion, which is impossible to manage or restrict.

[simoneb]

@toby-bushell as unappealing a solution as it may sound, have you tried the fork I mentioned here? That's the solution we are applying for now.

[toby-bushell]

Hi @simoneb, am I right in thinking that the workaround is to skip trying to resolve internal links? And explicitly list the embed types to resolve in Rich Text?

[simoneb]

@toby-bushell not entirely. If you list only the fields that you need when processing your rich text, they will be resolved. The rest you will not need can safely not include in the whitelist so it doesn't cause issues.

[toby-bushell]

@simoneb thanks for this and the fork, It looks like a good failsafe option, that we may well use.

We do however want to be able to link to other references safely within RichText and are keen on a permanent solution, as this is a very big feature of contentful. Even an understanding of the roadmap for this so we can make a call on how much time to dedicate to alternatives.

[gatsbot[bot]]

Hiya!

This issue has gone quiet. Spooky quiet. 👻

We get a lot of issues, so we currently close issues after 30 days of inactivity. It’s been at least 20 days since the last update here.

If we missed this issue or if you want to keep it open, please reply here. You can also add the label "not stale" to keep this issue open!

Thanks for being a part of the Gatsby community! 💪💜

[marcneander]

Spooky quiet indeed. But not ready to be closed

[olapersson]

Having the same issue, also found a mention of the same problem here https://github.com/contentful/rich-text/issues/53 (issue was closed but it seems to me that the problem is still present).

Cross referencing entries is a quite natural thing (inline link from one page to another for example, much better to be able to use a reference rather than a hardcoded string)

As a temporary workaround I used the forked version and queried body { body } on the node and then parsed the json response (richTextRenderer being my wrapper function that maps embedded entries, images etc.)

      {richTextRenderer({json: JSON.parse(data.body.body) })}

Downside is the resulting data has a different structure so once it's fixed I'll have to refactor all field access.

[Khaledgarbaya]

Hey @simoneb,
Any reason you didn't make a pull request since your fork fixed the issue?

[simoneb]

@Khaledgarbaya because the fix is not generic enough in my opinion, it's more of a workaround.

[Khaledgarbaya]

@simoneb it can be a good starting point for a fix and we can work together on it
there

[jessepinho]

@Khaledgarbaya I think a better solution would be to fix the underlying problem — that gatsby-source-contentful doesn't realize that it's already loaded a node into its internal node graph (not sure I'm using the correct terminology here), and keeps loading it in an infinite loop due to the cross-references. Probably a better solution would be for it to keep track of what it's already loaded, and point any references to the already-loaded node. Does that make sense? Kinda hard to explain by text.

[Khaledgarbaya]

This is mainly happening because there is a circular reference and somewhere in the SDK or the plugin it's not clear when to stop when resolving references.
@marcneander can you please share the json of your content types to use it as a test?

you can do that by calling this on the contentful-cli

Steps

1.

npm install -g contentful-cli

2.

contentful login

and follow the instruction

3.

contentful space generate migration -s <space_id> -e master

you will get a js file that defines the content types structure that you can share it with me.

Cheers,
Khaled

[marcneander]

Sorry for being slow... Here is it 7b9jienzv022-master-1557232700604.js.zip

[felixakiragreen]

Just wanted to share that my team is also struggling with resolving this issue. Will post more details soon.

[jessepinho]

FYI, I'm working on a fix to this based on @simoneb's great work. I want to PR it, but I want to tweak the options a bit first so that you also have an option of excluding fields (rather than specifying all the fields to include).

[jessepinho]

Oh sorry, the branch is here: https://github.com/gatsbyjs/gatsby/compare/master...jessepinho:topics/rich-text-field-filter

[thomaskrantz]

Just wanted to chime in that we're also experiencing the same issue, and I have tried using @jessepinho's PR but for some reason that didn't have any effect. Not sure how I can help though.

[jessepinho]

@thomaskrantz did you also pass a richText.includeEntryFields option? (I noticed btw that the documentation I included had a typo of includeEntryList when it should have been includeEntryFields)

[cameron-martin]

@Khaledgarbaya does your assignment of this mean that you're in the process of fixing it? I was going to take a look at this, but if you're looking at it then I'll hold off.

[Khaledgarbaya]

@cameron-martin please feel free to take a look at it

[dwikberg]

We are also experiencing this issue. I will Fedex a bottle of Dom Perignon champagne to whoever solves this!!

[simoneb]

@dwikberg If our workaround works for you as it did for us you can post the bottle to my address ;)

[dwikberg]

@dwikberg If our workaround works for you as it did for us you can post the bottle to my address ;)

We need a more permanent solution :(

[simoneb]

Considering that this has been open for 6 months now, I'd rather stick with the workaround, we've been running that in production since then and based on my experience it covers most common scenarios. I'm ok with a bottle of Pinot Noir as well as this is not a permanent solution.

[dwikberg]

Considering that this has been open for 6 months now, I'd rather stick with the workaround, we've been running that in production since then and based on my experience it covers most common scenarios. I'm ok with a bottle of Pinot Noir as well as this is not a permanent solution.

I think we are using a newer version with gatsby where the workaround does not cover some of the scenarios. Our build keeps on crashing :) But A for effort :)

[jayhostan]

Just run into the same thing. Is @simoneb 's workaround still the best temp. option?

[jessepinho]

@jayhostan I think my PR (#14612) provides a decent temp solution.

However, a longer-term solution (which I discussed offline with @Khaledgarbaya) would be to properly match Contentful’s real GraphQL API. In their docs, they describe a links property that lists all linked entries. Then you could simply control which fields of which content types you want to pull. I believe this would resolve the infinite-recursion issue, but I haven’t explored it enough yet to know for sure.

[kompounder]

Hi - is there a solution to this issue? It seems to severely limit Contentful's functionality with Gatsby. Any update would be much appreciated!

[pvdz]

We've landed a fix for the stack overflow caused by circular dependencies. That fix should resolve this issue. Please check it with the next version bump (or from master). If that does not fix the problem, please reopen with a new (minimal) repro.

Thanks everyone for helping out and for your patience.

[kompounder]

Thank you very much for taking a look at this.
Unfortunately Gatsby 2.18.5 continues to give me the same error on Contentful rich texts ("maximum call stack size exceeded") whenever a richtext field has an embedded reference to a content type that includes a richtext field.
Will try to submit a minimal repro.

[pvdz]

@kompounder oh :( If you can create a repro please let me know. Note that I suspect it's not actually related to contentful (though it might be). A repro that doesn't use contentful would be even better and is probably easier to create because you can do it locally.

[richhiggins]

this is still broken for me using the latest 2.18.6. same error message, looks to be caused by using a link type of entry within a Rich Text editor.

[richhiggins]

interestingly I have another project where I've used the rich text editor in a referenced content type (I called it paragraph) rather than a field within my page content type. my page entries include multiple paragraph referenced entries like so:

in this scenario linking to a page entry does not cause the stack bug, I'm presuming because the source entry type is paragraph rather than page.

Would that count as a (content model based) permanent fix @dwikberg 🍾?

[jessepinho]

@richhiggins it will happen if your linked entry links back to the current entry, thus creating an infinite loop

[pvdz]

It would be great if you could create a minimal repro with tihs, or at least a repo that I can just clone and build to repro the case if nothing else.

I'll try to revisit this on Monday. Thanks for the input!

[pvdz]

I'm removing the "not stale" label until somebody can come up with a repro. If nothing else this issue will be closed by the bot in about 30 days.

If somebody has a repro I can check out and confirm, please post it here. Thanks!

[cameron-martin]

There is a repro in the form of an e2e test in this PR: https://github.com/gatsbyjs/gatsby/pull/15358

It's not isolated from contentful, but it should demonstrate the issue still.

[pvdz]

Ok. Can repro. On it.

[pvdz]

@cameron-martin Thank you. I was able to add a repro to the tests that caught this stack overflow and able to prevent the stack overflow. Please report back after testing with #20039 .

[pvdz]

@cameron-martin This repro was fixed in #20039 which has been published. Please check whether this works for you.

I'm going to close this for now. If anyone has related problems feel free to reopen. Or a new issue and tag me.

Thanks!

[jessepinho]

🚨 I've created a minimal reproduction of this issue! 🚨

This appears to _still_ be a problem on the latest version of Gatsby, even after #20039 was merged.

Here is the repo: https://github.com/jessepinho/gatsby-contentful-infinite-loop-repro

Here's the Contentful Discovery app with all the data (note that a bug in the app prevents it from showing article records): https://discovery.contentful.com/entries/by-content-type?delivery_access_token=04QAYaRsr548atM4KwInDCWk8v84agx0uS7r8LrQutk&preview=false&preview_access_token=&space_id=j66qheg22uli

And a demonstration GIF:

@pvdz could you please reopen this issue? (I don't have rights to.)

[pvdz]

Great, thank you!

[pvdz]

Confirmed. Looking into a fix now.

[pvdz]

The fix (which at least resolves your repro) is in #20674

Follow that to see when it gets merged and to see what version it gets released in.

[jessepinho]

@pvdz WOW that was fast, thank you! When I get a chance, I'll try it with my real-life repo and see if it fixes the issue.

[pvdz]

With #20674 being merged this repro has been fixed. It should be published on the next patch bump (should be posted on PR).

Please feel free to reopen (with repro..) if you're still having issues after that.

[ma-pe]

I can confirm that this fix (released with v 2.7.8) solves my problem 👍

Thanks a lot!

edit: I meant version number 2.1.78.

[jessepinho]

I just upgraded and am still experiencing the problem in my own repo. @ma-pe what library do you have v2.7.8 of? There is no gatsby-source-contentful with that version...?

[pvdz]

@jessepinho are you saying the repro you posted is still having this problem with the above fix in place? I couldn't repro it anymore when it landed. But please let me know if you can repro it when all deps are up to date (it may help to just delete the node_modules entirely and make sure the package.json is up to date).

[vpreponen]

I assumed he meant 2.1.78. I also continued to experience the problem with v2.1.78 so I continued to use a work-around with legacy versions.

[pvdz]

@vpreponen The current published version is 2.1.85, can you check if you're still experiencing it with that version? (Is there anything blocking you from upgrading the patch bump?)

[jessepinho]

@pvdz not the repro I posted, but a separate repo that I was trying to reproduce with that minimal repro I posted. Apparently it was a little too minimal 😆 Will look into this further. And yeah I used v2.1.85, and upgrade Gatsby to 2.19.16.

[pvdz]

@jessepinho ok, if you manage to get another repro repo or test case please share it and I'll have a look and try to fix it :)

[vpreponen]

Deleting lock files and node_modules plus updating to gatsby v2.19.16 and gatsby-source-contentful v2.1.85 fixes the issue for me.

[inblock-io]

@jessepinho ok, if you manage to get another repro repo or test case please share it and I'll have a look and try to fix it :)

I have one here: https://github.com/gatsbyjs/gatsby/issues/27515