gatsby-source-contentful rich text embedded entries are not resolved

[sarahbethfederman]

Description

I've been working on using the rich text fields in gatsby and it seems that embedded entries are supposed to be resolved since this plugin uses the contentful SDK, but all I see is the id of the entry its supposed to resolve to.

Steps to reproduce

More info on what i've been doing here: https://github.com/contentful/rich-text/issues/46

Expected result

The JSON should have access to all fields of the entry. So for a link I should have things like the slug and title of the entry its linking to.

Actual result

I can only see the target.sys.id. Here's an example of the JSON string returned:

"childContentfulRichText": {
          "internal": {
            "content": "{\"data\":{},\"content\":[{\"data\":{},\"content\":[{\"data\":{},\"content\":[{\"data\":{},\"content\":[{\"data\":{},\"marks\":[],\"value\":\"\",\"nodeType\":\"text\"},{\"data\":{\"target\":{\"sys\":{\"id\":\"c4nCwMtcqBGKOkc0kuyIQGW\",\"type\":\"Link\",\"linkType\":\"Entry\"}}},\"content\":[{\"data\":{},\"marks\":[],\"value\":\"Dialog\",\"nodeType\":\"text\"}],\"nodeType\":\"entry-hyperlink\"},{\"data\":{},\"marks\":[],\"value\":\"\",\"nodeType\":\"text\"}],\"nodeType\":\"paragraph\"}],\"nodeType\":\"list-item\"},{\"data\":{},\"content\":[{\"data\":{},\"content\":[{\"data\":{},\"marks\":[],\"value\":\"\",\"nodeType\":\"text\"},{\"data\":{\"target\":{\"sys\":{\"id\":\"q3YCW7wFmoW8MwOCG4Yoc\",\"type\":\"Link\",\"linkType\":\"Entry\"}}},\"content\":[{\"data\":{},\"marks\":[],\"value\":\"Link\",\"nodeType\":\"text\"}],\"nodeType\":\"entry-hyperlink\"},{\"data\":{},\"marks\":[],\"value\":\"\",\"nodeType\":\"text\"}],\"nodeType\":\"paragraph\"}],\"nodeType\":\"list-item\"}],\"nodeType\":\"unordered-list\"},{\"data\":{},\"content\":[{\"data\":{},\"marks\":[],\"value\":\"\",\"nodeType\":\"text\"}],\"nodeType\":\"paragraph\"}],\"nodeType\":\"document\"}"
          }

Environment

  System:
    OS: macOS 10.14
    CPU: (8) x64 Intel(R) Core(TM) i7-6820HQ CPU @ 2.70GHz
    Shell: 3.2.57 - /bin/bash
  Binaries:
    Node: 8.9.4 - ~/.nvm/versions/node/v8.9.4/bin/node
    Yarn: 1.9.4 - /usr/local/bin/yarn
    npm: 6.5.0 - ~/.nvm/versions/node/v8.9.4/bin/npm
  Browsers:
    Chrome: 71.0.3578.98
    Firefox: 57.0.4
    Safari: 12.0
  npmPackages:
    gatsby: ^2.0.67 => 2.0.67 
    gatsby-image: ^2.0.25 => 2.0.25 
    gatsby-mdx: ^0.2.0 => 0.2.0 
    gatsby-plugin-sass: ^2.0.7 => 2.0.7 
    gatsby-plugin-sharp: ^2.0.15 => 2.0.15 
    gatsby-plugin-styled-components: ^3.0.4 => 3.0.4 
    gatsby-source-contentful: ^2.0.20 => 2.0.20 
    gatsby-transformer-sharp: ^2.1.9 => 2.1.9 
  npmGlobalPackages:
    gatsby-cli: 2.4.6

[sarahbethfederman]

Not sure if related to https://github.com/gatsbyjs/gatsby/issues/9821

[pieh]

I'm not sure just how much support we have for RichText field types ATM. As far as I know this field type is still in beta. (?)

I know @Khaledgarbaya did initial work on support for that. Maybe we "just" use too old version of contentful SDK ( https://github.com/gatsbyjs/gatsby/blob/5b6ab284da4da7d05761b12d79ceb29e5ebb47c1/packages/gatsby-source-contentful/package.json#L14 ) to take advantage of auto resolving those fields (there is version 7.0.5 on npm - https://www.npmjs.com/package/contentful)? But I don't have enough context on contentful to say for sure.

[pieh]

I'll try to setup our example ( https://github.com/gatsbyjs/gatsby/tree/master/examples/using-contentful ) to use that feature so it's easier to reproduce for anyone who want to look into it - just need to figure out what "embedded entries" actually are

[pieh]

So initial finding is that for some reason a lot of data is available on the first run (after deleting .cache directory):

but after restarting all of that data goes away and we only have access to sys fields.

[Khaledgarbaya]

Hey Folks,

The link resolution won't be possible on subsequent sync because there is no enough data available for the sdk to resolve it.
removing the .cache directory will remove the sync token thus the sdk will perform an initial sync with all the data

[polarathene]

@Khaledgarbaya would it be better to cache the data locally as a JSON file for Rich Text nodes? I've just added support to the plugin with a PR for downloading contentful assets and providing a field that can be queried in graphql to get local file links instead of the contentful urls. At least my understanding is there is some problem with how it's caching the data atm?

I've not tried with any embedded entries or messing with internal like@sarahbethfederman was doing. I just take the html string and combine with react-jsx-parser(might break on tags like <hr>).

{
    contentfulPageService {
      title
      description { childContentfulRichText { html } }
    }
}

Is the cache issue with sync token specific to the nodes internal content only? Or the way it's being queried by @pieh ?

[sarahbethfederman]

Just throwing this out there...is there a way to possibly use the contentful sync API to update the content? https://www.contentful.com/developers/docs/concepts/sync/

[sarahbethfederman]

Fwiw, updating from gatsby 2.0.83 to 2.0.90 breaks the resolution sometimes even on gatsby build, which previously worked (gatsby develop is the problem)

[sarahbethfederman]

Also it turns out that nested embedded entries (embedded entries that include an embedded entry) are not resolved either.

[Khaledgarbaya]

Hey @sarahbethfederman,
I would like to investigate the problem further with your help.
How can we work together on getting a small project that replicates this issue?
From there I can debug it and come up with a fix.

[sarahbethfederman]

I can't do that as it's an internal project and can't publish the keys or code. Have you tried reproducing with nested embedded entries?

[polarathene]

@sarahbethfederman you're not able to make a similar reproduction repo? Your issue can be generalized out of it's current context with the internal project right?

using-contentful example might be of use to you? You can register a personal account with Contentful(assuming the one you're using atm is related to business/work), setup a free space that demonstrates the problem well, then once sorted, delete the space and reproduction repo if you like?

@Khaledgarbaya can probably provide you with a space failing that.

[sarahbethfederman]

Huh weird. Nested embedded entries seem to have started working again

[Khaledgarbaya]

@sarahbethfederman,
Glad it works now, as I said if you still have a problem you can make a sample gatsby project and I can take care of the contentful side of things I will need to know the structure of the content.
If the issue is solved for you now you can close this issue

[polarathene]

It would be good to have an example of it in the documentation. I've not played with rich text much myself, but it sounds like there are currently gotcha's to using or understanding how to use it properly with gatsby/react, so it'd probably be really helpful to others that run into the same situation?

[FrankM1]

Removing the .cache directory seems to resolve this issue.

[sarahbethfederman]

This issue is absolutely not fixed. Gatsby develop is still completely unusable and I can't delete my cache.

[polarathene]

I can't delete my cache.

@sarahbethfederman Why is that?

If this is still an issue, providing a small reproduction repo for others to use can help with collaboration.

[sarahbethfederman]

I can't delete my cache because it breaks with another error https://github.com/contentful/rich-text/issues/46#issuecomment-451267836

This is my last day on the project so I can't make another repo but other people are having this issue too.

To be clear: gatsby develop has never resolved the entries properly. Gatsby build used to resolve all entries properly, but as I said in my above comment it broke when upgrading gatsby (the first few entries don't resolve), and I experienced an issue where nested embedded entries (embedded entries that include another embedded entry) didn't resolve.

The nested embedded entries are now resolving. Not being able to use gatsby develop is still a huge issue and not being able to update gatsby is still an issue

[dyllandry]

I put together a github repo that I think demonstrates this issue. I created a dummy contentful account, so both the space id and access token are in gatsby-config.js as plaintext.

When retrieving 'blogPosts' and accessing its 'body' rich text field, assets within have content field values of empty arrays. This url should open up the graphiql query automatically.

Let me know if I can be more helpful. I'm not as deep into this as everyone else but I would also like to find a solution. I don't know otherwise how to get images in my blog posts.

[skywinston]

We're having a similar issue with Contentful Rich Text fields, although the issue is that nested embeds can be recursively linked to other embeds, ad infinitum. On running gatsby develop we are getting about 5GB of data back because we were getting the full content of every nested embed. Are there plans to configure how deep this nesting goes? Our system is currently broken because we've hit the upper bounds of memory on the server.

[gatsbot[bot]]

Hiya!

This issue has gone quiet. Spooky quiet. 👻

We get a lot of issues, so we currently close issues after 30 days of inactivity. It’s been at least 20 days since the last update here.

If we missed this issue or if you want to keep it open, please reply here. You can also add the label "not stale" to keep this issue open!

Thanks for being a part of the Gatsby community! 💪💜

[martzoukos]

Hello there,

I am Contentful's product manager working on our Rich Text editor and we recently released an improvement in the gatsby-source-contentful plugin where it doesn't return a flattened HTML string anymore but the Rich Text AST as a JSON object, with all the embedded entries and assets resolved. Then you can use our rendering libraries in order to render the content into HTML/React/Plain text.

Thank you and let me know if that works.

[martzoukos]

Hello there,

I am Contentful's product manager working on our Rich Text editor and we recently released an improvement in the gatsby-source-contentful plugin where it doesn't return a flattened HTML string anymore but the Rich Text AST as a JSON object, with all the embedded entries and assets resolved. Then you can use our rendering libraries in order to render the content into HTML/React/Plain text.

I know that I've posted the same message in all your other issues too @sarahbethfederman but I hope other people who were having the same issue also benefit from this. Sorry for the spam!

[gatsbot[bot]]

Hey again!

It’s been 30 days since anything happened on this issue, so our friendly neighborhood robot (that’s me!) is going to close it.

Please keep in mind that I’m only a robot, so if I’ve closed this issue in error, I’m HUMAN_EMOTION_SORRY. Please feel free to reopen this issue or create a new one if you need anything else.

Thanks again for being part of the Gatsby community!

[tweetzal]

HIi, I have just linked to an entry in a RIch text field, and instead of a url to the entry I can type: entry-hyperlink id: XXXXXXXXXXXXXXX (with the actuall id that is) display on the front end.

Why doesn't it automatically link to the resources / asset by displaying a url?

[choffmeister]

@pieh I did some digging and found a way to consistently reproduce this:

1. Clear cache
2. Start gatsby in develop
3. Access content with a richtext field with embedded image via gatsby GQL. You get all data
   json { "content": [], "data": { "target": { "fields": { "file": { "en-US": { "contentType": "image/png", "details": { "image": { "height": 965, "width": 1680 }, "size": 378599 }, "fileName": "xxxxx.png", "url": "//images.ctfassets.net/xxxxx/xxxxx/xxxxx/xxxxx.png" } }, "title": { "en-US": "xxxxx" } }, "sys": { "createdAt": "2019-06-28T06:37:08.478Z", "environment": { "sys": { "id": "master", "linkType": "Environment", "type": "Link" } }, "id": "xxxxx", "revision": 1, "space": { "sys": { "id": "xxxxx", "linkType": "Space", "type": "Link" } }, "type": "Asset", "updatedAt": "2019-06-28T06:37:08.478Z" } } }, "nodeType": "embedded-asset-block" }
4. Change some content in contentful
5. Restart gatsby develop
6. Access content with a richtext field with embedded image via gatsby GQL. You only get minimal data
   json { "data": { "target": { "sys": { "id": "xxxxx", "type": "Link", "linkType": "Asset" } } }, "content": [], "nodeType": "embedded-asset-block" },

The problem is something with the sync in the contentful SDK. If I disable incremental sync (by manually setting the sync-token used to undefined) I always get the full information.

Side note: Updating to the latest contentful SDK version 7.x did not help.

A quick workaround would be to allow to configure gatsby-source-contentful to not use a sync token (but instead to always do a full sync).

[alexlee-dev]

For anyone coming here looking for a solution on how to get data all the time (not just on the first go), the above PR adds a handy option you can pass to your config.

{
  resolve: "gatsby-source-contentful",
    options: {
      forceFullSync: true
    }
}

https://www.gatsbyjs.org/packages/gatsby-source-contentful/#configuration-options

👨‍💻

[stuckj]

Hello there,

I am Contentful's product manager working on our Rich Text editor and we recently released an improvement in the gatsby-source-contentful plugin where it doesn't return a flattened HTML string anymore but the Rich Text AST as a JSON object, with all the embedded entries and assets resolved. Then you can use our rendering libraries in order to render the content into HTML/React/Plain text.

I know that I've posted the same message in all your other issues too @sarahbethfederman but I hope other people who were having the same issue also benefit from this. Sorry for the spam!

~@martzoukos, are your changes in the current version of gatsby-source-contentful now are they in a separate branch or forked repo? Inside the childContentfulRichText.internal.content node I can see stringified JSON that I can parse that looks like it has something I can kinda use for embedded objects. But, is there a version of this that isn't stringified so I don't have to separately parse this in my code? Not the end of the world, but it's an annoyance. :)~

UPDATE: Nevermind. :) I just found your doc that explains this: https://www.contentful.com/developers/docs/tutorials/general/rich-text-and-gatsby/. it seems like in this case I want json instead of childContentfulRichText.internal.content and then can use the rich text rendering libraries you guys have to handle rendering it. Perfect!

[abohannon]

Was there a solution found for this? We cannot get INLINE or BLOCK.EMBEDDED_ENTRY instances to maintain their data in development/preview. The target property becomes an empty object. This obviously makes development a huge headache.

This issue is not present with BLOCKS.EMBEDDED_ASSET.

I'm reluctant to use forceFullSync because it seems there would be considerable performance issues with fully syncing every change in preview.

[axe312ger]

Thanks to @coreyward and @daydream05 for digging this up.

This issue should now be solved with the new canary version of gatsby-source-contentful

Please check the install & migration instructions at https://github.com/gatsbyjs/gatsby/pull/25249

I'd love to hear if this fixes your issues. The new way to render Rich Text should give you full flexibility and resolve data properly.

[devpatelwebdeveloper]

I am having the same issue yet. Its not solved for sure. when I use raw instead of json I get error saying cannot query field raw on RichTextNode

[github-actions[bot]]

Hiya!

This issue has gone quiet. Spooky quiet. 👻

We get a lot of issues, so we currently close issues after 60 days of inactivity. It’s been at least 20 days since the last update here.
If we missed this issue or if you want to keep it open, please reply here.
As a friendly reminder: the best way to see this issue, or any other, fixed is to open a Pull Request. Check out gatsby.dev/contribute for more information about opening PRs, triaging issues, and contributing!

Thanks for being a part of the Gatsby community! 💪💜

[axe312ger]

@devpatelwebdeveloper you have to install the new canary version of the plugin, probably also clean gatsby cache to get the se fields. See #25249 on how to install & migrate your code

[github-actions[bot]]

Hiya!

This issue has gone quiet. Spooky quiet. 👻

We get a lot of issues, so we currently close issues after 60 days of inactivity. It’s been at least 20 days since the last update here.
If we missed this issue or if you want to keep it open, please reply here.
As a friendly reminder: the best way to see this issue, or any other, fixed is to open a Pull Request. Check out gatsby.dev/contribute for more information about opening PRs, triaging issues, and contributing!

Thanks for being a part of the Gatsby community! 💪💜

[axe312ger]

Closing here, lets continue further discussion in #25249