Gatsby: [gatsby-source-contentful] Multiple spaces support?

[hirviid]

Is it possible to configure the contentful plugin to support multiple spaces?
I need to pull content from multiple spaces and combine them.

[KyleAMathews]

Yes just create multiple instances of the plugin in gatsby-config.js

Avoid having types with the same name in diff spaces as the plugin doesn't handle that.

[hirviid]

Thank you for your quick response @KyleAMathews .

I tried this:

module.exports = {
  siteMetadata: {
    title: `My Project`,
  },
  plugins: [
    {
      resolve: `gatsby-source-contentful`,
      options: {
        spaceId: `...`,
        accessToken: `...`,
      },
    },
    {
      resolve: `gatsby-source-contentful`,
      options: {
        spaceId: `...`,
        accessToken: `...`,
      },
    },
    `gatsby-transformer-remark`,
  ],
};

But I get an error when I try to run/build the project:

GraphQL Error Unknown field `contentfulPand` on type `RootQueryType`

It recognizes the fields from the first configuration, but not from the second.

I inspected the .cache/redux-state.json file, and I found that it _does_ contain all content types, from both configurations, but only the content from the first configuration.

[KyleAMathews]

Won't have time to dig into this but it's probably something relatively simple that needs fixed if you'd like to out together a PR using instructions at https://www.gatsbyjs.org/docs/how-to-contribute/#contributing

[snulv]

I've believe I have found the reason for this issue. It seems that both of the plugins share a synch key. The Contentful SDK has a synch function that only downloads the data that has been changed since last time. And this key is saved in such a way that both plugins use the same key, and since the second plugin runs less than a second after the first plugin it is unlikely that any data has been changed since then.

[szimek]

@KyleAMathews The way we're using Contentful is that we have a different space for each language, but we have _exactly the same_ content type in every space. We do it because not every content is translated to all languages, e.g. when we have a region specific marketing campaign. I know that we're not the only ones using it like this.

Additionally, I'd need to generate a different folder structure for each space that includes locale, e.g. /<locale>/posts/<slug>. What would be the best way to solve it?

Would it make sense to have multiple gatsby configs, one for each space? Or maybe it would make sense to modify this plugin somehow to allow multiple spaces? Or maybe something there's some other solution?

We'll probably have similar issue with our translations - some content (e.g. labels like "next", "back" etc.) is not stored in Contentful, but is fetched from JSON files (that come from Transifex, but it's not important), with one file per language, e.g. en.json, de.json, fr.json etc.

EDIT: So far my idea is to have multiple Gatsby config files like, gatsby-config.en.js, gatsby-config.de.js and for each file, rename it to gatsby-config.js (looks like gatsby-cli doesn't allow to specify config file :/), run gatsby build, move somewhere the generated build, possibly changing folder structure (i.e. add locale). At the end merge all builds together and deploy! :) Can it even work? Is there an easier way? The biggest disadvantage is that it won't work with gatsby develop :/

[KyleAMathews]

@szimek you can create as many pages as you want in any way you want. Page creation is programmatic. So no need for multiple gatsby-node.js

[szimek]

@KyleAMathews Thanks, but how exactly can I do it?

If all spaces have the same content type, e.g. Post, even if I define gatsby-source-contentful plugin in gatsby-config.js multiple times with different options and then query allContentfulPost in createPages function, I'm getting entries only for the last (?) defined space.

Can I dynamically change options for Contentful plugin in createPages function, so I could run the query in createPages multiple times?

[EDIT] I tried to modify the Contentful plugin to attach space id to node id and its internal type, so that e.g. Post would have node id Post123456 and internal type ConentfulPost123456, so that I could query e.g. allConentfulPost123456. This would make them unique, even if I define this plugin more than once. Unfortunately, even though I can see all nodes being created thanks to my extensive console.log logging 😉, in the graphiql I still can't see all content types. Obviously, I don't know what I'm doing, but I hoped it might just work...

[szimek]

@KyleAMathews Another idea ;) Because I can't figure out how I can use gatsby-source-contentful plugin with multiple spaces that all have the same content type, how about generating JSON files from Contentful data and then using gatsby-source-filesystem with gatsby-transformer-json to create pages?

I could then "merge" data from multiple spaces to create any file structure I want, e.g.:

├── en
│   └── my-site.json
├── de
│   └── meine-seite.json
└── pl
    └── moja-strona.json

What do you think?

BTW. How does caching work exactly in Gatsby? We're planning to migrate about 5000 pages in 6 different languages from Wordpress to Contentful and it would be awesome to not regenerate everything from scratch e.g. if only one page gets published or modified in Contentful. I can also ask about it elsewhere, to not spam this thread anymore ;)

[KyleAMathews]

@szimek oh just remembered what "spaces" are in Contentful :-)

Have you tried adding new instances in the config? You can add as many instances of a plugin as you want e.g. when you want to pull in multiple spaces.

[KyleAMathews]

Yeah, the way to do this is to add additional instances of the plugin, 1 / space.

[szimek]

Not in all cases. Gatsby will merge content types with the same name from different spaces into a single graphql type.

It's kind of ok if these content types are really exactly the same, but of course it's possible that e.g. Post content type in one space will be completely different than content type of the same name in another space. To actually solve it, space id would probably need to be part of the generated type name, e.g. instead of contentfulPost it would be contentfulPostFromSpace123456 (or sth similar), which kind of sucks as well.

In our case, content types of the same names are (so far) exactly the same in all spaces, except for locale, which is different in every space. Thus, when we use contentful source plugin, we end up with entries, where e.g. title field looks like this:

title: {
  de: null, 
  en: "Hello",
  es: null, 
  fr: null,
  pl: null,
  pt: null,
  tr: null
  ...
}

which not very convenient...

In our case, we've decided (for now, we'll see how it works out in the end) to convert data from Contentful to JSON files with the structure we want using our own script and then use filesystem source plugin together with JSON and markdown transformer plugins.

[KyleAMathews]

One easy way to solve this would be to add support for adding a prefix to types. So for each instance, you could add a prefix of e.g. ES so then the type would be contentfulESPost.

[kiliw]

I just came across this issue because we also wanted to have content from two contentful spaces merged. I tried the way of @hirviid

module.exports = {
  siteMetadata: {
    title: `...`,
    description: `...`,
    author: `...`,
  },
  plugins: [
       {
      resolve: `gatsby-source-contentful`,
      options: {
        spaceId: `ABCD`,
        accessToken: '1234',
        host: `cdn.contentful.com`,
      },
    },
    {
      resolve: `gatsby-source-contentful`,
      options: {
        spaceId: `WXYZ`,
        accessToken: '7890',
        host: `cdn.contentful.com`,
      },
    },
     ],
}

For us the important thing was to rename the contentmodels (the content type ID does not matter apparently) in the one space to make sure there are no dublicates.

Thanks beeing able to use multiple spaces helped us to save a lot of time :)