Gatsby: Question: How do I query based on gatsby-source-filesystem name?

[saviomuc]

I have multiple folders for different page types. Ie. type1 , type2, etc. which contain markdown files.

pages
   --type1
   --type2

The files in each folder are transformed using gatsby-transformer-remark.
I want to query only content for type1, type2, etc.
For instance I want a type1 list page which contains a list of all type1 pages.

gatsby-config.js

   {
      resolve: 'gatsby-source-filesystem',
      options: {
        path: `${__dirname}/src/pages/type1`,
        name: 'type1',
      },
    },
    {
      resolve: 'gatsby-source-filesystem',
      options: {
        path: `${__dirname}/src/pages/type2`,
        name: 'type2',
      },
    },

My assumption was that you could query based on the name property.

Something like:

{
  someQueryForGatsbySourceFilesystem(
    filter: {
      name: {
        eq: "type1"
      }
    }
  ) {
    edges {
      node {
        childMarkdownRemark {
          html
        }
      }
    }
  }
}

I tried all different queries like allSite and allFile but could not find a solution.
I also tried

  allSitePlugin(filter: {
    pluginOptions: {
      name: {
        eq: "type1"
      }
    }
  })

which doesnt seem to return something which I can use to render content:

"Cannot query field \"childMarkdownRemark\" on type \"SitePlugin\".",

How would you do it? Use allFile and filter based on the path name (i.e. a regex which matches the folder type1 and markdown extension)?

 allFile(
  filter: {
    absolutePath:{regex:"/(type1)\/.*\\.md$/"}
  }
) {...}

[saviomuc]

I figured you could also filter the path using allMarkdownRemark:

{
   allMarkdownRemark(
    sort: { order: DESC, fields: [frontmatter___date]},
    filter: {fileAbsolutePath: {regex: "/(type1)/.*\\.md$/"}}
  ) {
      edges {
        node {
          excerpt(pruneLength: 250)
          id
          frontmatter {
            title
            date(formatString: "MMMM DD, YYYY")
            path
          }
        }
      }
    }
}

[KyleAMathews]

File nodes have the source instance name set as sourceInstanceName so you can use that for queries.

Using regex is a great solution as well which we use a lot on gatsbyjs.org for example.

[KyleAMathews]

BTW, if you're not using it already, GraphiQL is great for exploring what data is available.

[nwshane]

@KyleAMathews Is there a way to use sourceInstanceName in the allMarkdownRemark query? I was hoping that I could get the sourceInstanceName of the file node for each markdown node like this, so that I can separate my blog posts from my projects without using regex on the the file paths.

{
  allMarkdownRemark {
    edges {
      node {
        sourceInstanceName
      }
    }
  }
}

But the GraphQL debugger shows that sourceInstanceName is not available.

How do you recommend filtering markdown pages by source type, such as projects and blog posts? My current solution for this is to get the fileAbsolutePath of each markdown node:

{
  allMarkdownRemark {
    edges {
      node {
        fileAbsolutePath
      }
    }
  }
}

And then to test whether it includes certain strings, like /pages/blog/ or /pages/projects/. But this will break if the absolute path to the gatsby site directory itself contains one of these strings. What do you recommend as the best practice in this case? (Sorry if I missed something in the documentation.)

[KyleAMathews]

@nwshane just query allFile and filter there. You can filter for markdown files by internal.mediaType.

[chmac]

@KyleAMathews How does one combine a query on allFile with a sort on frontmatter? Is that possible?

[lukejanicke]

Trying to figure out something similar.

I can filter by subfolder. But I can’t see frontmatter in allFile.

UPDATE:

This worked for me.

I have folders posts and pages inside the folder content and my gatsby-config.js points to content.

export const query = graphql`
  query IndexQuery {
    allMarkdownRemark(
      filter: { fileAbsolutePath: {regex : "\/posts/"} },
      sort: {fields: [frontmatter___date], order: DESC},
    ) {
      totalCount
      edges {
        node {
          id
          frontmatter {
            title
            slug
            date(formatString: "DD MMMM YYYY")
            category
          }
          excerpt
        }
      }
    }
  }
`;

[chmac]

@lukejanicke Since my last update I dived into the node system a little more. The allFile nodes get parsed by the markdown handler and then new allMarkdownRemark nodes get created. So as you found, it’s not possible to sort by frontmatter in allFile because it’s only generated once the markdown parser kicks in. The regex works, but it’s a bit fragile for my taste. My solution was to copy the collection name (specified in the filesystem source plugin config) from the file node into the markdown node via an onCreateNode hook. Don’t have the code in front me to post an example, but it was very quick and then I have a field inside allMarkdownRemark that I can filter by.

[gbouteiller]

Another possibility as @KyleAMathews advised ( in my case, the source name is projects )

allFile(filter: {internal: {mediaType: {eq: "text/markdown"}}, sourceInstanceName: {eq: "projects"}}) {
  edges {
    node {
      childMarkdownRemark {
        frontmatter {
            ...
        }
      }
    }
  }
}

[chmac]

@gbouteiller True, that will get the frontmatter and allow one to filter by sourceInstanceName. The trouble is, you cannot sort by anything inside frontmatter using this approach. It seems like folks are talking about blog posts here, and they are typically sorted by date. The date is typically stored inside the frontmatter.

There is no way to do sort by something from frontmatter and filter by sourceInstanceName by default. The sourceInstanceName is not copied from the File node into the markdownRemark node. The only option is to use a regex or to copy some data as per my earlier post.

If sorting is not required, which might be true in many cases, then your approach makes a lot of sense. I hadn't realised the childMarkdownRemark was available as a field on file, that's super useful to know, thanks for sharing.

[tsiq-swyx]

had this issue to and was kind of disappointed that the "name" field doesnt work well with the markdownRemark. is there a small win here that we can do to pass this information on?

EDIT: after messing around a bit i realized that the id field exposes the path and you can filter based on that:

query AllQuery {
    DSes: allMarkdownRemark(
      limit: 3
      filter: { id: { regex: "/_design-systems/" } }
    ) {
      edges {
        node {
          frontmatter {
            title
            date
            company
            link
            image
            description
          }
          fields {
            slug
          }
        }
      }
    }

    Articles: allMarkdownRemark(
      limit: 3
      filter: { id: { regex: "/_articles/" } }
    ) {
      edges {
        node {
          frontmatter {
            title
            date
            company
            link
            image
            description
          }
          fields {
            slug
          }
        }
      }
    }
  }

the name field from source filesystem still doesnt really do anything for gatsby-transformer-remark, i would say it improves the dx a bit to flow that through as a queryable/filterable field since gatsby-transformer-remark never exists without gatsby-source-filesystem.

[chmac]

@tsiq-swyx I'm using something like this to pass the collection field from File to MarkdownRemark.

exports.onCreateNode = ({ node, boundActionCreators, getNode }) => {
  const { createNodeField } = boundActionCreators

  if (_.get(node, 'internal.type') === `MarkdownRemark`) {
    // Get the parent node
    const parent = getNode(_.get(node, 'parent'))

    // Create a field on this node for the "collection" of the parent
    // NOTE: This is necessary so we can filter `allMarkdownRemark` by
    // `collection` otherwise there is no way to filter for only markdown
    // documents of type `post`.
    createNodeField({
      node,
      name: 'collection',
      value: _.get(parent, 'sourceInstanceName'),
    })

The regex approach does work, but it's fragile.

[tsiq-swyx]

@chmac this is very clever, and it works, thank you for sharing your solution. @KyleAMathews sorry for tagging you (lmk if theres someone else to tag) but is it possible for gatsby-source-filesystem to work this way (passing sourceInstanceName to a node field or some other field) by default?

[chmac]

@tsiq-swyx I'm not 100% sure, but I'd imagine it might be a fairly simple pull request! ;-)

[tsiq-swyx]

i think the gatsby maintainers have strong opinions about these things so i think its worth discussing before sending in random unsolicited PRs :)

[chiumax]

Okay i got it working, but my createPages export only uses one template

[Behinder]

It is intresting to see that using gatsby-source-filesystem does not require AllFile in graphql at all. Which may bring confusion and it is not reflected in documentation.
I am using
const { createPage } = actions const result = await graphql(` query { allMarkdownRemark { edges { node { fields { slug } frontmatter { lang } } } } } `)
and it creates pages with no problem. Not sure however if same principle applies to gatsby-source-git

[NomanGul]

Thanks @chmac

I've to create pages based on the gatsby-source-filesystem name...

so here is my solution:

const { createFilePath } = require("gatsby-source-filesystem")

exports.onCreateNode = ({ node, actions, getNode }) => {
  const { createNodeField } = actions

  if (node.internal.type === `Mdx` && getNode(node.parent).sourceInstanceName === "careers") {
    const value = createFilePath({ node, getNode })

    createNodeField({
      name: "slug",
      node,
      value: `/careers${value}`
    })
  }
}

[Behinder]

This is intresting, Never found such option node.parent exists

On Monday, 27 April 2020, Noman Gul [e-mail] wrote:

Thanks @chmac https://github.com/chmac

I've to create pages based on the gatsby-source-filesystem name...

so here is my solution:

const { createFilePath } = require("gatsby-source-filesystem")
exports.onCreateNode = ({ node, actions, getNode }) => {
const { createNodeField } = actions

if (node.internal.type === Mdx && getNode(node.parent).sourceInstanceName === "careers") {
const value = createFilePath({ node, getNode })

createNodeField({
  name: "slug",
  node,
  value: `/careers${value}`
})

}
}

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/gatsbyjs/gatsby/issues/1634#issuecomment-619670883,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAEQPFX6KR7NUIY33Q7V4MDROTS7PANCNFSM4DUUPK6A
.

[Inambe]

I'm in the same situation which I think is fairly common case and after reading this thread; best and non-hacky solution, in my opinion, is:

{
  allMarkdownRemark(filter: {fileAbsolutePath: {regex: "/projects/"}}) {
    edges {
      node {
        frontmatter {
          title
        }
      }
    }
  }
}

My projects type of content is in /content/projects/.