Gatsby: graphql doesn't seems to return correct row

Created on 20 Dec 2018  路  8Comments  路  Source: gatsbyjs/gatsby

I am trying to display rows from my plugin component by calling a REST API. When i query against
gatsbyjs graphql (http://localhost:8000/___graphql), it always returns 5 records. I tried to restart multiple times and every time it fire off, i can see atleast 20 records are being return from my custom REST API endpoint.

My query looks pretty straight forward :-
{
allProductApi {
edges {
node {
id
name
description
imgUri
productUri
}
}
}
}

I tried delete .cache directory and start gatsby develop --no-cache. Any help would be great. Thanks.

question or discussion
Source
picture appcoreopc

All 8 comments

Thank you for opening this @appcoreopc

Can you please post more details about how you're querying the API? Are you using a specific plugin?

sidharthachatterjee picture sidharthachatterjee on 20 Dec 2018

yes i am building a very simple custom plugin.

const fetch = require("node-fetch")
const queryString = require("query-string")

exports.sourceNodes = (
  { actions, createNodeId, createContentDigest },
  configOptions
  ) => {
    const { createNode } = actions

    // Gatsby adds a configOption that's not needed for this plugin, delete it
    delete configOptions.plugins

    const productApiUrl = "http://localhost:3000/product";
    // Fetch a response from the apiUrl

    // Helper function that processes a photo to match Gatsby's node structure
    const mapDataToNode = product => {
      const nodeId = createNodeId(`productapi-${product.id}`)
      const nodeContent = JSON.stringify(product)
      const nodeData = Object.assign({}, product, {
        id: nodeId,
        parent: null,
        children: [],
        internal: {
          type: `ProductApi`,
          content: nodeContent,
          contentDigest: createContentDigest(product),
          username : 'appcoreopc'
        },
      })

      return nodeData
    }

    return (

      fetch(productApiUrl)
      // Parse the response as JSON
      .then(response => response.json())
      // Process the JSON data into a node
      .then(data => {
        // For each query result (or 'hit')            
        data.forEach(p => {
          const nodeData = mapDataToNode(p);
          createNode(nodeData);
        });      
      })
      )      
    }
appcoreopc picture appcoreopc on 20 Dec 2018

@appcoreopc based on the information at hand, i've created a simple express server that returns a json structure to emulate your own, when the route http://localhost:3000/product is hit.
My server will return 

[
  {
    "id": 1,
    "description": "desc1",
    "imgUri": "http://writingexercises.co.uk/images2/randomimage/heavybox.jpg",
    "productUri": "http://localhost:3000/page-1"
  },
  {
    "id": 2,
    "title": "dummy2",
    "description": "desc1",
    "imgUri": "http://writingexercises.co.uk/images2/randomimage/heavybox.jpg",
    "productUri": "http://localhost:3000/page-2"
  },
  {
    "id": 3,
    "title": "dummy3",
    "description": "desc1",
    "imgUri": "http://writingexercises.co.uk/images2/randomimage/heavybox.jpg",
    "productUri": "http://localhost:3000/page-3"
  },
  {
    "id": 4,
    "title": "dummy1",
    "description": "desc1",
    "imgUri": "http://writingexercises.co.uk/images2/randomimage/heavybox.jpg",
    "productUri": "http://localhost:3000/page-4"
  },
  {
    "id": 5,
    "title": "dummy1",
    "description": "desc1",
    "imgUri": "http://writingexercises.co.uk/images2/randomimage/heavybox.jpg",
    "productUri": "http://localhost:3000/page-3"
  },
  {
    "id": 6,
    "title": "dummy1",
    "description": "desc1",
    "imgUri": "http://writingexercises.co.uk/images2/randomimage/heavybox.jpg",
    "productUri": "http://localhost:3000/page-6"
  },
  {
    "id": 7,
    "title": "dummy1",
    "description": "desc1",
    "imgUri": "http://writingexercises.co.uk/images2/randomimage/heavybox.jpg",
    "productUri": "http://localhost:3000/page-7"
  },
  {
    "id": 8,
    "title": "dummy1",
    "description": "desc1",
    "imgUri": "http://writingexercises.co.uk/images2/randomimage/heavybox.jpg",
    "productUri": "http://localhost:3000/page-8"
  },
  {
    "id": 9,
    "title": "dummy1",
    "description": "desc1",
    "imgUri": "http://writingexercises.co.uk/images2/randomimage/heavybox.jpg",
    "productUri": "http://localhost:3000/page-9"
  },
  {
    "id": 10,
    "title": "dummy1",
    "description": "desc1",
    "imgUri": "http://writingexercises.co.uk/images2/randomimage/heavybox.jpg",
    "productUri": "http://localhost:3000/page-10"
  },
  {
    "id": 11,
    "title": "dummy1",
    "description": "desc1",
    "imgUri": "http://writingexercises.co.uk/images2/randomimage/heavybox.jpg",
    "productUri": "http://localhost:3000/page-11"
  },
  {
    "id": 12,
    "title": "dummy1",
    "description": "desc1",
    "imgUri": "http://writingexercises.co.uk/images2/randomimage/heavybox.jpg",
    "productUri": "http://localhost:3000/page-12"
  },
  {
    "id": 13,
    "title": "dummy1",
    "description": "desc1",
    "imgUri": "http://writingexercises.co.uk/images2/randomimage/heavybox.jpg",
    "productUri": "http://localhost:3000/page-13"
  },
  {
    "id": 14,
    "title": "dummy1",
    "description": "desc1",
    "imgUri": "http://writingexercises.co.uk/images2/randomimage/heavybox.jpg",
    "productUri": "http://localhost:3000/page-14"
  },
  {
    "id": 15,
    "title": "dummy1",
    "description": "desc1",
    "imgUri": "http://writingexercises.co.uk/images2/randomimage/heavybox.jpg",
    "productUri": "http://localhost:3000/page-15"
  },
  {
    "id": 16,
    "title": "dummy1",
    "description": "desc1",
    "imgUri": "http://writingexercises.co.uk/images2/randomimage/heavybox.jpg",
    "productUri": "http://localhost:3000/page-16"
  },
  {
    "id": 17,
    "title": "dummy1",
    "description": "desc1",
    "imgUri": "http://writingexercises.co.uk/images2/randomimage/heavybox.jpg",
    "productUri": "http://localhost:3000/page-17"
  },
  {
    "id": 18,
    "title": "dummy1",
    "description": "desc1",
    "imgUri": "http://writingexercises.co.uk/images2/randomimage/heavybox.jpg",
    "productUri": "http://localhost:3000/page-18"
  },
  {
    "id": 19,
    "title": "dummy1",
    "description": "desc1",
    "imgUri": "http://writingexercises.co.uk/images2/randomimage/heavybox.jpg",
    "productUri": "http://localhost:3000/page-19"
  },
  {
    "id": 20,
    "title": "dummy1",
    "description": "desc1",
    "imgUri": "http://writingexercises.co.uk/images2/randomimage/heavybox.jpg",
    "productUri": "http://localhost:3000/page-20"
  },
  {
    "id": 21,
    "title": "dummy1",
    "description": "desc1",
    "imgUri": "http://writingexercises.co.uk/images2/randomimage/heavybox.jpg",
    "productUri": "http://localhost:3000/page-21"
  },
  {
    "id": 22,
    "title": "dummy1",
    "description": "desc1",
    "imgUri": "http://writingexercises.co.uk/images2/randomimage/heavybox.jpg",
    "productUri": "http://localhost:3000/page-22"
  }
]

From your code, i've seen that you've been following the tutorial on how to create a source plugin with gatsby.
With that in mind i've created a reproduction following the documentation and i've end up with the following:
My folder structure is the following:

  • plugins

    • gatsby-source-products



      • gatsby-node.js





        • The contents of this file is a carbon copy of the one you've posted as you can see bellow





      • package.json



const fetch = require("node-fetch")
const queryString = require("query-string")

exports.sourceNodes = (
  { actions, createNodeId, createContentDigest },
  configOptions
  ) => {
    const { createNode } = actions

    // Gatsby adds a configOption that's not needed for this plugin, delete it
    delete configOptions.plugins

    const productApiUrl = "http://localhost:3000/product";
    // Fetch a response from the apiUrl

    // Helper function that processes a photo to match Gatsby's node structure
    const mapDataToNode = product => {
      const nodeId = createNodeId(`productapi-${product.id}`)
      const nodeContent = JSON.stringify(product)
      const nodeData = Object.assign({}, product, {
        id: nodeId,
        parent: null,
        children: [],
        internal: {
          type: `ProductApi`,
          content: nodeContent,
          contentDigest: createContentDigest(product),
          username : 'appcoreopc'
        },
      })

      return nodeData
    }

    return (

      fetch(productApiUrl)
      // Parse the response as JSON
      .then(response => response.json())
      // Process the JSON data into a node
      .then(data => {
        // For each query result (or 'hit')            
        data.forEach(p => {
          const nodeData = mapDataToNode(p);
          createNode(nodeData);
        });      
      })
      )      
    }

The plugin package.json looks like this

{
  "name": "gatsby-source-products",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "dependencies": {
    "node-fetch": "^2.3.0",
    "query-string": "^6.2.0"
  }
}

As per the documentation i've changed the gatsby-config.js file located in the root to accomodate the new plugin like so:

module.exports = {
  siteMetadata: {
    title: `Gatsby Default Starter`,
    description: `Kick off your next, great Gatsby project with this default starter. This barebones starter ships with the main Gatsby configuration files you might need.`,
    author: `@gatsbyjs`,
  },
  plugins: [
    `gatsby-plugin-react-helmet`,
    {
      resolve: `gatsby-source-filesystem`,
      options: {
        name: `images`,
        path: `${__dirname}/src/images`,
      },
    },
    `gatsby-transformer-sharp`,
    `gatsby-source-products`,
    `gatsby-plugin-sharp`,
    {
      resolve: `gatsby-plugin-manifest`,
      options: {
        name: `gatsby-starter-default`,
        short_name: `starter`,
        start_url: `/`,
        background_color: `#663399`,
        theme_color: `#663399`,
        display: `minimal-ui`,
        icon: `src/images/gatsby-icon.png`, // This path is relative to the root of the site.
      },
    },
    // this (optional) plugin enables Progressive Web App + Offline functionality
    // To learn more, visit: https://gatsby.app/offline
    // 'gatsby-plugin-offline',
  ],
}

opening up the gatsby graphiql editor located at http://localhost:8000/___graphql and using the following graphql query

{
  allProductApi {
    edges {
      node {
        id
        description
        imgUri
        productUri
      }
    }
  }
}

results in the following

{
  "data": {
    "allProductApi": {
      "edges": [
        {
          "node": {
            "id": "ffbf6acd-d6b5-5d62-9d14-a93b42f66c85",
            "description": "desc1",
            "imgUri": "http://writingexercises.co.uk/images2/randomimage/heavybox.jpg",
            "productUri": "http://localhost:3000/page-1"
          }
        },
        {
          "node": {
            "id": "09462d50-335e-56a0-ba1c-e555e0c4641a",
            "description": "desc1",
            "imgUri": "http://writingexercises.co.uk/images2/randomimage/heavybox.jpg",
            "productUri": "http://localhost:3000/page-2"
          }
        },
        {
          "node": {
            "id": "a75c341b-9ef5-53f2-8044-50594fbf7f10",
            "description": "desc1",
            "imgUri": "http://writingexercises.co.uk/images2/randomimage/heavybox.jpg",
            "productUri": "http://localhost:3000/page-3"
          }
        },
        {
          "node": {
            "id": "4cc361a7-6087-5cb1-85d7-c47eb31acae7",
            "description": "desc1",
            "imgUri": "http://writingexercises.co.uk/images2/randomimage/heavybox.jpg",
            "productUri": "http://localhost:3000/page-4"
          }
        },
        {
          "node": {
            "id": "a8a69652-2d05-5b2a-91b0-e350efdd7d24",
            "description": "desc1",
            "imgUri": "http://writingexercises.co.uk/images2/randomimage/heavybox.jpg",
            "productUri": "http://localhost:3000/page-3"
          }
        },
        {
          "node": {
            "id": "4d7a9de8-ed49-5d2c-99b9-ddb069eb82fd",
            "description": "desc1",
            "imgUri": "http://writingexercises.co.uk/images2/randomimage/heavybox.jpg",
            "productUri": "http://localhost:3000/page-6"
          }
        },
        {
          "node": {
            "id": "0dbbccf1-6161-5545-a618-1d8c2a02cffd",
            "description": "desc1",
            "imgUri": "http://writingexercises.co.uk/images2/randomimage/heavybox.jpg",
            "productUri": "http://localhost:3000/page-7"
          }
        },
        {
          "node": {
            "id": "ae2da04b-1a68-58ab-bb49-6effa01ec770",
            "description": "desc1",
            "imgUri": "http://writingexercises.co.uk/images2/randomimage/heavybox.jpg",
            "productUri": "http://localhost:3000/page-8"
          }
        },
        {
          "node": {
            "id": "46dbd8e5-5177-597a-b3c6-1fef1b489d4e",
            "description": "desc1",
            "imgUri": "http://writingexercises.co.uk/images2/randomimage/heavybox.jpg",
            "productUri": "http://localhost:3000/page-9"
          }
        },
        {
          "node": {
            "id": "39864cb9-ed1c-5f1a-ac3b-59a79d5881b8",
            "description": "desc1",
            "imgUri": "http://writingexercises.co.uk/images2/randomimage/heavybox.jpg",
            "productUri": "http://localhost:3000/page-10"
          }
        },
        {
          "node": {
            "id": "9bfc2384-9047-54a9-95b4-74498d27f487",
            "description": "desc1",
            "imgUri": "http://writingexercises.co.uk/images2/randomimage/heavybox.jpg",
            "productUri": "http://localhost:3000/page-11"
          }
        },
        {
          "node": {
            "id": "6424ff33-cf43-5c02-bcc2-1aceb7b80ab0",
            "description": "desc1",
            "imgUri": "http://writingexercises.co.uk/images2/randomimage/heavybox.jpg",
            "productUri": "http://localhost:3000/page-12"
          }
        },
        {
          "node": {
            "id": "5383c972-ddff-559b-b2cd-a7e7e143bab1",
            "description": "desc1",
            "imgUri": "http://writingexercises.co.uk/images2/randomimage/heavybox.jpg",
            "productUri": "http://localhost:3000/page-13"
          }
        },
        {
          "node": {
            "id": "fca74c2a-6d93-549b-9c5f-16761cec07a1",
            "description": "desc1",
            "imgUri": "http://writingexercises.co.uk/images2/randomimage/heavybox.jpg",
            "productUri": "http://localhost:3000/page-14"
          }
        },
        {
          "node": {
            "id": "60702145-c1fd-5307-ab6f-8c35be59dcd1",
            "description": "desc1",
            "imgUri": "http://writingexercises.co.uk/images2/randomimage/heavybox.jpg",
            "productUri": "http://localhost:3000/page-15"
          }
        },
        {
          "node": {
            "id": "0c0f422a-14fc-5726-843a-6e46ef8827fa",
            "description": "desc1",
            "imgUri": "http://writingexercises.co.uk/images2/randomimage/heavybox.jpg",
            "productUri": "http://localhost:3000/page-16"
          }
        },
        {
          "node": {
            "id": "407b543d-8cbd-59a2-ad82-5ad5c4a021b7",
            "description": "desc1",
            "imgUri": "http://writingexercises.co.uk/images2/randomimage/heavybox.jpg",
            "productUri": "http://localhost:3000/page-17"
          }
        },
        {
          "node": {
            "id": "dc580622-8626-5bf1-98f0-663d8dfe6306",
            "description": "desc1",
            "imgUri": "http://writingexercises.co.uk/images2/randomimage/heavybox.jpg",
            "productUri": "http://localhost:3000/page-18"
          }
        },
        {
          "node": {
            "id": "c1b45feb-520b-56f6-a89e-967d5e481a86",
            "description": "desc1",
            "imgUri": "http://writingexercises.co.uk/images2/randomimage/heavybox.jpg",
            "productUri": "http://localhost:3000/page-19"
          }
        },
        {
          "node": {
            "id": "70406ecb-287b-533c-97bf-b9e345cccaab",
            "description": "desc1",
            "imgUri": "http://writingexercises.co.uk/images2/randomimage/heavybox.jpg",
            "productUri": "http://localhost:3000/page-20"
          }
        },
        {
          "node": {
            "id": "aad267a6-6486-5961-9e66-1514ce10d68d",
            "description": "desc1",
            "imgUri": "http://writingexercises.co.uk/images2/randomimage/heavybox.jpg",
            "productUri": "http://localhost:3000/page-21"
          }
        },
        {
          "node": {
            "id": "62bd0348-5d65-5f27-8fab-768baa887783",
            "description": "desc1",
            "imgUri": "http://writingexercises.co.uk/images2/randomimage/heavybox.jpg",
            "productUri": "http://localhost:3000/page-22"
          }
        }
      ]
    }
  }
}

The only difference in here is that the ids returned from calling the server directly are the ones that i've created and by the graphql query ones are uuids generated by gatsby. I'm more than happy to post the code on github repos if you want. But so far and based on the information at hand everything is working as it should. And to take it a step further i changed the graphql query to the following

{
  allProductApi(limit: 1000) {
    edges {
      node {
        id
        description
        imgUri
        productUri
      }
    }
  }
}

And the same result as the one above. Sorry but i could not reproduce your issue.

jonniebigodes picture jonniebigodes on 21 Dec 2018
❤1

@jonniebigodes Thank you so much. Your setup look close to what i have.
That's alot of work to reproduce this issue (salute). Thank you so much, really appreciate it.

Your comment about 'id' could be the culprit, as my sample data might contains duplicated 'id'.

appcoreopc picture appcoreopc on 21 Dec 2018

@appcoreopc no need to thank, it was no problem at all. I'm glad to help out as much as i can. And also that i could help you pinpoint on the probable cause of your issue. Check if the culprit could be some duplicate ids and report back with some feedback.

jonniebigodes picture jonniebigodes on 21 Dec 2018

hi jonnie, yeap the duplicated id, is the reason why i am not getting rest of the records. thank you! :+1:

appcoreopc picture appcoreopc on 21 Dec 2018
🎉1

Thank you so much, @jonniebigodes 鉂わ笍

@appcoreopc Closing this for now, please feel free to comment if there is anything else

sidharthachatterjee picture sidharthachatterjee on 21 Dec 2018

@appcoreopc once again no need to thank, glad that you've figured it out. Thanks for using Gatsby 馃憤

jonniebigodes picture jonniebigodes on 21 Dec 2018
Was this page helpful?
0 / 5 - 0 ratings

Related issues

EdinK1 picture EdinK1  路  69Comments
Jivings picture Jivings  路  112Comments
kripod picture kripod  路  74Comments
cusspvz picture cusspvz  路  128Comments
jp887 picture jp887  路  98Comments