Apollo-client: fetchMore's updateQuery option receives the wrong variables

[jamesreggio]

The updateQuery method passed to ObservableQuery.fetchMore is receiving the original query variables instead of the new variables that it used to fetch more data.

This is problematic not so much because it breaks the updateQuery method contract (which is bad, but clearly doesn't seem to impact many people), but rather because when the results from fetchMore are written to the store in writeQuery, the old variables are used. This cases issues, say, if you were using an @include directive and the variable driving its value changes.

Repro is available here: https://github.com/jamesreggio/react-apollo-error-template/tree/repro-2499

Expected behavior

nextResults.variables shows petsIncluded: true and the pets list gets displayed.

Actual behavior

nextResults.variables shows petsIncluded: null and the pets list is not displayed because the write never happens due to the @include directive being evaluated with the old variables.

Version

• [email protected]

[jamesreggio]

Uploaded repro here: https://github.com/jamesreggio/react-apollo-error-template/tree/repro-2499

[jamesreggio]

So, my hack above doesn't actually resolve the problem :-/

To be clear, there are two — and possibly three — related issues at play here:

1. nextResults has the wrong variables — easy to fix.
2. The value returned by the updateQuery function is written to the cache with old variables. Perhaps this is okay, and perhaps it's not. I'm worried that if the new variables are used and are different than the original variables, the original observer will not 'see' the updated query data.
3. If it's determined that the original variables ought to be used during writeQuery, it then seems impossible to force the inclusion of fields that were originally excluded via a variable-driven directive.

[jbaxleyiii]

@jamesreggio as always, thank you for the thoughtful research and repo! I'll get on it ASAP!

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions to Apollo Client!

[jamesreggio]

Here's a GIF to keep this open.

[leoasis]

I think I've run into this same issue while trying to do pagination. I'm trying to do

fetchMore({
  variables: {
    page: variables.page + 1
  }
}, ...);

where variables come from data.variables but those never seem to get updated, they are always the initial variables. This causes the code to always attempt to fetch the second page.

[padupuy]

Any update ?
I use fetchMore function to create an infinite scroll with the FlatList Component (React Native).
As @jamesreggio said, the variables seems not be updated and my infinite scroll turns into an infinite loop of the same query result.

I also notice that queryVariables is null inside the updateQuery callback.

List of the packages I use :

• "apollo-cache-inmemory": "1.1.5"
• "apollo-cache-persist": "0.1.1"
• "apollo-client": "2.2.0"
• "apollo-link": "1.0.7"
• "apollo-link-error": "1.0.3"
• "apollo-link-http": "1.3.2"
• "apollo-link-state": "0.3.1"
• "graphql": "0.12.3"
• "graphql-tag": "2.6.1"
• "react": "16.0.0"
• "react-apollo": "2.0.4"
• "react-native": "0.51.0"

const { data } = this.props;

<FlatList
      data={data.feed}
      refreshing={data.networkStatus === 4}
      onRefresh={() => data.refetch()}
      onEndReachedThreshold={0.5}
      onEndReached={() => {
        // The fetchMore method is used to load new data and add it
        // to the original query we used to populate the list
        data.fetchMore({
          variables: { offset: data.feed.length + 1, limit: 20 },
          updateQuery: (previousResult, { fetchMoreResult,  queryVariables}) => {
            // Don't do anything if there weren't any new items
            if (!fetchMoreResult || fetchMoreResult.feed.length === 0) {
              return previousResult;
            }

            return {
              // Concatenate the new feed results after the old ones
              feed: previousResult.feed.concat(fetchMoreResult.feed),
            };
          },
        });
      }}
    />

[pleunv]

Encountering the same issue here. refetch({ ...variables }) updates the initial variables, fetchMore({ variables }) does not. If you are not getting the page index/cursor information in your response it's a bit difficult to get/keep a reference to this now (still trying to find a good approach). Is this intended behavior, and would there be an advisable workaround?

[pleunv]

In case this might interest someone, I'm currently working around this by passing the necessary paging parameters combined with an updater function from the child component calling the fetchMore, something like this:

const withQuery = graphql(MY_LIST_QUERY, {
  options: ({ pageSize, searchText }) => ({
    variables: {
      pageIndex: 1,
      pageSize,
      searchText: searchText
    }
  }),
  props: ({ data }) => ({
    data: {
      ...data,
      fetchMore: ({ pageIndex, updatePageIndexFn }) => {
        const nextPageIndex = pageIndex + 1;
        updatePageIndexFn(nextPageIndex);

        return data.fetchMore({
          variables: {
            pageIndex: nextPageIndex
          },
          updateQuery: (previousResult, { fetchMoreResult }) => ({
            items: [
              ...previousResult.items,
              ...fetchMoreResult.items
            ]
          })
        });
      }
    }
});

class ListComponent extends React.Component {
  constructor(props) {
    super(props);
    this.pageIndex = props.data.variables.pageIndex;
  }

  handleLoadMore = event => {
    event.preventDefault();
    this.props.data.fetchMore({
      pageIndex: this.pageIndex,
      updatePageIndexFn: index => this.pageIndex = index
    });
  }

  render() {
    return (
      <React.Fragment>
        <ul>{this.props.items.map(item => <li>{JSON.stringify(item, null, 2)}</li>)}</ul>
        <a href="" onClick={this.handleLoadMore}>Load More</a>
      </React.Fragment>
    );
  }
}

const ListComponentWithQuery = withQuery(ListComponent);

edit: What I initially assumed would work is this:

const withQuery = graphql(MY_LIST_QUERY, {
  options: ({ pageSize, searchText }) => ({
    variables: {
      pageIndex: 1,
      pageSize,
      searchText: searchText
    }
  }),
  props: ({ data }) => ({
    data: {
      ...data,
      fetchMore: () => data.fetchMore({
        variables: {
          pageIndex: data.variables.pageIndex + 1
        },
        updateQuery: (previousResult, { fetchMoreResult }) => ({
          items: [
            ...previousResult.items,
            ...fetchMoreResult.items
          ]
        })
      })
    }
});

[dehypnosis]

Is this intended?
data.variables not updated on fetchMore called with new variables

[ZiXYu]

still got this issue.. hard to do pagination with refetch now

[mefjuu]

Any chances to fix that 5-months old issue? It seems to be quite significant disfunction. A bit funny considering an activity of Apollo team on various React confs and theirs effort to promote Apollo integrations...

[stefanoTron]

same here, I'm using [email protected]

[Billy-]

@jbaxleyiii any update on this at all please?

Kind regards,

[BipinBhandari]

I am amazed how such a critical issue on Apollo is not being addressed! How are current apollo users handling cursor based pagination? I am having hard time coming with a hack to solve this issue.

Can anyone suggest a way to get cursor based pagination working, at least for now?

[himerus]

I'd love to see some help/movement on this issue as well.

Using latest (2.3.1) apollo-client.

Attempting to use the new component, and implement similar to mentioned above. I've been able to get the single 'load more' button to work per the examples on the 'pagination' page, and get those results to either add to the end of the previous result set (infinite scroll style) or replacing the previous result set (Next page style).

However, as soon as I try to implement numbered pages, and try passing in a page number to do calculations with to configure the actual offset, i continually get ObservableQuery with this id doesn't exist errors.

[himerus]

@ZiXYu did your comment intend to say you had to do pagination with refetch because if this issue? Instead of hard?

I was considering it as a path to take, but hadn't seen much in the way of documentation on it.

[abhiaiyer91]

I wrote a failing test case in the tests for fetchMore here: https://github.com/apollographql/apollo-client/pull/3500

[abhiaiyer91]

I just wanna confirm @hwillson that the expected behavior from this issue is correct, or the way it works is intended. This doesnt look like that is clear to everyone experiencing the issue.

[abhiaiyer91]

Also heres the fix, test pass and nextResult.variables is correct https://github.com/apollographql/apollo-client/pull/3500

[hwillson]

Hi all - https://github.com/apollographql/apollo-client/pull/3500 has been confirmed to fix this issue, and has just been merged. I'll be pushing a new apollo-client release later today (that will include this fix). Thanks for reporting this issue @jamesreggio, and thanks for the fix @abhiaiyer91!

[YannisMarios]

Hi,

I still have this issue with [email protected]. Is this fixed?

[hwillson]

@YannisMarios Yes, this should be fixed in 2.3.2. If you're still encountering this, could you put together a small runnable reproduction that shows this happening? Thanks!

[YannisMarios]

I really don't have time to put together a repro. Perhaps others can confirm it is not working.

[YannisMarios]

Hi this my code:

const options = {
    name: 'getMessages',
    options: (props) => {
        let after = props.endCursor || '';
        return {
            variables: {
                first: MESSAGES_PAGE_SIZE,
                after: after
            }
        }
    },
    force: true,
    props: ({
        ownProps,
        getMessages
    }) => {
        const {
            loading,
            messages,
            fetchMore
        } = getMessages;
        if (messages && messages.pageInfo) {
            console.log(messages.pageInfo.endCursor, 'before loadMoreRows()');
        }
        const loadMoreRows = () => {
            console.log(messages.pageInfo.endCursor, 'inside')
            return fetchMore({
                variables: {
                    first: MESSAGES_PAGE_SIZE,
                    after: endCursor
                },
                updateQuery: (previousResult, {
                    fetchMoreResult
                }) => {
                    return {
                        messages: {
                            totalCount: fetchMoreResult.messages.totalCount,
                            edges: [...previousResult.messages.edges, ...fetchMoreResult.messages.edges],
                            pageInfo: fetchMoreResult.messages.pageInfo,
                            __typename: 'MessageList'
                        }
                    };
                }
            })
        }
        return {
            loading,
            getMessages,
            loadMoreRows
        }
    }
}


export default compose(
    // withSearch({searchColumns}),
    graphql(getMessages, options)
)(MessageList);

MESSAGE_PAGE_SIZE is 20.

Upon initial load of the page: MTE3NQ== before loadMoreRows()

On first scroll:

MTE3NQ== inside loadMoreRows()
MTE5NQ== before loadMoreRows() // this is printed after the above.

I temporary fixed this issue with this by defining variable endCursor outside options, setting its value before loadMoreRows(). and now my infinite list works correctly again. I don't like this solution.

let endCursor;

const options = {
    name: 'getMessages',
    options: (props) => {
        let after = props.endCursor || '';
        return {
            variables: {
                first: MESSAGES_PAGE_SIZE,
                after: after
            }
        }
    },
    force: true,
    props: ({
        ownProps,
        getMessages
    }) => {
        const {
            loading,
            messages,
            fetchMore
        } = getMessages;
        if (messages && messages.pageInfo) {
            endCursor = messages.pageInfo.endCursor;
        }
        const loadMoreRows = () => {
            return fetchMore({
                variables: {
                    first: MESSAGES_PAGE_SIZE,
                    after: endCursor
                },
                updateQuery: (previousResult, {
                    fetchMoreResult
                }) => {
                    return {
                        messages: {
                            totalCount: fetchMoreResult.messages.totalCount,
                            edges: [...previousResult.messages.edges, ...fetchMoreResult.messages.edges],
                            pageInfo: fetchMoreResult.messages.pageInfo,
                            __typename: 'MessageList'
                        }
                    };
                }
            })
        }
        return {
            loading,
            getMessages,
            loadMoreRows
        }
    }
}

FYI records are fetched from database with mongoose's cursor:

const inifinteList = (first, after) => {
    let edgesArray = [],
    let cursorNumeric = parseInt(Buffer.from(after, 'base64').toString('ascii'));
    if (!cursorNumeric) cursorNumeric = 0;

    let edgesAndPageInfoPromise = new Promise((resolve, reject) => {
        let stream = model.where('index').gt(cursorNumeric).find({}, (err, result) => {
            if (err) {
                console.error("---Error " + err);
            }
        }).limit(first).cursor();

        stream.on('data', res => {
            edgesArray.push({
                cursor: Buffer.from((res.index).toString()).toString('base64'),
                node: res
            });
        });

        stream.on('error', error => {
            console.log(error);
            reject(error);
        });

        stream.on('end', () => {
            let endCursor = edgesArray.length > 0 ? edgesArray[edgesArray.length - 1].cursor : NaN;
            let hasNextPageFlag = new Promise((resolve, reject) => {
                if (endCursor) {
                    let endCursorNumeric = parseInt(Buffer.from(endCursor, 'base64').toString('ascii'));
                    model.where('index').gt(endCursorNumeric).count((err, count) => {
                        count > 0 ? resolve(true) : resolve(false);
                    });
                } else {
                    resolve(false);
                }
            });
            resolve({
                edges: edgesArray,
                pageInfo: {
                    endCursor: endCursor,
                    hasNextPage: hasNextPageFlag
                }
            });
        });
    });

    let totalCountPromise = new Promise((resolve, reject) => {
        if (totalCount === 0) {
            totalCount = model.count((err, count) => {
                if (err) reject(err);
                resolve(count);
            });
        } else resolve(totalCount);
    });

    let returnValue = Promise.all([edgesAndPageInfoPromise, totalCountPromise]).then((values) => {
        return {
            edges: values[0].edges,
            totalCount: values[1],
            pageInfo: {
                endCursor: values[0].pageInfo.endCursor,
                hasNextPage: values[0].pageInfo.hasNextPage
            }
        };
    });

    return returnValue;
};

[geminiyellow]

@hwillson same error, please fix it . 🙇

and "apollo-client": "^2.3.2",
and code is:

import React, { Component } from 'react';
import { Query } from 'react-apollo';
import { Button, Text, View } from 'react-native';
import { QUERY_GROUP_SUMMARY } from '../graphql/group.resolve';

const handleFetchMore = (fetchMore, { account_id, date, page, path }) => {
  fetchMore({
    variables: { account_id, date, page: page + 1, path },
    updateQuery: (previousResult, { fetchMoreResult }) => {
      const { GroupSummary: { groups: newGroup, menu_items: newMenu, remain } } = fetchMoreResult;
      const { GroupSummary: { groups: oldGroup, menu_items: oldMenu } } = previousResult;
      return {
        GroupSummary: {
          remain,
          groups: [...oldGroup, ...newGroup],
          menu_items: [...oldMenu, ...newMenu],
        },
      };
    },
  });
};

class GroupList extends Component {
  render() {
    const { date, account_id } = this.props;
    return (
      <Query query={QUERY_GROUP_SUMMARY} variables={{ account_id, date, page: 0, path: '' }}>
        {
          ({ loading, error, data, fetchMore, variables }) => {
            if (loading) return "Loading...";
            if (error) return `Error! ${error.message}`;

            const { GroupSummary: { groups, menu_items, remain } } = data;
            const { page } = variables;
            return (
              <View>
                <Text>{`${page} PAGE`}</Text>
                <Text>{`${groups.length} GROUP`}</Text>
                <Text>{`${menu_items.length} MENU`}</Text>
                {remain && <Button title={'Load More'} onPress={() => handleFetchMore(fetchMore, variables)} />}
              </View>
            );
          }
        }
      </Query>
    );
  }
}

export default GroupList;

[mortenko]

I have the same issue also after upgrading to apollo-client@^2.3.2

[JeffersonFilho]

This issue still happening on "apollo-client": "^2.3.7"

[gcangussu]

Same issue on apollo-client": "^2.4.2". Happens even if I set fetchPolicy to no-cache.

edit: There was no problem with Apollo, the bug happened because we were mutating an object that was passed by props. So it was the same object, mutated, but still the same reference.

[amannn]

I'm seeing the same issue when I invoke a fetchMore call and change the route so the component that has invoked the call unmounts. I've seen this across various apps that use react-apollo.

You can also see it here in action:

1. Go to https://spectrum.chat/react
2. Scroll to the bottom
3. Before the next page arrives, click on the support button in the main navigation
4. The error appears in the console

spectrum.chat runs on [email protected].

[ebenoist]

Same here on 2.4.3 -- I'll try and work around this for now.

[gcangussu]

I've just updated my old comment, there was no problem with the Apollo library in my case. After hours of debugging I've discovered that one object was being mutated and expected to trigger a different render. Off course it did not, Apollo thought it was the same object as before.

[myhendry]

Using HOC, it seems to work. The offset variable can be changed with fetchMore. the limit variable i passed in through HOC. https://stackoverflow.com/questions/53440377/apollo-graphql-pagination-failed-with-limit However, using rendered props, i face the same issue whereby the offset variable cannot be changed (and is stuck to the initial offset variable) using fetchmore https://stackoverflow.com/questions/53446467/offset-variable-does-not-update-with-apollo-fetchmore it would be excellent if there is a fix to allow the rendered props method to work correctly. After all, i understand rendered props is the preferred approach going forward. Thank you

[mnlbox]

@hwillson It's still not working for me.
Any plan to fix this?

[FunkSoulNinja]

I had this problem and found a workaround for the issue.
To give you some context: I was using React and had a button to fetch more posts inside a Query component using cursor-based pagination.

Instead of calling the fetchMore function received from the Query component, I would:
1) Let my component do it's job with my initial variables or whatever.
2) Directly call the query method on the client object with different variables and save the response onto a variable named "res":

const res = await client.query({ query: getPosts, variables: { ...differentVariables })

3) Use client.readQuery to read from the cache and save it to a variable:

const prev = client.readQuery({ query: getPosts, variables: { ...myVariables })

4) Lastly, combine the items and write it directly to the cache like so:

const updatedData = { ...prev, posts: [...prev.posts, ...res.data.posts] } client.writeQuery({ query: getPosts, variables: { ...myVariables }, data: updatedData })

This worked flawlessly for me.

[JClackett]

Still having a similar issue:

  const nextPage = (skip: number, currentSearch = "") =>
    fetchMore({
      variables: { houseId, skip, search: currentSearch },
      updateQuery: (prev, { fetchMoreResult }) => {
        if (!fetchMoreResult || !prev.allCosts || !fetchMoreResult.allCosts)
          return prev
        return Object.assign({}, prev, {
          allCosts: {
            ...prev.allCosts,
            costs: [...prev.allCosts.costs, ...fetchMoreResult.allCosts.costs],
          },
        })
      },
    })

I call this function when reaching the bottom of the page.

It works perfectly when the search variable remains as "", when reaching the bottom the skip variable changes and new data is refetched and works great.

However when i change the search variable and refetch the original query, and then try and paginate, i get an "ObservableQuery with this id doesn't exist: 5" error, I had a look in the source code/added some logs and it seems that apollo is using the old queryId (from the query where the search variable was "") for the fetchMore. I can see the data being successfully fetched in the network with all the correct variables but saving it back to the cache seems to be where it's erroring, am i missing something with this implementation?

Note: I am using react-apollo-hooks, which i know isn't production ready or even provided by apollo-client right now, but from what i've seen it looks like its something to do with the fetchMore API

Note: @FunkSoulNinja solution works for me, however would be nice to be able to use the provided API for this kind of feature

[MarttiR]

This is still an issue on 2.4.13. Could @hwillson consider reopening this?

The workaround by @FunkSoulNinja above works, but becomes convoluted – here's a more complete example of how it goes together:

import React from "react";
import { Query, withApollo } from "react-apollo";
import gql from "graphql-tag";

const QUERY = gql`
  query someQuery(
    $limit: Int
    $offset: Int
  ) {
    someField(
      limit: $limit
      offset: $offset
    ) {
      totalItems
      searchResults {
        ... on Something {
          field
        }
        ... on SomethingElse {
          otherField
        }
      }
    }
  }
`;

const SearchPage = props => {
  const { client } = props;

  const defaultVariables = {
    limit: 10,
    offset: 0,
  };

  return (
    <Query query={QUERY} variables={defaultVariables}>
      {({ loading, error, data, updateQuery }) => {
        const { someField } = data;
        const { searchResults, totalItems } = someField;
        return (
          <>
            <ResultList results={searchResults} total={totalItems} />
            <Button
              onClick={async () => {
                const nextVariables = Object.assign({}, defaultVariables, {
                  offset: searchResults.length,
                });
                const prevVariables = Object.assign({}, defaultVariables, {
                  offset: searchResults.length - defaultVariables.limit,
                });
                const [next, prev] = await Promise.all([
                  client.query({
                    query: QUERY,
                    variables: nextVariables,
                  }),
                  client.query({
                    query: QUERY,
                    variables: prevVariables,
                  }),
                ]);
                const updatedData = {
                  ...prev.data,
                  someField: {
                    ...prev.data.someField,
                    searchResults: [
                      ...prev.data.someField.searchResults,
                      ...next.data.someField.searchResults,
                    ],
                  },
                };
                updateQuery(
                  (_prev, { variables: prevVariables }) => updatedData
                );
                client.writeQuery({
                  query: QUERY,
                  variables: nextVariables,
                  data: updatedData,
                });
              }}
            >
              Load more
            </Button>
          </>
        );
      }}
    </Query>
  );
};

export default withApollo(SearchPage);

[FunkSoulNinja]

@MarttiR You could also use the client object from the Query component render prop function without having to use the withApollo HOC.

[kandasj]

@FunkSoulNinja can you post the full solution please. I'm having the same issue. struggling to put together your solution

[yumemi-omi]

Still have the same issue(2.6.4)

[RyotaBannai]

There is a workaround for this... or maybe the solution

1. set fetchPolicy as default
2. add @connection directive to the field in your query so that apollo can identify which to concat...

like below...

const GET_ITEMS = gql`
  query GetItems($skip: Int, $take: Int, $current: Int) {
    getItems(skip: $skip, take: $take, current: $current) @connection(key: "items") {
      id
      data
      type
      list {
        id
        name
      }
    }
  }
`;

let fetch_options = { skip: 0, take: 2, current: 0 };
export const Pagination: React.FC<Props> = () => {
  const { called, loading, data, fetchMore } = useQuery(GET_ITEMS, {
    variables: fetch_options,
    // fetchPolicy: "cache-and-network", //   Do not set 
  });
  if (called && loading) return <p>Loading ...</p>;
  if (!called) {
    return <div>Press button to fetch next chunk</div>;
  }
  return (
    <div>
      <Button
        onClick={(e: any) => {
          fetch_options = {
            ...fetch_options,
            current: fetch_options.current + 1,
          };
          fetchMore({
            variables: fetch_options,
            updateQuery: (
              previousQueryResult,
              { fetchMoreResult, variables }
            ) => {
              if (!fetchMoreResult) return previousQueryResult;
              return {
                getItems: previousQueryResult.getItems.concat(
                  fetchMoreResult.getItems
                ),
              };
            },
          });
        }}
      >
        Fetch Next
      </Button>
  );
};

[mnguyen96]

@RyotaBannai That didnt work for me :/