Graphql-js: Graphql support for sort and filter functionalities

[sudheerj]

Graphql provides an excellent standard for queries and mutations.Is there any support for sorting and filtering(multiple fields) for Graphql queries.If not is there any way to achieve with Graphql + mongoose schema.

[sudheerj]

We can pass sort fields as simple input variables.But filtering with complex conditions(set of AND,OR,NOT) it will be problematic.

for example,
"filters": {
"region": "APAC",
"code": {
"$in": [ "HK1", "SG1" ]
},
"account_number": {
"$nin": [ "1" ]
}

[smolinari]

This might be interesting for you.

https://github.com/graphql/graphql-js/issues/585

Scott

[sudheerj]

Thanks.But it is not implemented in GraphQL directly.Is there any way to achieve it?

[smolinari]

Did you read this response?

https://github.com/graphql/graphql-js/issues/585#issuecomment-262402544

Scott

[sudheerj]

Yes I read the response as why it won't support.I'am not sure is there someway to integrate with sequelizejs operators with Graphql. Also Iam looking for a workaround how we can pass entire filter conditions as JSON input string.

[smolinari]

Byron didn't say GraphQL doesn't support what you want, he said it doesn't support AND and OR operators (as in the sense of how they are used in SQL).

What he said in the last sentence of his comment is the key to you finding your answer.

Typically we use lists of values to represent multiple filters or multiple selections.

In other words, you can include any form of arguments to your queries to support the server (the resolver), so it can return the data the client needs and in the form it needs it.

https://facebook.github.io/graphql/#sec-Object-Field-Arguments
http://graphql.org/learn/queries/#arguments

With no disrespect intended, I'd venture to say you haven't yet made the "mental model shift" that Byron mentioned in his post. Once you've made it, all of this will make a lot more sense. :smile:

Scott

[sudheerj]

Thanks for the detailed response.But passing the list of values as object input field values will be difficult for complex queries.And also we should parse the JSON in mongoose schema resolver etc.I need to support around 20+ operators to cover all the request scenarios.

For example,

query RootQuery{
               accounts (["region": "APAC","code": {"$in": [ "HK1", "SG1" ]},"account_number": {"$nin": [ "1" ]}){
                  account_name
                }
      }

[smolinari]

What UI component are you trying to feed data to? Is it by chance a search component of some kind?

If it isn't a search component, does that component really need ALL 20+ operators to fetch the data it needs? Remember, GraphQL is UI component based. That is part of the "mental model shift".

Scott

[sudheerj]

We are applying complex search criteria on DataTable UI components.We may not get all 20+ operators or combinations but a maximum possibilities only.Also each JSON request need to be converted as Graphql query format.

[smolinari]

You mean for something like this? https://datatables.net/manual/data/

Are you pulling in more than 10K rows / documents?

Scott

[sudheerj]

Yes you are right.But the number of records won't be huge.

[smolinari]

No matter what, the only available possibility offered to you by GraphQL to filter and order the data on the server is to insert the filtering and ordering information through query arguments.

Scott

[leebyron]

GraphQL itself does not have specific syntax dedicated to describing "sort" or "filter" semantics since that would require the underlying representation make such things available, even at the cost of performance.

Instead, most API which provide sorting or filtering do so via Field Arguments. A field which returns values that could be sorted or filtered accept arguments describing how the client would like them.

I've seen AND/OR/NOT operators described via wrapping structures before, much like the example you described at the top of your question.

Also remember that you should use Variables when your query will be changing per request, that will avoid converting JSON descriptions of your filters and sorts into any kind of query syntax.

For example:

query GetAccounts($filter: AccountsFilter) {
  accounts(filter: $filter) {
    account_name
  }
}

And then sent with variables:

{
  "filter": {
    "region": "APAC",
    "code": {"in": [ "HK1", "SG1" ]},
    "account_number": {"nin": [ "1" ]}
  }
}

[sudheerj]

Thanks Smolinari and Leebyron for your comments.Finally the only way I can see is that passing JSON variable as dynamic arguments.

Now I need to figure it out how to deal with mongoose on subdocument queries.Because

Parent.child

access not able to fetch records.

for example,
"ibx_presence.ibx_code": { "$in": [ "SV1", "CH1" ] }

[sudheerj]

I'am not able to fetch IBX details because it is created as separate collection/table in mongoose.So the details won't be available in parent collection directly.I will try to make it as single collection otherwise it is going to be complex.

[sudheerj]

Hi leebyron,
Can you please let us know how to define type AccountsFilter for JSON query variable and where we need to place it in GraphIql IDE for the execution.Thanks in advance.

[leebyron]

In graphiql you'll see a "variables" editor in the bottom left. It usually defaults to closed, you can click it to open it.

[sudheerj]

Thanks Leebyron.That is for JSON variable declaration inside query variable section.I think we should define "AccountsFilter " Type inside schema file.Is it right?

For example in this case,
type AccountFilter{ region: String, code : String, account_number: String }

[leebyron]

You probably want input types, but otherwise that looks right to me

[sudheerj]

I tried to apply JSON string as variable for the but it fails with an error as below

The schema and model are defined as follows

```
let RootQuery = new GraphQLObjectType({
name: 'Query', //Return this type of object

``` fields: () => ({
accounts: AccountQueries.accounts,
ibx: IBXQueries.IBXCode,
ibxes: IBXQueries.ibxes
})
});

export default {
accounts: {
type: new GraphQLList(AccountType),
args: {
skip: {
type: GraphQLInt
},
limit:{
type: GraphQLInt
},
sort_field: {
type: GraphQLString
},
sort_order:{
type: GraphQLString
},
filter:{
type: GraphQLString
}
},
resolve: Account.getListOfAccounts
}

[sudheerj]

Hi Leebyron,

If I want to define custom input type(ex,AccountsFilter) then how and where I need to define the type.I got an error if I defined String type for JSON query variable.This issue is the only pending part in my POC.Thanks in advance for the help.

[leebyron]

Can you clarify what error you're getting? I don't have enough information to help you. The error in your screenshot says that you must supply an operation, that seems not relevant to your problem.

I recommend checking out some articles on graphql.org and answers on stack overflow for more help. I'm sure you can solve this

[sudheerj]

I just want to send below JSON object as query variable in GraphIql IDE.

{
  "filter": {
    "region": "APAC",
    "code": {"in": [ "HK1", "SG1" ]},
    "account_number": {"nin": [ "1" ]}
  }
}

1. Can we pass it as String input type.But it fails as in the snapshot.
   2.Can we pass it as custom input type(for example, AccountsFilter).But I'am not sure where to define this type in Graphiql IDE.

I tried to find articles/SO about JSON object as dynamic input variables with Graphiql usage. But no luck.

[leebyron]

You can use JSON variables like this if you expect a variable of an input type. That input type needs to be defined in your GraphQL server, not as part of the query.

You can define your variable as a String type if you wish, but you'll need to supply the variable value as a String as well, not as a JSON object

[leebyron]

Check out GraphQLInputType for defining these on the server with graphql-js

[sudheerj]

The above JSON expression is a direct mongoose expression so if we can pass entire JSON string then it will be easy to execute on DB schema.
Account.find({filter}).exec

It will contain the operators such as $in,$nin,$lt,$gt etc as well.The expression sometimes may contain upto 10 fields as well.

I didn't really find resources on how to use GraphQLInputType in this case.This kind of objects may not suite and didn't workout

new GraphQLInputObjectType({
  name: 'CoordsInput',
  fields: {
    region: { type: GraphQLString }
  }
});

The below two links talk exactly about my requirement
https://forums.meteor.com/t/operators-in-graphql/29368
https://github.com/graphql/graphql-js/issues/290

[leebyron]

You can use strings if you like, just ensure you quote the variable values as well.

[sudheerj]

Hi leebyron ,
In Graphiql entire value in single quote is not expected and shown as error

{
"filter": ' {
"region": "APAC",
"code": {"$in": [ "HK1", "SG1" ]},
"account_number": {"$nin": [ "1" ]}
}'
}

If I sent it as normal JSON input but as String type then I got an error as "Must provide an operation"

Thank you so much for your guidance.

[sudheerj]

Hi leebyron ,
Can you please mention how the above JSON object treated as valid string for GraphiQL query variable.

Best regards,

[josephsavona]

@sudheerj a couple things: first, it looks like in your graphiql screenshot the query is missing a name, which is causing the error. Second, your query has $filter: String, but it should be typed as the type of the "filter" argument - presumably some InputObject that you defined.

[sudheerj]

1.Sorry the query name is missed in the screenshot.Even with the name I got the same error

1. I want to pass entire JSON value as String because its a Mongoose scheema which is going to be executed directly on the DB.How we can pass it as a String value?

Single quote didn't workout as below

{
"filter": ' {
"region": "APAC",
"code": {"in": [ "HK1", "SG1" ]},
"account_number": {"nin": [ "1" ]}
}'
}

[robrichard]

@sudheerj do you have '&operationName=null' in the URL? I've seen that happen sometimes in graphiql and I get the same error, but I haven't been able to reliably reproduce.

[sudheerj]

I didn't find any operationName in the URL.But I can get this error.

[jordantogether]

I think I stand on the same side as @smolinari. I thought it was a glaring omission for GraphQL, but with it supporting data from multiple sources, it's bound to be extremely slow (a la oData for CRM) if it starts doing that filtering logic after it's received data from the various sources.

In my case, the solution was simple: the parameters that go through to the GET query (i.e. my_url.com/query?here=come&dat=boi) which are immediately passed through to the data source in GraphQL, which can perform some of the more complicated logic in the back-end and provide the appropriate results.

But then, because there are multiple uses of that data in multiple scenarios on the front end, GraphQL is perfect for simply querying only a few fields out of those results and returning them.

I totally agree with the "mental model" issue - because if you're wanting GraphQL to do this, chances are, you're looking for GraphQL to be a database, rather than a query language for multiple data sources. IMHO.