Graphql-js: Operators with GraphQL

[helixether]

I've recently dove into GraphQL and am loving it so far. Being consistent with why GraphQL was made and what it is, it obviously does not supports as it is not a data store. This includes and/ors.

However, I have been researching online and was wondering if there is a GraphQL-recommended way of implementing operators. If not, should there be?

I've seen people take a few different approaches, and our team has thought of some of their own, and here are two of our most popular (internally) ideas. I'd love to get feedback on them.

Goal: Allowing developers to be able to specify all operators commonly used and keep consistent/original with GraphQL as much as possible.

1. Allow input to be dynamic, either the original type (numbers for number, strings for strings, etc,.) or JSON.
2. PROS: Developers can still query for fields naturally most of the time, in the native GraphQL way. It is only when they have to do custom queries that they'll have to write JSON.
3. CONS: Input type isn't consistent can provide problems. Fact that we're accepting JSON makes writing the query lengthy and it goes a bit against GraphQL's clean, simple and natural query syntax.

Example:

{
  // By default an array is treated like an 'or'
  // By default no operator$ defined means an 'and'
  case(caseStatus:"[closed, open]", caseNumber:"2012") {
    caseNumber
  }
}
// Customized
{
  // The caseStatus field will be translated into an or where caseStatus can be any of the ones defined in 'val'
  // The operator$ field at the end makes this entire evaluation an or as well
  // It translates to
  // Select * From sys_case Where (caseStatus IN ['closed', 'open']) OR (caseNumber = '2012')
  case(caseStatus:"{ val:"[closed, open]", operator$:"or"}", caseNumber:"2012", operator$:"or") {
    caseNumber
  }
}

1. Have a unique args or q object that developers can pass in if they need to do something other than a plain equals. This would be JSON.
2. PROS: Fully supports all operators and respects composite conditions ((eval && eval2) || (eval3)
3. CONS: Once more, JSON diverges from the clean GraphQL syntax in a way that it makes writing the query complex. And this makes writing the query complex for even simple things, such as field x = 'y' && field z = 's'

Example:

var conditionOne = {
    properties: {
        caseNumber: '2',
        caseTitle: '3'
    }
};
var conditionTwo = {
    properties: {
        caseCategory: '4',
        caseCategory2: '5',
        operator: 'or'
    }
};
var conditionThree = {
    properties: {
        caseNumber: 'someVal',
        caseStatus: 'someOtherVal',
        operator: 'or'
    },
    operator: 'or'
};
case(args: [conditionOne, conditionTwo, conditionThree]) {
  caseNumber
}

// Translates to
// Select *
// From sys_case
// Where (case_number='2' AND case_title='3') 
// AND (case_category='4' OR case_category_2='5')
// OR (case_number='someVal' OR case_status='someOtherVal')

In conclusion, the feeling I get is going to JSON is the right route to go, even though it might be unfortunate. Example two seems to be the simplest, and even it is quite a bit of code. Maybe a mix of having support for simple filtering (caseStatus:'something') (caseStatus:'!something') can be done and then for more advanced operators a JSON override in a q or args can be specified.

I'd love to hear what the community thinks.

[smolinari]

I am also new to GraphQL and not sure at all if this might help, but I believe this is where directives might come in handy.

Server implementations may also add experimental features by defining completely new directives.

http://graphql.org/learn/queries/#directives
https://medium.com/the-graphqlhub/graphql-tour-directives-558dee4fa903

Scott

[asiandrummer]

@helixether - let me know if I'm misunderstanding anything, but I feel like you could structure your types/queries so that queries could represent the operators you want to implement. Arguments are usually more generic than setting the conditions for your query, so I'm not sure operators would make sense in arguments.

There are different approaches as to how to model the sequlized queries, but just tackling your listed examples something like this would be possible I think.

// First example
{
  case(caseStatus: ["closed", "open"]) {
    caseNumber
  }
  // For "OR" operator, just query again with different argument
  case(caseNumber: "2012") {
    caseNumber
  }
{

// Second example
{
  // We could create another type that can query the categories
  // and have a field that queries the specific cases in those categories
  caseCategories(caseId: [4, 5]) {
    case(caseNumber: "2", caseTitle: "3") {
    }
  }

  case(caseNumber: "someVal") {
    caseNumber
  }
  case(caseStatus: "someOtherVal") {
    caseNumber
  }
}

[helixether]

Hey @asiandrummer thanks for the response. With the following, the problem is that there would be two queries done to the database when I'd like just one done.

{
  case(caseStatus: ["closed", "open"]) {
    caseNumber
  }
  // For "OR" operator, just query again with different argument
  case(caseNumber: "2012") {
    caseNumber
  }
}

With the second example, that's exactly how I see graphql, but the issue here is that the caseCategories query is contextual; I'd have to define it so that the schema knows it is being queried for categories. Whereas I need all fields on the entity (case) searchable and don't know which ones will be searched against and with what operators.

In addition, there are fields that are arrays for example userList, and if I assume that when an array is passed as a value it is an or query, then I'd never be able to do an exact value match for userList.

I was playing around with the idea of having operators likeor and and and that they'd be queryable like so

case(caseStatus:'or('caseStatus', '123')

The or being a graphql schema that resolves a string. The issue here is that I won't be able to do composites ((caseStatus = something && caseTitle = something) || (type = something && name = something)).

@leebyron I'd love to hear what you think too.

@leebyron @asiandrummer And I wonder, did you guys come up with this problem in facebook or the newsfeed? If so, how was it tackled?

[leebyron]

GraphQL doesn't have support for AND/OR operators for exactly the reason you mentioned at the top of your issue: it does not map to any kind of storage. I think to properly understand why it does not have these requires a shift in mental models. We think of GraphQL as more similar to RPC languages than database languages.

To help understand the mental model shift, it's useful to think about fields and arguments like function calls in programming languages. So suppose we ask a similar question of javascript: Why can't you type: getCaseByStatus("open" OR "closed")? The answer, I would assume, is because JavaScript does not know what OR means in this context. Should the values "open" and "closed" be combined in some way to produce another value? Should the execution of the function getCaseByStatus change in some way? JavaScript can't be sure - because the concept of field-based filtering is not part of JavaScript's semantics.

However, what if we wanted it to be? What if we wanted to write a JavaScript program that modeled field-based filtering so we could use it in our programs? We would need to design an API! For example, perhaps we would write getCaseByStatus(["open", "closed"]) or if we were concerned about ambiguity between AND and OR we might make a slightly more complex API in exchange for the ability to model more kinds of calls. Perhaps we can borrow from sequelize as an example, and make explicit operators like getCaseByStatus({ or: [ "open", "closed" ]}).

In GraphQL, like in programming languages, the arguments provided to a field do not have semantic meaning. An argument called caseStatus on a field case might coincidentally correspond to a field called caseStatus on the Case type returned, however from the GraphQL engine's point of view, it is only that: coincidence. This is important because it leaves GraphQL to be flexible enough to support any kind of API that may best represent your data.

Another aspect is how APIs are often used. Since GraphQL is designed to fit the networked boundary, it typically models product domain rather than database domain. So in practice, we find it quite rare that we want to represent something like an AND/OR on a selection. So for example there isn't a single AND/OR style field argument in the entirety of Facebook's news feed API.

At Facebook, when we encounter a case like this we try to consider it from the client's point of view and design an API which is easy and flexible from the client's point of view - written in terms of the product domain. Typically we use lists of values to represent multiple filters or multiple selections.

[leebyron]

I'm going to close the issue since it's a question answered - but feel free to continue the conversation.

[sudheerj]

Is there any way/workaround to workout with operators(AND,OR,NOT,IN etc)

For example,
for example,
"filters": {
"region": "APAC",
"code": {
"$in": [ "HK1", "SG1" ]
},
"account_number": {
"$nin": [ "1" ]
}

[dlindenkreuz]

@sudheerj You might want to check out the Graphcool docs for some inspiration on how to design such a filter API with boolean operator support. However, this won't tell you how to implement that on the server side.

[andrewmclagan]

I believe a really well documented and clearer example of the approaches to this issue as Lee Byron outlined can be gained from reading the Github GraphQL API documentation.

It really does require a rethink of your actually approach to your API query language. Essentially it should not be fetching data as your would fetch data from your storage engine: MySQL, MongoDB, Elasticsearch ... whatever.... this is the domain of your data fetching layer - not the domain of your user/developer facing API language.

Looking at the Issue object in the Github GraphQL docs you can see that each field has arguments that denote the formation of the API query language: { after, before, first, last, orderBy, since, }

[Aaron3]

@andrewmclagan passing args works for simple search, but a complex search that can search hundreds of different fields of different types and combine with different AND / OR isn't covered by the simple args.

Doing a search for X==123 AND (Y=abc OR Y=def) is how a person building a search query thinks, it's not part of the data fetching layer. The data fetching layer doesn't know that Y=abc and Y=def are both considered equivalent by the user. Furthermore, the user does not know if Y is a simple column or a complex query (LEFTJOIN Y_TABLE on...) which the data fetching layer would know.

Our specific use cases involves searching across 2 different types of databases, 200 different static dimensions, a dozen complex dimensions (which have logic of their own either pre/post processing or during data fetching), different types and different ways of comparison.

The user does not need to know that there are different databases and that some dimensions are columns and some are aliases. The data layer does not need to know about the use-case of the user for a general query or the specific function of any dimension.

Things the user doesn't need to know, the externally exposed API query language doesn't need to know. This is not to say that all apis need to expose this much information but very good use-cases exist where there is no other option.

[AviIhej]

I still need to address case sensitivity for my graphql resolvers. Please help if you have a solution. I am usng prisma, and there is no built in case sensitivity check for search queries.

[smolinari]

@AviIhej - and there never will be. Not in GraphQL. You have to "design" case sensitivity into your own API.

Scott

[channell]

I came to this topic looking to see how GraphQL could be used for complex queries where the predicate is applied to relationships that are not part of the result graph (e.g. you might want to query for customers that are payment delinquent or contact unreachable): As developers, this suggest a need for Boolean conjunct predicates (AND,OR), but there is an alternative.

If you consider the complex case as that of [1] projecting from the universe of data (applying predicates) and then [2] filtering the projected graph (in the example above delinquent customers are a sub-set of customers; there is an alternative approach:
[a] Introduce the notion of 'set', that is recursive (it has a child that us 'set'}
[b] 'set' has every type entity as a child
[c] filters can be applied to 'set', but 'set' is not something that would be needed in the result
[d] 'set' can express a syntax tree without in a lisp like
{
set: [account (status: overdrawn), set: { address (unreachable : true) investigation (count : 0}]
... normal graphQL query
}
that can be translated to to a SQL query like "..WHERE (account.status = 'overdrawn OR (address.unreachable = true AND investigation.count = 0))

[StingyJack]

I came to this topic looking to see how GraphQL could be used for complex queries...

@channell - dont let the name "GraphQL" mislead you, its not a data query language. If you need a data querying language, there's nothing wrong with the well known and understood standard SQL.