Apollo-feature-requests: Support for custom scalars

[Akryum]

Migrated from: apollographql/apollo-client#585

[fbartho]

It'd be great if we had support for custom scalars in Apollo Client. A pretty crucial feature would be for custom scalars to be provided by native code. apollo-link-state users would particularly benefit from this.

The canonical example is Date objects (from ISOString or Unix Timestamp "raw" formats), but it would be valuable to vend other complex objects from a local object pool in certain circumstances beyond just dates.

In the original thread there was a reference that maybe a library-provided semi-static list of common scalars might be an acceptable compromise, but immediately, people would want momentjs dates instead of JS Dates.

[bebbi]

The thumbs up total of this and OPs of issues linking here are 22+15+37=74.
Most need json parse and date formats.

[brunoreis]

I'm thinking about creating a hoc where I can pass a query, the custom scalar type and a conversion function. I would compose that hoc adding it just after the query hoc so that it would convert data injected from that query.

Can someone point me how to inspect the schema of a specific query to detect what are the paths to the fields that are of that custom scalar type?

[jedwards1211]

@bebbi why do people need to parse json types on the client side? Unlike dates, json can be serialized just fine by server-side custom scalars and will be received as json by the client.

[jedwards1211]

@brunoreis you may be able to use graphql's undocumented visitWithTypeInfo, which allows you to traverse the queries while keeping track of the type of each field.

However, it may not be ideal for use on the client because you have to construct a TypeInfo object from an entire GraphQLSchema. I don't know if it would support a partial schema containing only the bare minimum fields necessary to determine what to parse on the client.

Another option is to fetch the type metadata from the server and build a custom index to it; I did this in apollo-magic-refetch. But I had to write my own code to keep track of the types while traversing the query with visit, and there are edge cases I haven't taken care of yet.

I bet there are some hooks in the Apollo caching layer that would allow you to handle custom scalars anywhere in the schema this way, without having to wrap a bunch of your components in an HoC.

[couturecraigj]

@jedwards1211 wouldn't that defeat the purpose of a type system? Maybe somebody wants to store mixed data to graphql in certain instances. That is when they would use a JSON type. I know it is not ideal but geo-data for instance can be incrementally adopted through GeoJSON in JSON (mixed) data type that is just stored as a string. Not having to parse it each time the client receives it would be huge.

[jedwards1211]

@couturecraigj I'm not sure what you're thinking would defeat the purpose if a type system, I was suggesting that there's probably a way to make the apollo cache parse all raw values of a date type (a custom scalar in the schema that serialize to an ISO string on the wire) back into a Date object and store them as such in the cache.

[slorber]

Hi,

In addition to custom scalar types, I want to add that it is very important for me that the choosen solution works with codegen tools. If my Date scalar is serialized as String and deserialized or injected as props as JS Date on the client, it's important the codegen tools give me a JS Date type in the end and not a string. This seems complicated to solve, but important to mention.

[bebbi]

@jedwards1211 In my case for example, I'd like not having to worry about de-serialization, because this seems to create multiple redundant places where I worry about the data model.

[jedwards1211]

@bebbi I not sure I fully understand you.
Without worrying about de-serialization, any date field in your schema will be received by the client as a string or number (whichever you choose to serialize it as in the custom scalar type on the server).

The only way to convert back to a date on the client is to have de-serialization code.

Of course it would be terrible to have to write code for every single date field or every query that requests a date field -- which is why I'm suggesting that if the client has enough information about the schema to know which fields are dates, it would be possible to write a single piece of code that deserializes any date field encountered in the schema.

[bebbi]

@jedwards1211 As far as I can understand we're on the same page.
It's json in my case. I've got various queries using fragments using that json type and would like de-serialization of that type to be controlled in one single place.

[jedwards1211]

@bebbi so when your server loads the JSON it needs to send from the database or a file or whatever, is it a raw string instead of an object?

If you use graphql-type-json for the field and your resolver returns an object instead of a raw string, it will be sent to the client as an object and the client won't need to deserialize at all.

[gullitmiranda]

@jedwards1211 the https://github.com/taion/graphql-type-json works only to send json to the server, not the opposite.
what we need is to the client parse a json send by the server.

[jedwards1211]

@gullitmiranda I don't understand why that's been your experience, seems to me something must be misconfigured. Here is proof that it's able to send json to the client in my project (the jsonFieldMappings field below uses graphql-type-json). You can see that the client (GraphiQL) receives it as an object instead of a raw string:

[gullitmiranda]

interesting @jedwards1211 . was not aware of this since I don't use the graphql-js on the server side. but from what I've been researching, this was not a standard that is accepted by the default implementation of the graphql, so many libs do not support or do not intend to support this.

some conversations:

• rejection of proposal in graphql-js: https://github.com/graphql/graphql-js/pull/172
• discussion about this in the implementation of graphql to elixir (that is what I use): https://github.com/absinthe-graphql/absinthe/issues/206. by the way, in this issue the graphql-type-json is mentioned.

[jedwards1211]

even if it's not exactly standard, the GraphQL docs do say that

A standard GraphQL POST request should use the application/json content type, and include a JSON-encoded body

Regardless of the method by which the query and variables were sent, the response should be returned in the body of the request in JSON format

In most GraphQL service implementations, there is also a way to specify custom scalar types

So for built-in types, a standard GraphQL server is already serializing to and from JSON, and the most basic possible custom scalar system -- one that just hands you a JSON node representing the scalar in the request or accepts a JSON node from you to represent the variable in the response -- would only necessitate using the identity function for JSON custom scalar serialization/deserialization**.

Which is exactly what a commenter recently mentioned in the issue you linked:

scalar :raw_json do
  parse fn _, _ -> raise "not relevant" end
  serialize fn value -> value end
end

So in other words, any GraphQL server that supports custom scalars is capable of sending a field value that the client will receive as a JSON object instead of a string.

** The only caveat is that it also has to be possible to represent the value as a literal within the GraphQL query itself, not just within the JSON request/response body. But there is a clean mapping between JSON and GraphQL AST nodes, as demonstrated in graphql-type-json's parseLiteral function.

[jedwards1211]

@slorber do you have enough control over the codegen tools you use to tell them what JS type is associated with a given GraphQL custom scalar type?

[gullitmiranda]

ye @jedwards1211

Which is exactly what a commenter recently mentioned in the issue you linked:

i see and try to use in outputs and worked. don't work with input's for now, but it's no big deal.

[jedwards1211]

Cool. I don't know elixir but it looks like inputs may work if you make the parse fn return the argument instead of raising an error.

[gullitmiranda]

the parse really work, but the absinthe add errors because the typing.

  1) test create template mutation succeeds with valid params (MyAppWeb.Schema.MyContext.TemplateMutationTest)
     test/my_app_web/schema/my_context/mutations/template_mutation_test.exs:12
     match (=) failed
     code:  assert {:ok, data} = run(create_template_mutation(), variables: to_input(params), context: context)
     right: {:error,
             [
               %{
                 locations: [],
                 message: "Argument \"input\" has invalid value $input.\nIn field \"fields\": Expected type \"JSON\", found {key1: \"value\", key2: \"value2\", list1: [1, {keyInList: \"value_in_list\"}]}.\nIn field \"key1\": Unknown field.\nIn field \"key2\": Unknown field.\nIn field \"list1\": Unknown field."
               }
             ]}
     stacktrace:
       test/my_app_web/schema/my_context/mutations/template_mutation_test.exs:54: (test)

[dan-turner]

Any movement on this?

[jamiewinder]

I've noticed support for custom scalars is referenced in the GraphQL Tools docs, but as far as I can see it isn't in the pipeline for apollo-client (including 3.0, or at all). Could someone in the know confirm this?

[thomasporez-tinyclues]

This is definitely a must have ! Any news about appolo-client plans for support ?
In the meantime, did someone can point me to a workaround or something because it's so convenient !

[cmhuynh]

While a complete fix needs thoughtful architecture and discussion (from the original ticket and cloned to this), a fix for Date might be "isolated" from that, can benefit a lot.

Discussion of Date as a custom scalars can impact onboard experience to this library, like this

• Found the library, follow Basics example, it runs so smoothly on CodeSandbox
• The example work for string, simply change to Date and found it doesn't work.

Re-produce the issue at

• https://codesandbox.io/s/github/cmhuynh/apollo-link-rest/tree/test_date_parse/examples/simple
• https://github.com/cmhuynh/apollo-link-rest/blob/test_date_parse/examples/simple/src/Film.js#L26

I'd love to have Date supported, not custom type and therefore upvote this (part) the ticket.

[FredyC]

I don't get it. I went through 2 years old history discussing this and nothing viable seems to come out of it?

I wonder, when I look at the example from graphql-tools page, they are using resolvers, but we do have resolvers option when instantiating ApolloClient instance. Could it be possibly that simple to just add resolver? I am definitely going to try that in the following days.

import { makeExecutableSchema } from 'graphql-tools';
import GraphQLJSON from 'graphql-type-json';

const schemaString = `

scalar JSON

type Foo {
  aField: JSON
}

type Query {
  foo: Foo
}

`;

const resolveFunctions = {
  JSON: GraphQLJSON
};

const jsSchema = makeExecutableSchema({ typeDefs: schemaString, resolvers: resolveFunctions });

Edit: Nevermind, turns out that resolvers on ApolloClient is reserved for LocalState only :/

[FredyC]

Ok, I think I've found fairly good approach, at least to handle ISO dates. With that link, I can easily transform iso date strings to Date instances before they get to components. It is surely slightly fragile as we need to remember to add more fields when they are used. Eventually, I want to make some simple generator based on the schema to remove that barrier.

https://github.com/with-heart/apollo-link-response-resolver

    BusinessHourInterval: {
      openAt: parseISO,
      closeAt: parseISO,
    },

For the reverse direction when I am using Date for the mutation variables, it actually works out of the box and dates are converted. Not sure why such "magic" exists only in one direction, though.

As for the TypeScript, we are using graphql-code-generator and that's simple scalars config to set proper types and works very nicely.

[eturino]

I have been working on a custom ApolloLink to deal with this situation.

https://github.com/eturino/apollo-link-scalars

If you pass it a GraphQLSchema and optionally a map of parsing/serializing functions per scalar type, it will parse all scalars on the responses, and serialize all the scalars on the inputs.

It can optionally also validate enums, throwing an error if an invalid enum value is received.

By default, for each scalar, it will apply the parsing/serializing functions defined for that type in the GraphQLSchema. The functions passed in the typesMap argument take precedence.

As a small disclaimer: this is a POC. I just finished testing it on my team's app but I haven't tested performance.

import { withScalars } from "apollo-link-scalars"
import { ApolloLink } from "apollo-link";
import { HttpLink } from "apollo-link-http";
import { schema } from './my-schema'

const link = ApolloLink.from([
  withScalars({ schema }),
  new HttpLink({ uri: "http://example.org/graphql" })
]);

// we can also pass a custom map of functions. These will have priority over the GraphQLTypes parsing and serializing functions from the Schema.
const typesMap = {
  CustomScalar: {
    serialize: (parsed: CustomScalar) => parsed.toString(),
    parseValue: (raw: string | number | null): CustomScalar | null => {
      return raw ? new CustomScalar(raw) : null
    }
  }
};

const link2 = ApolloLink.from([
  withScalars({ schema, typesMap }),
  new HttpLink({ uri: "http://example.org/graphql" })
]);

//cc @FredyC

[alfechner]

There is FieldPolicy in Apollo Client 3.

It allows us to serialize/unserialize values when read/write from/to the cache.

It's on field rather than on type level. Thus one needs to configure the transformation for every field again. However it mitigates the problem of serializing the cache.

[geekox86]

@Akryum Why are you guys not engaged on this high-demand feature?!

[Akryum]

Read the comment literally before yours before writing there is no progress 😅

[geekox86]

@Akryum I have read it before, but the requested feature (custom scalars) is to have serialize/deserialize on scalar type level not field level.

I really appreciate everything that you are guys doing, but reading through years of going back and forth on this feature seems, at least to me, weird from your side, as your responsiveness on other issues has been and is usually excellent.

[geekox86]

@Akryum and I also took my words back on no plan and no progress as this was only my exaggerations, and I apologize for it.

Though, any progress on having this feature per type?

[FredyC]

I think it doesn't have a priority because there are viable solutions in userland, see https://github.com/apollographql/apollo-feature-requests/issues/2#issuecomment-538373369

[jedwards1211]

Out of curiosity, why would one ever want to define serialization on a field level? If two fields are the same type, I'm having trouble imagining a use case for serializing them differently.

Er...nevermind, I just realized I have various JSON fields throughout my schema, and different postprocessing on them could theoretically be useful.

[jedwards1211]

@FredyC FWIW apollo-link-response-resolver is now unmaintained, better to recommend apollo-link-scalars I guess

[jedwards1211]

@slorber

In addition to custom scalar types, I want to add that it is very important for me that the choosen solution works with codegen tools. If my Date scalar is serialized as String and deserialized or injected as props as JS Date on the client, it's important the codegen tools give me a JS Date type in the end and not a string. This seems complicated to solve, but important to mention.

If codegen tools don't allow you to customize this, they need improvement. To deal with stuff like this I made my own GraphQL -> Flow type generator that allows comments in the schema and documents to customize the type generation very specifically: graphql-typegen

[nateq314]

There is FieldPolicy in Apollo Client 3.

It allows us to serialize/unserialize values when read/write from/to the cache.

It's on field rather than on type level. Thus one needs to configure the transformation for every field again. However it mitigates the problem of serializing the cache.

For reference, the documentation for the FieldPolicy feature that @alfechner mentioned above is here: https://www.apollographql.com/docs/react/v3.0-beta/caching/cache-field-behavior/

[pluqueTheLuxe]

Read the comment literally before yours before writing there is no progress 😅

@Akryum, I think @geekox86 meant that this feature request is in the apollo-feature-requests backlog since 2018 and in the original backlog since 2016 and stands as #1 feature when you sort by "thumbs up".

Moreover, I think it's not much to say that custom scalar (especially: Date and DateTime) are frustrating to work with in GQL server/client communication

[dora-gt]

https://github.com/eturino/apollo-link-scalars
This is an amazing project. Thank you so much.

[dobrinov]

There is FieldPolicy in Apollo Client 3.

It allows us to serialize/unserialize values when read/write from/to the cache.

It's on field rather than on type level. Thus one needs to configure the transformation for every field again. However it mitigates the problem of serializing the cache.

I tried this with Apollo 3 and it works just fine but it does not handle one use case which https://github.com/eturino/apollo-link-scalars covers. If you use no-cache as fetchPolicy, the read and merge methods of the fieldPolicy won't be executed.

[coler-j]

How do you handle typescript types with FieldPolicies though? (for example, this SO ticket)

If your custom scalar maps to string but in your FieldPolicy you convert it to a DateTime then Ts will still think it is a string....

[UchihaYuki]

I think the work should be done on "@graphql-codegen/typescript-apollo-angular". It can generate code to run parsers and serializers on custom scalars (It already have enough info to do this, since it know the whole schema picture). It will be better if we can just reuse "graphql-scalars". The code on the client side shouldn't be affected much. just some ideas...

[dbbk]

I have to say it baffles me as to why this is so hard with Apollo Client... the only solution I've been able to find is using https://github.com/eturino/apollo-link-scalars, but I'm quite nervous of bundling my whole schema file in.

My use case is a custom serialize format for Date (I want to format a native JS date to a YYYY-MM-DD string).

[myknbani]

I'm not sure if it's just a low priority item, or language/reflection differences, but their Android client (Java/Kotlin) has this:

https://www.apollographql.com/docs/android/essentials/custom-scalar-types/

[jamesdh]

@myknbani likewise, the GraphQL-Java server implementation makes defining custom scalars and data mappings very easy to do.

[stephenh]

Fwiw if you want to use FetchPolicys to do this instead of the link approach (that requires having the schema at runtime), we have a graphql-code-generator plugin that will generate the "for every entity, for every field that is a custom scalar, use this merge policy" object literal to pass to apollo-client / InMemoryCache:

https://github.com/homebound-team/graphql-typescript-scalar-type-policies