Absinthe: Use compile-time functions or module attributes in schemas

[sheharyarn]

Environment

Elixir: v1.9.2 (compiled with Erlang/OTP 20)
Absinthe: v1.5.1

Overview

When using module attributes or functions in enum, the code fails to compile. This bug was previously recorded in #448 and was marked resolved in master and closed, but after upgrading to Absinthe 1.5 still doesn't work.

This is different from #945 where functions could previously be called in the value/2 attributes in enum's do block.

Relevant Code

Examples of code that cause the compilation to fail:

enum :post_type, values: MyApp.Post.Type.__enum_map__()

@post_types [:abc, :def, :ghi]
enum :post_type, values: @post_types

Expected behavior

For the application to compile and work correctly.

Actual behavior

The code fails with the following error (when used with the module attribute)

== Compilation error in file lib/my_app_gql/schema/post_schema.ex ==
** (Protocol.UndefinedError) protocol Enumerable not implemented for {{:., [], [Module, :__get_attribute__]}, [], [MyApp.GraphQL.Schema.PostSchema, :post_types, 21]} of type Tuple
    (elixir) lib/enum.ex:1: Enumerable.impl_for!/1
    (elixir) lib/enum.ex:141: Enumerable.reduce/3
    (elixir) lib/enum.ex:3023: Enum.map/2
    (elixir) lib/keyword.ex:873: Keyword.update/4
    expanding macro: Absinthe.Schema.Notation.enum/2
    lib/my_app_graphql/schema/post_schema.ex:21: MyApp.GraphQL.Schema.PostSchema (module)
    (elixir) lib/kernel/parallel_compiler.ex:229: anonymous fn/4 in Kernel.ParallelCompiler.spawn_workers/7

Previous Workarounds

Before Absinthe 1.5, the workarounds mentioned in #448 would allow getting around the problem, but now they don't work either:

post_type_enum =
  quote do
    enum :post_type, values: unquote(MyApp.Post.Type.__enum_map__())
  end

Macro.expand_once(post_type_enum, __ENV__)

[benvp]

I just had the same issue using a macro defined directly inside the type module but if I extract it into it's own module and define the macro there it does compile properly. You can surely make the macro dynamic, but I'm not really experiences with this 😉 and it fits my needs.

defmodule MyApp.GraphQL.Schema.Macros do
  defmacro currency_enum do
    quote do
      enum :currency, values: unquote(Money.Currency.known_current_currencies())
    end
  end
end

defmodule MyApp.GraphQL.Schema.Types do
  require MyApp.GraphQL.Schema.Macros

  MyApp.GraphQL.Schema.Macros.currency_enum()

  # ...other type definitions...
end

[sheharyarn]

Wanted to confirm @benvp's recommendation. An absinthe codebase where I had defined it within the same module did not work, but works in a different project where I was importing a custom helper from another module:

defmodule Hutti.GraphQL.Helpers do

  @doc "Generate an Absinthe Notation Enum from an EctoEnum definition"
  defmacro enum_from_ecto(name, module) when is_atom(name) do
    name   = Macro.expand(name,   __CALLER__)
    module = Macro.expand(module, __CALLER__)
    values = module.__enum_map__()

    quote do
      enum(unquote(name), values: unquote(values))
    end
  end

  # ...
end

Usage:

enum_from_ecto(:post_type, MyApp.Post.Type)

[hickscorp]

@sheharyarn it's a really nice use of macro. However, when we try to use it, we get a compiler warning from the file calling it:

{
    "resource": "..../schema/notifier/types.ex",
    "owner": "_generated_diagnostic_collection_name_#0",
    "severity": 4,
    "message": "this clause cannot match because a previous clause at line 1 always matches",
    "source": "Elixir",
    "startLineNumber": 1,
    "startColumn": 1,
    "endLineNumber": 1,
    "endColumn": 41
}

[sheharyarn]

when we try to use it, we get a compiler warning from the file calling it

Looks like you have (at least) two clauses of the same macro/function defined and the second one can never match. https://elixirforum.com/t/help-in-understanding-code-that-always-matches/11812

[youroff]

Is there a good reason to capture values as AST at all in this case? But anyways, here's our dirty but compact workaround:

  Code.eval_string """
    enum :role, values: #{inspect RoleEnum.__enum_map__()}
  """, [], __ENV__

[dylan-chong]

I'm actually going to have a go at making a PR to get the old 1.4 behaviour back (Being able to evaluate the functions at compile time)

@benwilson512 Do you remember why you decided to with the hydrate solution in 1.5, removing eval capabilities?

[dylan-chong]

I've had a go at making 1.5 eval code. My solution was to disable the explicit raises in notation.ex when a value is a tuple (e.g. a function call AST), marking it with metadata, and in compile.ex in build_types, walk the AST and Code.eval_quoted any marked nodes can get expanded out. Can elaborate more on this later if needed

The problem with the approach is that compile.ex puts the resulting AST in a module called YourSchema.Compiled. This means that any private functions, aliases, imported functions and module attributes won't work. 1.4 worked fine because it put the AST in the YourSchema module.

@benwilson512 Thoughts?

[benwilson512]

@dylan-chong another option would be to use {:unquote, [], [val]} more liberally so that the unquote ends up happening in the Blueprint module.

[dylan-chong]

Oh that's a neat trick! Any thoughts on the YourSchema.Compiled comment?

[benwilson512]

@dylan-chong If you use {:unquote it won't be an issue because it gets unquoted inside the __absinthe_blueprint__ function which remains inside the YourSchema module instead of the YourSchema.Compiled module.

[dylan-chong]

Holy crap you're right! That's magical! 🙏
Tests for local function calls and module attributes are working! Amazing!
The test file is not compiling for this:

value :module_function_call, as: -1, description: Absinthe.Type.EnumTest.hello("Bobby")

the error is

== Compilation error in file test/absinthe/type/enum_test.exs ==
** (UndefinedFunctionError) function Absinthe.Type.EnumTest.hello/1 is undefined (function not available)
    Absinthe.Type.EnumTest.hello("Bobby")
    test/absinthe/type/enum_test.exs:71: Absinthe.Type.EnumTest.TestSchema.__absinthe_blueprint__/0
    (absinthe 1.5.3) lib/absinthe/schema.ex:359: Absinthe.Schema.__after_compile__/2
    (stdlib 3.12.1) lists.erl:1263: :lists.foldl/3
    test/absinthe/type/enum_test.exs:17: (module)
    (stdlib 3.12.1) erl_eval.erl:680: :erl_eval.do_apply/6
    (elixir 1.11.0-rc.0) lib/kernel/parallel_compiler.ex:416: Kernel.ParallelCompiler.require_file/2
    (elixir 1.11.0-rc.0) lib/kernel/parallel_compiler.ex:316: anonymous fn/4 in Kernel.ParallelCompiler.spawn_workers/7
bash returned exit code 127

i have definitely defined this in the EnumTest module

  def hello(name) do
    "Hello #{name}"
  end

Is this because the nested module (TestSchema) is compiled before the outer module (EnumTest)?
Do we need some sort of Code.ensure_compiled? check somewhere?
That sounds iffy

[benwilson512]

@dylan-chong Yes that is why, and no that can't be avoided. All functions called within the schema must exist in modules outside the schema, and outside any modules that wrap the schema.

[dylan-chong]

Interesting. Thanks, I'll try and put up a PR in the next couple weeks with some tests too

[binaryseed]

Hi folks, here's an example of dynamic values using hydrate:

https://github.com/absinthe-graphql/absinthe/blob/master/test/absinthe/schema/hydrate_dynamic_values_test.exs

Hopefully this addresses the various needs in this thread...

[dylan-chong]

Thanks for replying @binaryseed

the hydrate function doesn't suit our needs. We have a thousand+ fields that look like the following

  object :order do
    field(:id, non_null(:string), description: Descriptions.external_id_from_provider("order"))
    field(:provider, non_null(:string), description: Descriptions.provider("order"))
    field(:profile_id, :uuid,
      description: Descriptions.profile_id("order") <> @profile_id_description
    )
    field(:sid, :uuid, description: "Deprecated. Please use `profile_id`.")

This would be incredibly tedious to do for all our fields. Not to mention, hydrate is not the most convenient api

We do need to keep the values inline, otherwise we're going to have to stay with Absinthe 1.4

Re this:

Thanks, I'll try and put up a PR in the next couple weeks with some tests too

I've had to push it back, but still keen to get around to it at some point (unless someone beats me to it)

[dylan-chong]

PR #1005 for fixing the descriptions part of the problem (Fixes 95% of the function evaluation problems, but not enum values just yet). Will continue with those in the future

[dylan-chong]

As per the comments in #1005, the descriptions are now evaluated, but the following are not:

• enum values (as a list)
• enum value (singular values)
• field/arg default_value

i've had a go at evaluating enum values. here is the diff so far

diff --git a/lib/absinthe/schema/notation.ex b/lib/absinthe/schema/notation.ex
index 120d922a..cc702baa 100644
--- a/lib/absinthe/schema/notation.ex
+++ b/lib/absinthe/schema/notation.ex
@@ -1463,22 +1463,18 @@ defmodule Absinthe.Schema.Notation do
   end

   def handle_enum_value_attrs(identifier, raw_attrs, env) do
-    value =
-      case Keyword.get(raw_attrs, :as, identifier) do
-        value when is_tuple(value) ->
-          raise Absinthe.Schema.Notation.Error,
-                "Invalid Enum value for #{inspect(identifier)}. " <>
-                  "Must be a literal term, dynamic values must use `hydrate`"
-
-        value ->
-          value
-      end
+    value = wrap_in_unquote(Keyword.get(raw_attrs, :as, identifier))

     raw_attrs
     |> expand_ast(env)
     |> Keyword.put(:identifier, identifier)
     |> Keyword.put(:value, value)
-    |> Keyword.put_new(:name, String.upcase(to_string(identifier)))
+    |> Keyword.put_new_lazy(:name, fn ->
+      quote do
+        unquote(wrap_in_unquote(identifier))
+        |> String.upcase()
+      end
+    end)
     |> Keyword.delete(:as)
     |> Keyword.update(:description, nil, &wrap_in_unquote/1)
     |> handle_deprecate
diff --git a/test/absinthe/type/enum_test.exs b/test/absinthe/type/enum_test.exs
index 98dc8a01..c60e42bd 100644
--- a/test/absinthe/type/enum_test.exs
+++ b/test/absinthe/type/enum_test.exs
@@ -127,4 +127,16 @@ defmodule Absinthe.Type.EnumTest do
       end
     end)
   end
+
+  describe "enum values (list) macro evaluation" do
+    Absinthe.Fixtures.FunctionEvaluationHelpers.function_evaluation_test_params()
+    |> Enum.each(fn %{
+                      test_label: test_label,
+                      expected_value: expected_value
+                    } ->
+      test "for #{test_label} (evaluates only value to '#{expected_value}')" do
+        type = Enums.TestSchemaValues.__absinthe_type__(unquote(test_label))
+        assert type.values == [unquote(expected_value)]
+      end
+    end)
 end
diff --git a/test/support/fixtures/enums.ex b/test/support/fixtures/enums.ex
index d4d30836..ef8dfdd6 100644
--- a/test/support/fixtures/enums.ex
+++ b/test/support/fixtures/enums.ex
@@ -182,4 +182,31 @@ defmodule Absinthe.Fixtures.Enums do
       description "hello #{@module_attribute}"
     end
   end
+
+  defmodule TestSchemaValues do
+    use Absinthe.Schema
+    @module_attribute "goodbye"
+
+    defmodule NestedModule do
+      def nested_function(arg1) do
+        arg1
+      end
+    end
+
+    query do
+    end
+
+    def test_function(arg1) do
+      arg1
+    end
+
+    enum :normal_string, values: ["string"]
+    enum :local_function_call, values: [test_function("red")]
+    enum :function_call_using_absolute_path_to_current_module, values: [Absinthe.Fixtures.Enums.TestSchemaDescriptionMacro.test_function("red")]
+    enum :standard_library_function, values: [String.replace("red", "e", "a")]
+    enum :function_in_nested_module, values: [NestedModule.nested_function("hello")]
+    enum :external_module_function_call, values: [Absinthe.Fixtures.FunctionEvaluationHelpers.external_function("hello")]
+    enum :module_attribute_string_concat, values: ["hello " <> @module_attribute]
+    enum :interpolation_of_module_attribute, values: ["hello #{@module_attribute}"]
+  end
 end

however there is compile time validation that runs before the values can be evaluated. so this is going to require some architectural changes, which is beyond me.

== Compilation error in file lib/absinthe/schema/prototype.ex ==
** (FunctionClauseError) no function clause matching in Regex.run/3

    The following arguments were given to Regex.run/3:

        # 1
        ~r/[_A-Za-z][_0-9A-Za-z]*/

        # 2
        {:|>, [context: Absinthe.Schema.Notation, import: Kernel], [:input_field_definition, {{:., [], [{:__aliases__, [alias: false], [:String]}, :upcase]}, [], []}]}

        # 3
        []

    Attempted function clauses (showing 1 out of 1):

        def run(%Regex{} = regex, string, options) when is_binary(string)

    (elixir 1.11.0-rc.0) lib/regex.ex:315: Regex.run/3
    lib/absinthe/phase/schema/validation/names_must_be_valid.ex:34: Absinthe.Phase.Schema.Validation.NamesMustBeValid.valid_name?/1
    lib/absinthe/phase/schema/validation/names_must_be_valid.ex:20: Absinthe.Phase.Schema.Validation.NamesMustBeValid.validate_names/1
    lib/absinthe/blueprint/transform.ex:16: anonymous fn/3 in Absinthe.Blueprint.Transform.prewalk/2
    lib/absinthe/blueprint/transform.ex:109: Absinthe.Blueprint.Transform.walk/4
    (elixir 1.11.0-rc.0) lib/enum.ex:1521: Enum."-map_reduce/3-lists^mapfoldl/2-0-"/3
    lib/absinthe/blueprint/transform.ex:145: anonymous fn/4 in Absinthe.Blueprint.Transform.walk_children/5
    (elixir 1.11.0-rc.0) lib/enum.ex:2185: Enum."-reduce/3-lists^foldl/2-0-"/3

[dylan-chong]

@benwilson512 @binaryseed Is enum value evaluation something that you'd plan to do in the future?

[benwilson512]

@dylan-chong yes, I hope to have something out this week addressing it.