Ecto: Cast virtual fields populated in select

[mbuhot]

Following from discussion on elixir forum it would be ideal if virtual fields declared in a schema are automatically cast to the declared ecto type when populated in a select query.

Environment

• Elixir version (elixir -v):

Erlang/OTP 20 [erts-9.0] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:10] [hipe] [kernel-poll:false] [dtrace]

Elixir 1.4.5

• Database and version (PostgreSQL 9.4, MongoDB 3.2, etc.):

Postgresql 9.6

• Ecto version (mix deps):

ecto 2.2.0-rc.0 (Hex package) (mix)
locked at 2.2.0-rc.0 (ecto) 412cb660

• Database adapter and version (mix deps):

postgrex 0.13.3 (Hex package) (mix)
locked at 0.13.3 (postgrex) c277cfb2

• Operating system:

Macos Sierra 10.12.3

Current behavior

Given a schema:

schema "users" do
    field :name, :string
    field :date_of_birth, :date
    field :next_birthday, :date, virtual: true
  end

Running a query that populates the next_birthday field produces an erlang date tuple, instead of a %Date{} struct like the date_of_birth field:

Repo.all(from u in User, select: %{u | next_birthday: fragment("current_date")})

[%User{__meta__: #Ecto.Schema.Metadata<:loaded, "users">,
  id: 1,
  name: "Joe",
  date_of_birth: ~D[1985-01-02], 
  next_birthday: {2017, 8, 12}
}] 

Expected behavior

The value populating the virtual field should be cast to the type declared in the schema.

Repo.all(from u in User, select: %{u | next_birthday: fragment("current_date")})

[%User{__meta__: #Ecto.Schema.Metadata<:loaded, "users">,
  id: 1,
  name: "Joe",
  date_of_birth: ~D[1985-09-12], 
  next_birthday: ~D[2017-08-12]
}] 

Which is equivalent to adding casting the value with type/2:

Repo.all(from u in User, select: %{u | next_birthday: type(fragment("current_date"), :date)})

[josevalim]

Unfortunately this is very complex to do because of some optimizations we perform. We may not be able to do this at all but we can revisit it for 3.0.

[hauleth]

You can use Ecto.Query.API.type/2 for now as a quick fix.

[1player]

Unfortunately Ecto.Query.API.type/2 is not a good fit when one wants to cast the field to a custom type.

In my case the field is generated by MySQL's GROUP_CONCAT function, which returns a comma separated string array (e.g. foo,bar,baz), which I'd like to convert to a list with a custom Ecto.Type.

I'm confident Jose will get to this eventually. 👍

[zachdaniel]

@mbuhot I just tested this on master with my own schemas, and used map update syntax to select into a virtual date field, and it was correctly cast as a datetime. Would you mind checking and see if perhaps this issue has been resolved?

[josevalim]

@zachdaniel very good point. Ecto 3.0 got rid of the tuple date types, so this issue is by definition fixed. :D

[michalmuskala]

It's fixed for date structs, just because the load callback for date structs is a no-op. It's not fixed for custom data types that implement some logic in a load callback.

[josevalim]

Correct. But I am not sure we should fix it because struct/map updates

never cast.

José Valimwww.plataformatec.com.br
http://www.plataformatec.com.br/Founder and Director of R&D

[zachdaniel]

Do we do the casting appropriately in select_merge if you are setting a source field?

[bencates]

Casting does not appear to work on select_merge:

defmodule MyApp.Example do
  use Ecto.Schema

  schema "example" do
    field :message, :string
    field :has_message, :boolean, virtual: true
  end
end

MyApp.Repo.insert!(%MyApp.Example{message: "hello, world"})

example =
  MyApp.Repo.one from e in MyApp.Example,
    select_merge: %{has_message: not is_nil(e.message)}

# Under MySQL `example.has_message` will be an integer instead of a boolean.
assert example.has_message == true # FAILS

# This is a comple error.
example =
  MyApp.Repo.one from e in MyApp.Example,
    select_merge: %{has_message: type(not is_nil(e.message), :boolean)}

# This produces a runtime error under MySQL, since it has no native boolean type.
example =
  MyApp.Repo.one from e in MyApp.Example,
    select_merge: %{has_message: type(fragment("? IS NOT NULL", e.message), :boolean)}

I don't see that there is a correct way to cast a select_merge if the underlying database doesn't directly support the desired types.

[josevalim]

@bencates what happens if you do:

  MyApp.Repo.one from e in MyApp.Example,
    select: %{has_message: not is_nil(e.message)}

What does it return? A map with an integer or a boolean? Any way, please open up a new issue.