Marshmallow: Support Enum for Select-field

[floqqi]

I have got some use cases where I use a Enum in my models.

Example:

# definitions.py
from enum import Enum

class Gender(Enum):
    male = 'm'
    female = 'f'

# models.py
from definitions import Gender

class Person:
    def __init__(self, gender: Gender):
        self.gender = gender

Now it would be great if I could use the Enum in marshmallow.fields.Select:

# schemas.py
from marshmallow import fields, Schema
from definitions import Gender
from models import Person

class PersonSchema(Schema):
    gender = fields.Select(Gender)

    @staticmethod
    def make_object(data) -> Person:
        return Person(**data)

For backwards-compatibility enum34 could be used.

[floqqi]

If you think this might be a good idea I could provide a pull request if I find some time over the weekend.

[floqqi]

Ah, I see it's deprecated. Shouldn't then marshmallow.validate.OneOf support Enum?

[sloria]

@floqqi I would accept a PR adding Enum support to OneOf.

I'd rather not add enum34 as a dependency. We can just support Enum if it is available in the user's Python environment:

HAS_ENUM = False  # Check this in OneOf
try:
    import enum
except ImportError:
    pass
else:
    HAS_ENUM = True

[sloria]

Another option is to add a separate Enum validator. That way, we can avoid type-checking in OneOf.

[ctolsen]

I've been using an Enum field like this:

class Enum(fields.Field):
    """Validates against a given set of enumerated values."""
    def __init__(self, enum, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.enum = enum
        self.validators.insert(0, OneOf([v.value for v in self.enum]))

    def _serialize(self, value, attr, obj):
        return self.enum(value).value

    def _deserialize(self, value):
        return self.enum(value)

    def _validate(self, value):
        if type(value) is self.enum:
            super()._validate(value.value)
        else:
            super()._validate(value)

Feel free to grab some of that if it helps.

[floqqi]

I will take a look at this tomorrow. But I think Enum should be a separate field kinda like @ctolsen mentioned. Or would you prefer a validator like OneOfEnum, @sloria?

[sloria]

What advantages do you see in making it a field and not a validator?

[justanr]

I think a validator is the way to go. Maybe as a subclass of OneOf - but I've not looked at that particular code so I'm not sure if it'd work gracefully or not.

However, with a field it'd be possible to serialize/deserislize, which could very beneficial.

[justanr]

Was thinking about this today as I have a use case for it (but sadly with only DRF), but I think it might be better as an extension to marshmallow instead of included in core. If I have time this weekend, I'll knock up an example.

[justanr]

@floqqi I wrote this pretty quick, but I feel it's a decent first draft of what I'd expect this sort of thing to be.. It might fit whatever use case you had in mind.

@sloria I'm wondering if there should be like a "Marshmallow Extensions" section of the docs as there's been some nifty things coming out Issues. Not to toot my own horn, but there's been the DRF Compat library and the neat Polyfield as well.

[sloria]

@justanr Yes, I've been meaning to add an "Ecosystem" page for a while. I think the GitHub wiki would be a good place for this. I'll try to get to this soon.

[sloria]

@justanr 's marshmallow_enum solves the OP's use case, so closing this. I've added marshmallow_enum to the new Ecosystem wiki page: https://github.com/marshmallow-code/marshmallow/wiki/Ecosystem

[tispratik]

Is marshmallow_enum available with MM3 version compatibility?

[justanr]

@tispratik File an issue on the marshmallow_enum tracker if you're seeing any incompatibility and we'll get it sorted.

[metheoryt]

I've finished with own implementation, feel free to use

class EnumField(field.Field):

    def __init__(self, enumtype, use_name=False, as_string=False, *args, **kwargs):
        """
        :param enumtype: the enum.Enum (or enum.IntEnum) subclass
        :param use_name: use enum's property name instead of value when serialize 
        :param as_string: serialize value as string
        """
        super(EnumField, self).__init__(*args, **kwargs)
        if not isinstance(enumtype, (enum.Enum, enum.IntEnum)):
            raise ValidationError('Expect enum type, got {} instead'.format(enumtype.__class__.__name__))
        self._enumtype = enumtype
        self.use_name = use_name
        self.as_string = as_string

    def _serialize(self, value, attr, obj):
        if value is not None:
            if self.use_name:
                return value.name
            elif self.as_string:
                return str(value.value)
            else:
                return value.value

    def _deserialize(self, value, attr, data):
        try:
            if self.use_name:
                return self._enumtype[value]
            elif issubclass(self._enumtype, IntEnum):
                return self._enumtype(int(value))
            elif issubclass(self._enumtype, Enum):
                return self._enumtype(value)
        except Exception:
            raise ValidationError('does not fit to any variant')

[shulcsm]

Any reason for this to not be in tree?

[deckar01]

The Enum class is part of the standard library in python 3. If the intention is to map a string to an Enum instance, it would make sense for this to be a field rather than just a validator. We could probably support the Enum interface without needing to import enum to preserve compatibility with python 2. A new field doesn't have to be part of a major release. It could be added in any minor version.

https://docs.python.org/3/library/enum.html

[sloria]

I'm open to it, but we're more focused on stabilizing the API for the final 3.0 release than we are with new features at the moment. We may revisit this after 3.0 is released.

[justanr]

@deckar01 Supporting the read from and to interface of Enum can be supported without importing the module.

If this is of interest to have in marshmallow itself, I'm happy to contribute the code over. We can reconvene this when y'all are comfortable taking it on and figure out a migration path for it.

[leonliu]

I've finished with own implementation, feel free to use

class EnumField(field.Field):

    def __init__(self, enumtype, use_name=False, as_string=False, *args, **kwargs):
        """
        :param enumtype: the enum.Enum (or enum.IntEnum) subclass
        :param use_name: use enum's property name instead of value when serialize 
        :param as_string: serialize value as string
        """
        super(EnumField, self).__init__(*args, **kwargs)
        if not isinstance(enumtype, (enum.Enum, enum.IntEnum)):
            raise ValidationError('Expect enum type, got {} instead'.format(enumtype.__class__.__name__))
        self._enumtype = enumtype
        self.use_name = use_name
        self.as_string = as_string

    def _serialize(self, value, attr, obj):
        if value is not None:
            if self.use_name:
                return value.name
            elif self.as_string:
                return str(value.value)
            else:
                return value.value

    def _deserialize(self, value, attr, data):
        try:
            if self.use_name:
                return self._enumtype[value]
            elif issubclass(self._enumtype, IntEnum):
                return self._enumtype(int(value))
            elif issubclass(self._enumtype, Enum):
                return self._enumtype(value)
        except Exception:
            raise ValidationError('does not fit to any variant')

I think you should use issubclass() instead of isinstance()

[LukeMarlin]

Is there any foreseeable movement regarding this as 3.0 is now out?
I'm in the case where I have an Enum and I'd like to reuse it for a Schema field validation. I could use OneOf with a comprehension list, but directly passing it an Enum, or using a field.Enum would definitely look nicer.
For the time being I'll probably use marshmallow_enum as suggested, I just thought that this might be worth re-discussing , especially since @justanr already has a working codebase

[deckar01]

The issue description shows defining custom values, which I assume are expected to be used in the serialized format, but none of the implementations in this thread take that into account. They all treat the symbol name as the value. This could also be implemented as a validator that leaves the string value or instead of a field that uses the enum member internally. All combinations of these use cases are valid, but the ambiguity might be an argument not to put this in the core.

If you just want the enum symbol names to be validated, you can use the enum's member dict directly.

fields.String(validate=OneOf(Color.__members__))

https://docs.python.org/3/library/enum.html#iteration

[LukeMarlin]

It seems that marshmallow_enum allows both behaviours (according to it's README), either the member name, or the values.
I didn't use it yet so I cannot tell if this is practical or not.

[deckar01]

You are correct. Looking at the code examples closer, use_name/as_string also handle this. Sorry for the confusion.

[hf-krechan]

Hello together,

is there some progress to implement an Enum field into the marshmallow core?
I came a long way from jsons which got me into some issues in the interaction between my objects and sqlalchemy.
So I restartet my json serialisation journey with marshmallow and so far everything worked like a charm. Thanks for that =)
But when I had an enumeration attribute in my class, I got some issues.
With the validator OneOf I can check, that the string is part of the enumeration set. But I lose the information, that this attribute is part of an enumeration after the deserialisation.
So it would be great, if the functionality of marshmallow-enum package would be part of marshmallow itself.

Here is a minimal working example to show what I mean:

import attr
from enum import Enum

from marshmallow import Schema, fields, post_load
from marshmallow_enum import EnumField

# ---------------------------------------------
# create enum
material_dict = {"wood": "wood", "glas": "glas", "aluminium": "aluminium"}

Material = Enum("Material", material_dict)


# ---------------------------------------------
# create Desk class


@attr.s()
class Desk:
    height: float = attr.ib()
    material: Material = attr.ib()


# ---------------------------------------------
# WITHOUT EnumField


class DeskSchema(Schema):
    height: float = fields.Float()
    material: Material = fields.Str()

    @post_load
    def make_Desk(self, data, **kwargs) -> Desk:
        return Desk(**data)


my_desk = Desk(material=Material.glas, height=82.5)
print(my_desk)

schema = DeskSchema()
json_string = schema.dumps(my_desk)
print(json_string)

my_desk_deserialised = schema.loads(json_string)
print(my_desk_deserialised)

# ---------------------------------------------
# WITH EnumField


class DeskSchemaWithEnum(Schema):
    height: float = fields.Float()
    material: Material = EnumField(Material)

    @post_load
    def make_Desk(self, data, **kwargs) -> Desk:
        return Desk(**data)


schema_with_enum = DeskSchemaWithEnum()
json_string_with_enum = schema_with_enum.dumps(my_desk)
print(json_string_with_enum)

my_desk_deserialised_with_enum = schema_with_enum.loads(json_string_with_enum)
print(my_desk_deserialised_with_enum)

The output will be:

Desk(material=<Material.glas: 'glas'>, height=82.5)
{"height": 82.5, "material": "Material.glas"}
Desk(material='Material.glas', height=82.5)
{"height": 82.5, "material": "glas"}
Desk(material=<Material.glas: 'glas'>, height=82.5)

So the deserialisation with the schema_with_enum gives us exactly the same object like after the initialisation.

[lafrech]

Just to be sure I understand. Is there any blocker when using marshmallow-enum or are you saying it does the job and you'd like it in the core?

(I'd also like this in the core. Just never got the time to look into it since marshmallow 3 is out.)

[hf-krechan]

I mean the second option, it seems to do the job and I would like it in the core =)
I just started yesterday using marshmallow and marshmallow-enum. So I am not totally sure if there are some blocker or not.
Today I will go on with the implementation of marshmallow and marshmallow-enum into our project.
If there are some issues regarding to this topic, I will let you know.