Mongoengine: The unique_with attribute enforces other fields to be required

[gukoff]

BaseDocument._unique_with_indexes method looks like this:

    @classmethod
    def _unique_with_indexes(cls, namespace=""):
        """
        Find and set unique indexes
        """
        unique_indexes = []
        for field_name, field in list(cls._fields.items()):
            sparse = field.sparse
            # Generate a list of indexes needed by uniqueness constraints
            if field.unique:
                unique_fields = [field.db_field]

                # Add any unique_with fields to the back of the index spec
                if field.unique_with:
                    if isinstance(field.unique_with, str):
                        field.unique_with = [field.unique_with]

                    # Convert unique_with field names to real field names
                    unique_with = []
                    for other_name in field.unique_with:
                        parts = other_name.split('.')
                        # Lookup real name
                        parts = cls._lookup_field(parts)
                        name_parts = [part.db_field for part in parts]
                        unique_with.append('.'.join(name_parts))
                        # Unique field should be required
                        parts[-1].required = True
                        sparse = (not sparse and
                                  parts[-1].name not in cls.__dict__)
                    unique_fields += unique_with

                # Add the new index to the list
                fields = [("%s%s" % (namespace, f), pymongo.ASCENDING)
                          for f in unique_fields]
                index = {'fields': fields, 'unique': True, 'sparse': sparse}
                unique_indexes.append(index)

            if field.__class__.__name__ == "ListField":
                field = field.field

            # Grab any embedded document field unique indexes
            if (field.__class__.__name__ == "EmbeddedDocumentField" and
                    field.document_type != cls):
                field_namespace = "%s." % field_name
                doc_cls = field.document_type
                unique_indexes += doc_cls._unique_with_indexes(field_namespace)

        return unique_indexes

And I believe, this part shouldn't be so implicit:

# Unique field should be required
parts[-1].required = True

Because the following document won't work as expected, even though required=False is explicitly written:

class MongoDocument(Document):
    id1 = StringField(required=False, null=True)
    id2 = StringField(required=False, null=True, unique_with='id1')

1. Why "Unique field should be required"? Unique indexes on nullable fields work like a charm.
2. My proposal is as follows: either remove this logic completely or raise an exception if the field isn't declared as required explicitly:

if not parts[-1].required:
    raise ...

[wojcikstefan]

Agreed this is weird... I'll try to take a look at this soon.

[jenarvaezg]

Hi @wojcikstefan, any update on this?
Thanks :-)

[kimeshan]

Any update on this @wojcikstefan ?

[bptarpley]

This is a problem for my data model--just pinging to let devs know it would be nice to have fields specified in unique_with which don't require to have values...

[amiguet]

Also encountered this limitation. Any update on this? Or workaround?

[AyluinReymaer]

Hey guys,

This workaround works for me:

source is unique_with sourceId, current behavior is that sourceId is required, expected behaviour is to not be required.

class mongoengine_model(mongoengine.Document):
    source = mongoengine.StringField(unique_with="sourceId")
    sourceId = mongoengine.StringField()

mongoengine_model.sourceId.required = False

By adding the line mongoengine_model.sourceId.required = False immediately after the model is defined, the field is not required anymore and all objects are properly created even without specifying the sourceId.

One thing that could be argued is that it would break the unique_with functionality by not adding the unique indexes on the mongodb collection, however, I checked and the index is properly formed.

Hopefully this helps until this is fixed.