Uwazi: Trigger reindex only on relevant actions when saving a template (13 pts)

[konzz]

As users we don't want to reindex all documents of a template with any change because it's a waste of time and resources.

Changes to a template that SHOULD NOT trigger a reindex:

• Save without changes 👍🏾
• Reordering of properties
• Rename the template 👍🏾
• Changing the color of the template 👍🏾
• Only changing the following values of any property:
  
  
  
  • Hide label 👍🏾
  
  
  • Require property 👍🏾
  
  
  • Show in card 👍🏾
  
  
  • Use as filer 👍🏾
  
  
  • Default filter 👍🏾
  
  
  • Change the style type for image and preview fields 👍🏾
  
  
  • Change the full width type for image and preview field 👍🏾
  
  

Changes to a template that SHOULD trigger a reindex:

• Renaming any property
• Deleting a property
• Adding any property (except preview but could be ignore for this development)
• Changing the "select list" of a Select or a Multiselect property
• Chaning the "relationship type" or the entities (content) of a relationship field
• Inheriting a property
• Changing the inherited property

By default there should be a reindex.

Issue reported on #2783

[grafitto]

Also, I think there should not be a reindex when a new template is created. Am I right? @konzz @RafaPolit

[txau]

@grafitto that particular case is not very relevant since upon template creation there are no entities and the reindex will be super quick. So just do it as it serves better for your current flow but do not condition your solution to this case.

[grafitto]

I have 2 Ideas on how we can go about this:

1. Have a flag for each item (property) on the form which indicates whether that property has been changes, We then send the flags to the server alongside the the template details, for example have something like :-

{
    // template details
    ...
    reindexMetadata: {
        color: true, // true if color has been changes, false/undefined otherwise
        ...
        properties: [
        {
            label:  true,
            ...
        }
        ]
    }
}

with reindexMetadata the server will be able to know whether to reindex or not.

1. Do everything on the server, just after the server receives the template, it first fetches the old template from the DB, compares and comes up with a list of changed properties, then matches them against a predefined list of properties that should trigger reindexing, if any of the changed property matches then a reindex is done.

(I prefer the second one because I think it would be easier to build on it in future and also maintain)

What do you think? @txau @konzz @RafaPolit @fnocetti

[txau]

@grafitto second options seems more robust to me. It can be fine-tuned to track whether a change has to trigger the re-index or not. I don't see an automatic comparison here, like a diff, but rather a detailed checklist.

It also makes sense to only check for changes that need to trigger the re-index, at the first condition found there is no need to keep checking since the full re-index needs to be performed.

@konzz I have my doubts regarding the "delete a property" condition. Didn't we already implement re-index only if there was a name collision when creating/renaming another property? So:

1. Template TA has property PA
2. User deletes PA and saves (at this point re-index is not really needed)
3. User creates a new property also named PA or renames an existing property to PA. Since there is a naming/type collision re-index is needed at this stage.

I understand for the sake of simplicity to just do the re-index upon deletion, but since they are expensive maybe we want this granularity?

[grafitto]

@grafitto second options seems more robust to me. It can be fine-tuned to track whether a change has to trigger the re-index or not. I don't see an automatic comparison here, like a diff, but rather a detailed checklist.

It also makes sense to only check for changes that need to trigger the re-index, at the first condition found there is no need to keep checking since the full re-index needs to be performed.

Okay. working on it

[RafaPolit]

@konzz I have my doubts regarding the "delete a property" condition. Didn't we already implement re-index only if there was a name collision when creating/renaming another property? So:

I don't know if there is a confusion here. We have two types of reindex. A full-site reindex (that is only triggered when mappings change) and we have a reindex per-template: entities that belong to a template. Those entities should be reindexed upon deletion, upon many changes. Still, if you just flag for "show in cards" that shouldn't trigger the template-based reindex.

[txau]

@RafaPolit yeah, I'm well aware of that. AFAIK you can ignore a property deletion from a template and you don't need to re-index UNLESS you try to create that property again with different mapping type.

[RafaPolit]

No, I don't think so. Because those results will still show in full text searches, for example. You need to reindex after deletion.

[txau]

@RafaPolit Ok, for the sake of simplicity we can reindex after deletion.

[grafitto]

Should there be reindexing when a common property Priority sorting has been changed?

[txau]

@grafitto it doesn't need to reindex when priority sorting changes.