Uwazi: CSV import - validation error possible scenario

[RafaPolit]

Currently, CSV import does not validate required fields. There is a common scenario where a user:

• uploads a CSV with missing 'required properties'
• tries to publish those entities
• is presented with a very 'un descriptive' validation error

No clue as to what the error is. The only way to find out which are the missing fields is to go into edit mode for a single entity, and then try to save. Only then is the red 'warning' shown into input that is missing info.

We should improve thing to show, for example, in library that an entity is missing required fields, therefore unpublishable. Give better feedback as to what validation is failing. Perhaps even create a "CSV import report" where you say that some fields are missing (should we reject the import if a required field is missing?)

There are several approaches to be considered here @txau @simonfossom

[daneryl]

related #2742

this is not a bug, and not specific for csv imports but for all entity validation messages.

[RafaPolit]

But if we don't allow saving an entity without a required field, why can we 'save it' via the CSV import with a required field missing? What is the difference?

[daneryl]

@RafaPolit oh i see, we are missing the required validation on server, the important one, this is not a csv related but any entity save, it is a bug ok.

we should try to unify validations, on react app and api, if we cant then we should prioritize on api instead of the client app.

[nwyu]

One row in our csv file was missing a required field (in this case, it was the title).

The import was rejected up to this point, but the first 16 of the 34 entities still came into the unpublished side. The remaining entities beyond the one missing the required content did not.

[daneryl]

@nwyu yes this behaviour was by design, you should fix csv, remove all imported and reimport again, @RafaPolit @txau should we rethink this process so that it does not import anything if there is an error ? this will require a pre validation of all data before importing.

[nwyu]

@daneryl We have removed all that were imported, fixed the csv and re-imported again successfully.

However, as user feedback, my hunch is that it is less confusing if the erroneous csv file were to be rejected from the start, so that no entities get imported.

[RafaPolit]

I'm inclined to agree about the pre-flight check to reject the entire CSV. The problem is that it could spend half an hour on huge CSVs just diagnosing things. Maybe we could reject the CSV at the first error and only report that, but then when they upload again there could be a second and third error to fix.

I am not really clear as to which of the 'problems' is better to keep.

[txau]

@RafaPolit @daneryl Agree. We rather do a full run and provide a proper report with all the errors?

[daneryl]

dry run is being discussed in #3000
validation messages is being discussed in #2742