Activiti: Application Data cleanup

[ffazzini]

As an admin
I want to delete application data
So that I am able to use the same application again with no history

A new admin endpoint would be called to perform this solution.

Benefits:

1. For testing, a user won't need to create a new application.

2. It would be possible to migrate old data to a different source before deleting.

[salaboy]

@ffazzini to add more details:

This will require to create endpoints in several components, such as

• Runtime Bundle
• Query Service
• Audit Service
• Application Service

It will be really nice to be able to do it in an Event Driven way, where the components receive a message saying which is the criteria to archive data.

[miguelruizdev]

Specific requirements regarding endpoints:

• DELETE
  
  
  
  • Delete all data
  
  
  • Delete data related to a process instance
  
  
  • Delete existing data before a given timestamp
  
  
• GET/EXPORT
  
  
  
  • Export all data
  
  

[miguelruizdev]

@salaboy regarding the scope of the things to delete, do we only care about process instances and standalone tasks (I suppose that tasks within processes are deleted in a cascade fashion, aren't they?)
Things like process definitions are tied to a specific RB, and not the Application itself, right?

[salaboy]

@miguelruizdev you are correct.. if you start by cleaning up processes and tasks (standalone) that will be great..
I would include some kind of filter.. for example: COMPLETED processes or tasks, ALL, etc.. you don't want to clean up running ones.. all the time.

[miguelruizdev]

@salaboy after a chat with @ryandawsonuk we came to the conclusion that we need to perform a filter regarding standalone and regular tasks when we delete them.
At the controller level, the endpoint would be meant to delete just standalone ones, since those are the only ones which can be deleted this way.

[miguelruizdev]

So far, we have performed filtered retrievals using the GetTasksPayload.
I think that we can sort this out by adding a standalone boolean field there.
What are your thoughts, @salaboy ?

[salaboy]

@miguelruizdev yes.. that sounds correct.. having a filter in GetTasksPayload makes a lot of sense.

[miguelruizdev]

Given the fact that conditional property annotation is applied at a class level, and not at the method level, we need to add the deletion endpoints to specific controllers if we want this feature to be enabled via properties.
What are your thoughts, @salaboy ?

[miguelruizdev]

Also, the fact that the deletion endpoints should return all the deleted data (in an EXPORT fashion) is a feature that may not be scalable enough when dealing with big amounts of data.
Dumping all service data can take ages, so will would face a lot of uninteded time outs...
Maybe export capabilities should be implemented via other means, and not enpoints (?)

[miguelruizdev]

Since the nature of these endpoints is to become utility endpoints to help us out when testing, the semantics of "deletion" do not apply anymore, and we should call them cleanup endpoints.
Waiting for your feedback, @salaboy

[salaboy]

@miguelruizdev I liked your suggestion of export, I think that we can call them export and add a flag to delete as optional.
My thinking is as follow:
1) We might want to migrate data from one version to another or from one pod to another. We can call the endpoint to fetch all the data
2) If the request adds a flag -> (query param) delete=true it should remove the data
3) we can add another flag called archive=true to not delete the data but just to marked as deleted

@erdemedeiros Do you have thoughts about this?

[miguelruizdev]

It seems like a reasonable way to go.
Controllers just for admins, right?

[salaboy]

@miguelruizdev yes.. admin level only and the @ConditionalOnProperty should be at the class level.. meaning that we need separate controller classes for these endpoints.

[erdemedeiros]

@salaboy @miguelruizdev, sorry guys, I've missed this comment. It sounds confusing to me to have delete as optional. If you want to retrieve entities only there is already a GET endpoint for that. And if we set delete to false then we are calling a DELETE endpoint that's acting as a GET at the end of the day. Same I would see archive as a separated endpoint (using POST probably). However it would make some sense to return the items that have been deleted as result of calling the endpoint.

[miguelruizdev]

We can discuss things over the code here:
https://github.com/Activiti/activiti-cloud-query-service/pull/196

[erdemedeiros]

Delete endpoint is available in audit and query.